2024-03-30T01:25:59.175 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-30T01:25:59.175 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} --name rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629316 -- /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629316/orig.config.yaml 2024-03-30T01:25:59.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T01:25:59.431 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629316 branch: wip-rishabh-testing-20240327.051042-reef description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} email: null first_in_suite: false job_id: '7629316' kernel: client: sha1: distro flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-testing-20240327.051042-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240327.051042-reef sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 8052 sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f sleep_before_teardown: 0 subset: 497/512 suite: fs suite_branch: wip-rishabh-testing-20240327.051042-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBk7UkPPupy5hguDJvEkH8EUjYZrclRc4Pzj7u0lFqnlJ0AxhsJzhyHneA6dkbFuycMKrdvxhr+4QslNGpeHyng= smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBACPKumA6RhDjM0xJarFiqwgHezSGIpTCARRsLgrPgL7MtPPobgvaUhhNBuaTuTx+ubiaFwiJHnStvdh03dHPlY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_multifs_auth teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_18:05:24 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-30T01:25:59.433 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa; will attempt to use it 2024-03-30T01:25:59.435 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks 2024-03-30T01:25:59.435 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-30T01:25:59.437 INFO:teuthology.task.internal:Checking packages... 2024-03-30T01:25:59.469 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '470100a4ccbcf7769052bf09a5413881ccf1b97f' 2024-03-30T01:25:59.469 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-30T01:25:59.469 INFO:teuthology.packaging:ref: None 2024-03-30T01:25:59.469 INFO:teuthology.packaging:tag: None 2024-03-30T01:25:59.469 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240327.051042-reef 2024-03-30T01:25:59.470 INFO:teuthology.packaging:sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T01:25:59.470 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-testing-20240327.051042-reef 2024-03-30T01:25:59.687 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1018.g470100a4 2024-03-30T01:25:59.688 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-30T01:25:59.694 INFO:teuthology.task.internal:no buildpackages task found 2024-03-30T01:25:59.694 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-30T01:25:59.744 INFO:teuthology.task.internal:Saving configuration 2024-03-30T01:25:59.761 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-30T01:25:59.767 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T01:25:59.791 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629316', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 01:21:23.805283', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBk7UkPPupy5hguDJvEkH8EUjYZrclRc4Pzj7u0lFqnlJ0AxhsJzhyHneA6dkbFuycMKrdvxhr+4QslNGpeHyng='} 2024-03-30T01:25:59.812 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629316', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 01:21:23.807002', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBACPKumA6RhDjM0xJarFiqwgHezSGIpTCARRsLgrPgL7MtPPobgvaUhhNBuaTuTx+ubiaFwiJHnStvdh03dHPlY='} 2024-03-30T01:25:59.813 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-30T01:25:59.819 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-30T01:25:59.819 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-03-30T01:25:59.819 INFO:teuthology.run_tasks:Running task console_log... 2024-03-30T01:25:59.928 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6446b62160>, signals=[15]) 2024-03-30T01:25:59.928 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-30T01:25:59.934 INFO:teuthology.task.internal:Opening connections... 2024-03-30T01:25:59.934 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-03-30T01:25:59.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T01:26:00.021 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:26:00.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T01:26:00.097 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-30T01:26:00.102 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-03-30T01:26:00.124 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-03-30T01:26:00.124 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-03-30T01:26:00.185 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Red Hat Enterprise Linux" 2024-03-30T01:26:00.185 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8.6 (Ootpa)" 2024-03-30T01:26:00.185 INFO:teuthology.orchestra.run.smithi031.stdout:ID="rhel" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="fedora" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8.6" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.redhat.com/" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:26:00.186 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-30T01:26:00.187 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-30T01:26:00.187 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-30T01:26:00.187 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-30T01:26:00.187 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-03-30T01:26:00.225 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-03-30T01:26:00.248 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-03-30T01:26:00.248 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.6 (Ootpa)" 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.6" 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-30T01:26:00.310 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2024-03-30T01:26:00.311 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-30T01:26:00.311 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T01:26:00.311 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:26:00.311 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-30T01:26:00.311 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-30T01:26:00.311 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-30T01:26:00.311 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-30T01:26:00.312 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-03-30T01:26:00.340 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-30T01:26:00.351 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-30T01:26:00.386 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-30T01:26:00.386 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-03-30T01:26:00.389 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-03-30T01:26:00.409 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-30T01:26:00.415 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-30T01:26:00.415 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-03-30T01:26:00.449 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-03-30T01:26:00.482 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-30T01:26:00.561 INFO:teuthology.run_tasks:Running task kernel... 2024-03-30T01:26:00.576 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-30T01:26:00.576 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2024-03-30T01:26:00.576 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-30T01:26:00.576 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T01:26:00.577 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T01:26:00.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T01:26:00.599 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-03-30T01:26:00.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T01:26:00.600 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-03-30T01:26:00.662 INFO:teuthology.orchestra.run.smithi059.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-30T01:26:00.662 INFO:teuthology.task.kernel:Running kernel on smithi059: 4.18.0-513.9.1.el8_9.x86_64 2024-03-30T01:26:00.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum install -y kernel 2024-03-30T01:26:00.663 INFO:teuthology.orchestra.run.smithi031.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-30T01:26:00.664 INFO:teuthology.task.kernel:Running kernel on smithi031: 4.18.0-513.9.1.el8_9.x86_64 2024-03-30T01:26:00.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-03-30T01:26:02.205 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2024-03-30T01:26:02.205 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2024-03-30T01:26:02.254 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2024-03-30T01:26:02.254 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2024-03-30T01:26:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:26:02.481 INFO:teuthology.orchestra.run.smithi031.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T01:26:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:26:02.605 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:26:02.605 INFO:teuthology.orchestra.run.smithi059.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T01:26:02.605 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:26:04.225 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2024-03-30T01:26:04.280 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-03-30T01:26:05.351 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-03-30T01:26:05.722 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-03-30T01:26:09.425 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 408 kB/s | 24 kB 00:00 2024-03-30T01:26:09.738 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 416 kB/s | 24 kB 00:00 2024-03-30T01:26:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-03-30T01:26:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-03-30T01:26:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-03-30T01:26:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-03-30T01:26:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-03-30T01:26:10.401 DEBUG:teuthology.orchestra.run.smithi031:> echo no | sudo yum reinstall kernel || true 2024-03-30T01:26:10.558 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-03-30T01:26:10.559 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-03-30T01:26:10.639 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-03-30T01:26:10.640 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-03-30T01:26:10.640 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-03-30T01:26:10.778 DEBUG:teuthology.orchestra.run.smithi059:> echo no | sudo yum reinstall kernel || true 2024-03-30T01:26:10.787 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2024-03-30T01:26:10.788 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2024-03-30T01:26:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:26:11.002 INFO:teuthology.orchestra.run.smithi031.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T01:26:11.002 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:26:11.167 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2024-03-30T01:26:11.167 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2024-03-30T01:26:11.370 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:02 ago on Sat 30 Mar 2024 01:26:09 AM UTC. 2024-03-30T01:26:11.392 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:26:11.392 INFO:teuthology.orchestra.run.smithi059.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T01:26:11.392 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:26:11.459 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-30T01:26:11.459 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-30T01:26:11.509 INFO:teuthology.orchestra.run.smithi031.stderr:Error: No packages marked for reinstall. 2024-03-30T01:26:11.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum reinstall -y kernel || true 2024-03-30T01:26:11.759 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:02 ago on Sat 30 Mar 2024 01:26:09 AM UTC. 2024-03-30T01:26:11.837 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-30T01:26:11.845 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-30T01:26:11.892 INFO:teuthology.orchestra.run.smithi059.stderr:Error: No packages marked for reinstall. 2024-03-30T01:26:11.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum reinstall -y kernel || true 2024-03-30T01:26:11.948 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2024-03-30T01:26:11.948 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2024-03-30T01:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T01:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:26:12.331 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2024-03-30T01:26:12.331 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2024-03-30T01:26:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:26:12.545 INFO:teuthology.orchestra.run.smithi059.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T01:26:12.545 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:26:12.560 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:03 ago on Sat 30 Mar 2024 01:26:09 AM UTC. 2024-03-30T01:26:12.634 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-30T01:26:12.634 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-30T01:26:12.678 INFO:teuthology.orchestra.run.smithi031.stderr:Error: No packages marked for reinstall. 2024-03-30T01:26:12.733 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-03-30T01:26:12.904 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:03 ago on Sat 30 Mar 2024 01:26:09 AM UTC. 2024-03-30T01:26:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-03-30T01:26:12.907 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-03-30T01:26:12.907 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-03-30T01:26:12.907 DEBUG:teuthology.parallel:result is None 2024-03-30T01:26:12.976 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-30T01:26:12.976 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-30T01:26:13.005 INFO:teuthology.orchestra.run.smithi059.stderr:Error: No packages marked for reinstall. 2024-03-30T01:26:13.059 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2024-03-30T01:26:13.226 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-03-30T01:26:13.226 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-03-30T01:26:13.226 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-03-30T01:26:13.226 DEBUG:teuthology.parallel:result is None 2024-03-30T01:26:13.227 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-30T01:26:13.234 INFO:teuthology.task.internal:Creating test directory... 2024-03-30T01:26:13.234 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T01:26:13.237 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T01:26:13.256 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-30T01:26:13.270 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-30T01:26:13.295 INFO:teuthology.task.internal:Creating archive directory... 2024-03-30T01:26:13.295 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T01:26:13.299 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T01:26:13.337 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-30T01:26:13.344 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-30T01:26:13.344 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T01:26:13.372 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T01:26:13.413 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T01:26:13.413 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T01:26:13.430 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T01:26:13.434 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T01:26:13.436 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-30T01:26:13.443 INFO:teuthology.task.internal:Configuring sudo... 2024-03-30T01:26:13.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T01:26:13.478 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T01:26:13.514 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-30T01:26:13.522 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-30T01:26:13.522 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T01:26:13.554 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T01:26:13.576 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T01:26:13.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T01:26:13.775 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T01:26:13.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T01:26:13.873 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:26:13.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T01:26:13.947 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T01:26:13.990 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T01:26:14.058 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T01:26:14.098 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T01:26:14.156 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:26:14.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T01:26:14.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-03-30T01:26:14.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-03-30T01:26:14.288 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T01:26:14.321 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T01:26:14.730 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-30T01:26:14.739 INFO:teuthology.task.internal:Starting timer... 2024-03-30T01:26:14.739 INFO:teuthology.run_tasks:Running task pcp... 2024-03-30T01:26:14.790 INFO:teuthology.run_tasks:Running task selinux... 2024-03-30T01:26:14.813 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-30T01:26:14.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-03-30T01:26:14.931 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-03-30T01:26:14.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-03-30T01:26:15.000 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-03-30T01:26:15.001 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-30T01:26:15.002 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-03-30T01:26:15.032 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-03-30T01:26:15.032 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-03-30T01:26:15.061 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-03-30T01:26:15.061 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi059.front.sepia.ceph.com': 'permissive'} 2024-03-30T01:26:15.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T01:26:15.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T01:26:15.104 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T01:26:15.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T01:26:15.137 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-30T01:26:15.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-03-30T01:26:15.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-03-30T01:26:15.207 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-30T01:26:15.217 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-30T01:26:15.237 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-30T01:26:15.238 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi059.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-30T01:36:51.891 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi059.front.sepia.ceph.com')] 2024-03-30T01:36:51.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-03-30T01:36:51.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T01:36:51.978 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-03-30T01:36:52.051 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-03-30T01:36:52.052 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-03-30T01:36:52.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T01:36:52.131 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-03-30T01:36:52.203 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-03-30T01:36:52.204 INFO:teuthology.run_tasks:Running task clock... 2024-03-30T01:36:52.215 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-30T01:36:52.215 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T01:36:52.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T01:36:52.218 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T01:36:52.218 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T01:36:52.256 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T01:36:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T01:36:52.284 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T01:36:52.302 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T01:36:52.357 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-03-30T01:36:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-03-30T01:36:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T01:36:52.398 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-03-30T01:36:52.406 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T01:36:52.412 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-03-30T01:36:52.430 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T01:36:52.448 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T01:36:52.479 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2024-03-30T01:36:52.483 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T01:36:52.483 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-03-30T01:36:52.483 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T01:36:52.483 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T01:36:52.483 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T01:36:52.483 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T01:36:52.544 INFO:teuthology.orchestra.run.smithi059.stderr:bash: ntpq: command not found 2024-03-30T01:36:52.549 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T01:36:52.549 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-03-30T01:36:52.549 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T01:36:52.549 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T01:36:52.549 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T01:36:52.549 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T01:36:52.550 INFO:teuthology.run_tasks:Running task install... 2024-03-30T01:36:52.559 DEBUG:teuthology.task.install:project ceph 2024-03-30T01:36:52.559 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}} 2024-03-30T01:36:52.559 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'} 2024-03-30T01:36:52.559 INFO:teuthology.task.install:Using flavor: default 2024-03-30T01:36:52.566 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-30T01:36:52.566 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-30T01:36:52.567 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'tag': None, 'wait_for_package': False} 2024-03-30T01:36:52.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T01:36:52.569 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'tag': None, 'wait_for_package': False} 2024-03-30T01:36:52.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T01:36:52.712 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/ 2024-03-30T01:36:52.712 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1018.g470100a4 2024-03-30T01:36:52.723 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/ 2024-03-30T01:36:52.723 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1018.g470100a4 2024-03-30T01:36:52.855 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T01:36:52.855 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:36:52.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T01:36:52.891 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T01:36:52.891 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/470100a4ccbcf7769052bf09a5413881ccf1b97f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T01:36:52.912 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T01:36:52.912 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:36:52.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T01:36:52.949 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T01:36:52.949 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/470100a4ccbcf7769052bf09a5413881ccf1b97f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T01:36:52.967 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T01:36:53.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T01:36:53.055 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T01:36:53.086 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-03-30T01:36:53.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-03-30T01:36:53.115 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T01:36:53.147 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-03-30T01:36:53.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-03-30T01:36:53.525 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2024-03-30T01:36:53.581 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2024-03-30T01:36:54.938 INFO:teuthology.orchestra.run.smithi031.stdout:61 files removed 2024-03-30T01:36:54.938 INFO:teuthology.orchestra.run.smithi059.stdout:61 files removed 2024-03-30T01:36:54.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T01:36:54.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T01:36:55.378 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2024-03-30T01:36:55.380 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2024-03-30T01:36:57.255 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 318 kB/s | 80 kB 00:00 2024-03-30T01:36:57.432 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 187 kB/s | 80 kB 00:00 2024-03-30T01:36:57.639 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 29 kB/s | 9.9 kB 00:00 2024-03-30T01:36:57.756 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 37 kB/s | 9.9 kB 00:00 2024-03-30T01:36:57.956 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2024-03-30T01:36:58.115 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2024-03-30T01:36:58.140 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-03-30T01:36:58.283 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-03-30T01:36:59.508 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2024-03-30T01:37:03.123 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 3.3 MB/s | 16 MB 00:04 2024-03-30T01:37:03.550 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 465 kB/s | 24 kB 00:00 2024-03-30T01:37:04.968 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 66 MB 00:01 2024-03-30T01:37:07.107 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 432 kB/s | 24 kB 00:00 2024-03-30T01:37:08.642 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 66 MB 00:01 2024-03-30T01:37:16.103 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-03-30T01:37:19.478 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 MB/s | 60 MB 00:01 2024-03-30T01:37:19.751 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-03-30T01:37:22.952 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 60 MB 00:01 2024-03-30T01:37:36.554 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T01:37:36.555 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T01:37:36.574 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-30T01:37:36.576 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-30T01:37:36.577 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-30T01:37:36.579 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-30T01:37:36.582 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-30T01:37:36.586 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-30T01:37:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-30T01:37:36.830 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-03-30T01:37:36.846 INFO:teuthology.orchestra.run.smithi031.stdout:=================================================================================================================================== 2024-03-30T01:37:36.846 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout:=================================================================================================================================== 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.4 k 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:18.2.2-1018.g470100a4.el8 ceph 5.5 M 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 854 k 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1018.g470100a4.el8 ceph 166 k 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:18.2.2-1018.g470100a4.el8 ceph 1.6 M 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 138 k 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 1.8 M 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 7.4 M 2024-03-30T01:37:36.847 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 49 k 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 13 M 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:18.2.2-1018.g470100a4.el8 ceph 47 M 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 263 k 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 173 k 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 261 k 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 18 k 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 31 k 2024-03-30T01:37:36.848 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 775 k 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 139 k 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:18.2.2-1018.g470100a4.el8 ceph 196 k 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:18.2.2-1018.g470100a4.el8 ceph 371 k 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 366 k 2024-03-30T01:37:36.849 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 107 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 95 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.5 M 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 181 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.7 M 2024-03-30T01:37:36.850 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.6 M 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 22 M 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 23 k 2024-03-30T01:37:36.851 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:18.2.2-1018.g470100a4.el8 ceph 2.3 M 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 245 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:18.2.2-1018.g470100a4.el8 ceph 4.8 M 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 18 M 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 15 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:18.2.2-1018.g470100a4.el8 ceph 24 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:18.2.2-1018.g470100a4.el8 ceph 142 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-30T01:37:36.852 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 475 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.5 M 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-30T01:37:36.853 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 45 k 2024-03-30T01:37:36.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 129 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-30T01:37:36.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-30T01:37:36.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-30T01:37:36.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling module streams: 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql 10 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:37:36.858 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-03-30T01:37:36.859 INFO:teuthology.orchestra.run.smithi031.stdout:=================================================================================================================================== 2024-03-30T01:37:36.859 INFO:teuthology.orchestra.run.smithi031.stdout:Install 130 Packages 2024-03-30T01:37:36.859 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 16 Packages 2024-03-30T01:37:36.859 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:37:36.859 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 213 M 2024-03-30T01:37:36.859 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-03-30T01:37:37.386 INFO:teuthology.orchestra.run.smithi031.stdout:(1/146): ceph-18.2.2-1018.g470100a4.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2024-03-30T01:37:37.678 INFO:teuthology.orchestra.run.smithi031.stdout:(2/146): ceph-fuse-18.2.2-1018.g470100a4.el8.x8 2.9 MB/s | 854 kB 00:00 2024-03-30T01:37:37.778 INFO:teuthology.orchestra.run.smithi031.stdout:(3/146): ceph-immutable-object-cache-18.2.2-101 1.6 MB/s | 166 kB 00:00 2024-03-30T01:37:38.245 INFO:teuthology.orchestra.run.smithi031.stdout:(4/146): ceph-mds-18.2.2-1018.g470100a4.el8.x86 4.9 MB/s | 2.3 MB 00:00 2024-03-30T01:37:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:(5/146): ceph-base-18.2.2-1018.g470100a4.el8.x8 4.8 MB/s | 5.5 MB 00:01 2024-03-30T01:37:38.604 INFO:teuthology.orchestra.run.smithi031.stdout:(6/146): ceph-mgr-18.2.2-1018.g470100a4.el8.x86 4.3 MB/s | 1.6 MB 00:00 2024-03-30T01:37:38.979 INFO:teuthology.orchestra.run.smithi031.stdout:(7/146): ceph-common-18.2.2-1018.g470100a4.el8. 12 MB/s | 22 MB 00:01 2024-03-30T01:37:39.063 INFO:teuthology.orchestra.run.smithi031.stdout:(8/146): ceph-mon-18.2.2-1018.g470100a4.el8.x86 6.7 MB/s | 4.8 MB 00:00 2024-03-30T01:37:39.138 INFO:teuthology.orchestra.run.smithi031.stdout:(9/146): ceph-selinux-18.2.2-1018.g470100a4.el8 326 kB/s | 24 kB 00:00 2024-03-30T01:37:40.030 INFO:teuthology.orchestra.run.smithi031.stdout:(10/146): ceph-radosgw-18.2.2-1018.g470100a4.el 12 MB/s | 13 MB 00:01 2024-03-30T01:37:40.075 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T01:37:40.077 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T01:37:40.088 INFO:teuthology.orchestra.run.smithi031.stdout:(11/146): cephfs-mirror-18.2.2-1018.g470100a4.e 4.4 MB/s | 261 kB 00:00 2024-03-30T01:37:40.098 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-30T01:37:40.099 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-30T01:37:40.101 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-30T01:37:40.103 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-30T01:37:40.106 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-30T01:37:40.111 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-30T01:37:40.113 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-30T01:37:40.147 INFO:teuthology.orchestra.run.smithi031.stdout:(12/146): libcephfs-devel-18.2.2-1018.g470100a4 529 kB/s | 31 kB 00:00 2024-03-30T01:37:40.272 INFO:teuthology.orchestra.run.smithi031.stdout:(13/146): libcephfs2-18.2.2-1018.g470100a4.el8. 6.1 MB/s | 775 kB 00:00 2024-03-30T01:37:40.339 INFO:teuthology.orchestra.run.smithi031.stdout:(14/146): libcephsqlite-18.2.2-1018.g470100a4.e 2.1 MB/s | 142 kB 00:00 2024-03-30T01:37:40.355 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-03-30T01:37:40.371 INFO:teuthology.orchestra.run.smithi059.stdout:=================================================================================================================================== 2024-03-30T01:37:40.371 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-03-30T01:37:40.371 INFO:teuthology.orchestra.run.smithi059.stdout:=================================================================================================================================== 2024-03-30T01:37:40.371 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-03-30T01:37:40.371 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-30T01:37:40.371 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-30T01:37:40.371 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.4 k 2024-03-30T01:37:40.371 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:18.2.2-1018.g470100a4.el8 ceph 5.5 M 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 854 k 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1018.g470100a4.el8 ceph 166 k 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:18.2.2-1018.g470100a4.el8 ceph 1.6 M 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 138 k 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 1.8 M 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 7.4 M 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 49 k 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 13 M 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:18.2.2-1018.g470100a4.el8 ceph 47 M 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 263 k 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 173 k 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 261 k 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 18 k 2024-03-30T01:37:40.372 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 31 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 775 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 139 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:18.2.2-1018.g470100a4.el8 ceph 196 k 2024-03-30T01:37:40.373 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:18.2.2-1018.g470100a4.el8 ceph 371 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 366 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 107 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 95 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.5 M 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 181 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-30T01:37:40.374 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.7 M 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.6 M 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 22 M 2024-03-30T01:37:40.375 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 23 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:18.2.2-1018.g470100a4.el8 ceph 2.3 M 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 245 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:18.2.2-1018.g470100a4.el8 ceph 4.8 M 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 18 M 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 15 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:18.2.2-1018.g470100a4.el8 ceph 24 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:18.2.2-1018.g470100a4.el8 ceph 142 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-30T01:37:40.376 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 475 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.5 M 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-30T01:37:40.377 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 45 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 129 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-30T01:37:40.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-30T01:37:40.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-30T01:37:40.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-30T01:37:40.381 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling module streams: 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql 10 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout:=================================================================================================================================== 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout:Install 130 Packages 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 16 Packages 2024-03-30T01:37:40.382 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:37:40.383 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 213 M 2024-03-30T01:37:40.383 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-03-30T01:37:40.390 INFO:teuthology.orchestra.run.smithi031.stdout:(15/146): librados-devel-18.2.2-1018.g470100a4. 2.7 MB/s | 139 kB 00:00 2024-03-30T01:37:40.498 INFO:teuthology.orchestra.run.smithi031.stdout:(16/146): libradosstriper1-18.2.2-1018.g470100a 4.3 MB/s | 475 kB 00:00 2024-03-30T01:37:40.718 INFO:teuthology.orchestra.run.smithi059.stdout:(1/146): ceph-18.2.2-1018.g470100a4.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2024-03-30T01:37:41.107 INFO:teuthology.orchestra.run.smithi031.stdout:(17/146): librgw2-18.2.2-1018.g470100a4.el8.x86 11 MB/s | 6.5 MB 00:00 2024-03-30T01:37:41.166 INFO:teuthology.orchestra.run.smithi031.stdout:(18/146): python3-ceph-argparse-18.2.2-1018.g47 770 kB/s | 45 kB 00:00 2024-03-30T01:37:41.216 INFO:teuthology.orchestra.run.smithi031.stdout:(19/146): python3-ceph-common-18.2.2-1018.g4701 2.5 MB/s | 129 kB 00:00 2024-03-30T01:37:41.385 INFO:teuthology.orchestra.run.smithi059.stdout:(2/146): ceph-fuse-18.2.2-1018.g470100a4.el8.x8 1.3 MB/s | 854 kB 00:00 2024-03-30T01:37:41.416 INFO:teuthology.orchestra.run.smithi031.stdout:(20/146): ceph-osd-18.2.2-1018.g470100a4.el8.x8 6.5 MB/s | 18 MB 00:02 2024-03-30T01:37:41.441 INFO:teuthology.orchestra.run.smithi031.stdout:(21/146): python3-cephfs-18.2.2-1018.g470100a4. 874 kB/s | 196 kB 00:00 2024-03-30T01:37:41.494 INFO:teuthology.orchestra.run.smithi059.stdout:(3/146): ceph-immutable-object-cache-18.2.2-101 1.5 MB/s | 166 kB 00:00 2024-03-30T01:37:41.500 INFO:teuthology.orchestra.run.smithi031.stdout:(22/146): python3-rbd-18.2.2-1018.g470100a4.el8 6.1 MB/s | 366 kB 00:00 2024-03-30T01:37:41.525 INFO:teuthology.orchestra.run.smithi031.stdout:(23/146): python3-rados-18.2.2-1018.g470100a4.e 3.3 MB/s | 371 kB 00:00 2024-03-30T01:37:41.576 INFO:teuthology.orchestra.run.smithi031.stdout:(24/146): python3-rgw-18.2.2-1018.g470100a4.el8 1.4 MB/s | 107 kB 00:00 2024-03-30T01:37:41.609 INFO:teuthology.orchestra.run.smithi031.stdout:(25/146): rbd-fuse-18.2.2-1018.g470100a4.el8.x8 1.1 MB/s | 95 kB 00:00 2024-03-30T01:37:41.701 INFO:teuthology.orchestra.run.smithi031.stdout:(26/146): rbd-nbd-18.2.2-1018.g470100a4.el8.x86 1.9 MB/s | 181 kB 00:00 2024-03-30T01:37:41.801 INFO:teuthology.orchestra.run.smithi031.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1018.g 233 kB/s | 23 kB 00:00 2024-03-30T01:37:41.927 INFO:teuthology.orchestra.run.smithi031.stdout:(28/146): rbd-mirror-18.2.2-1018.g470100a4.el8. 9.9 MB/s | 3.5 MB 00:00 2024-03-30T01:37:41.944 INFO:teuthology.orchestra.run.smithi031.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1018.g470100a 974 kB/s | 138 kB 00:00 2024-03-30T01:37:42.161 INFO:teuthology.orchestra.run.smithi031.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1018.g47010 7.7 MB/s | 1.8 MB 00:00 2024-03-30T01:37:42.211 INFO:teuthology.orchestra.run.smithi031.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1018.g47 4.8 MB/s | 245 kB 00:00 2024-03-30T01:37:42.253 INFO:teuthology.orchestra.run.smithi031.stdout:(32/146): ceph-mgr-rook-18.2.2-1018.g470100a4.e 1.2 MB/s | 49 kB 00:00 2024-03-30T01:37:42.294 INFO:teuthology.orchestra.run.smithi031.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1018.g4 354 kB/s | 15 kB 00:00 2024-03-30T01:37:42.395 INFO:teuthology.orchestra.run.smithi031.stdout:(34/146): ceph-volume-18.2.2-1018.g470100a4.el8 2.6 MB/s | 263 kB 00:00 2024-03-30T01:37:42.452 INFO:teuthology.orchestra.run.smithi059.stdout:(4/146): ceph-mds-18.2.2-1018.g470100a4.el8.x86 2.4 MB/s | 2.3 MB 00:00 2024-03-30T01:37:42.479 INFO:teuthology.orchestra.run.smithi031.stdout:(35/146): cephadm-18.2.2-1018.g470100a4.el8.noa 2.0 MB/s | 173 kB 00:00 2024-03-30T01:37:42.546 INFO:teuthology.orchestra.run.smithi031.stdout:(36/146): cephfs-top-18.2.2-1018.g470100a4.el8. 272 kB/s | 18 kB 00:00 2024-03-30T01:37:42.754 INFO:teuthology.orchestra.run.smithi031.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2024-03-30T01:37:42.786 INFO:teuthology.orchestra.run.smithi059.stdout:(5/146): ceph-base-18.2.2-1018.g470100a4.el8.x8 2.4 MB/s | 5.5 MB 00:02 2024-03-30T01:37:42.804 INFO:teuthology.orchestra.run.smithi031.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 1.2 MB/s | 59 kB 00:00 2024-03-30T01:37:42.847 INFO:teuthology.orchestra.run.smithi031.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2024-03-30T01:37:42.955 INFO:teuthology.orchestra.run.smithi031.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2024-03-30T01:37:42.994 INFO:teuthology.orchestra.run.smithi059.stdout:(6/146): ceph-mgr-18.2.2-1018.g470100a4.el8.x86 2.9 MB/s | 1.6 MB 00:00 2024-03-30T01:37:43.005 INFO:teuthology.orchestra.run.smithi031.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 875 kB/s | 44 kB 00:00 2024-03-30T01:37:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2024-03-30T01:37:43.106 INFO:teuthology.orchestra.run.smithi031.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 661 kB/s | 33 kB 00:00 2024-03-30T01:37:43.156 INFO:teuthology.orchestra.run.smithi031.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2024-03-30T01:37:43.232 INFO:teuthology.orchestra.run.smithi031.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2024-03-30T01:37:43.290 INFO:teuthology.orchestra.run.smithi031.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 6.4 MB/s | 384 kB 00:00 2024-03-30T01:37:43.374 INFO:teuthology.orchestra.run.smithi031.stdout:(47/146): ceph-mgr-diskprediction-local-18.2.2- 5.2 MB/s | 7.4 MB 00:01 2024-03-30T01:37:43.399 INFO:teuthology.orchestra.run.smithi031.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 748 kB/s | 81 kB 00:00 2024-03-30T01:37:43.442 INFO:teuthology.orchestra.run.smithi031.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 423 kB/s | 18 kB 00:00 2024-03-30T01:37:43.492 INFO:teuthology.orchestra.run.smithi031.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 94 kB/s | 11 kB 00:00 2024-03-30T01:37:43.584 INFO:teuthology.orchestra.run.smithi031.stdout:(51/146): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2024-03-30T01:37:43.609 INFO:teuthology.orchestra.run.smithi031.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 419 kB/s | 49 kB 00:00 2024-03-30T01:37:43.634 INFO:teuthology.orchestra.run.smithi031.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2024-03-30T01:37:43.668 INFO:teuthology.orchestra.run.smithi031.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2024-03-30T01:37:43.693 INFO:teuthology.orchestra.run.smithi031.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2024-03-30T01:37:43.718 INFO:teuthology.orchestra.run.smithi031.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2024-03-30T01:37:43.771 INFO:teuthology.orchestra.run.smithi031.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 435 kB/s | 34 kB 00:00 2024-03-30T01:37:43.811 INFO:teuthology.orchestra.run.smithi031.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2024-03-30T01:37:43.836 INFO:teuthology.orchestra.run.smithi031.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 991 kB/s | 64 kB 00:00 2024-03-30T01:37:43.861 INFO:teuthology.orchestra.run.smithi031.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2024-03-30T01:37:43.895 INFO:teuthology.orchestra.run.smithi031.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 408 kB/s | 24 kB 00:00 2024-03-30T01:37:43.912 INFO:teuthology.orchestra.run.smithi031.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 583 kB/s | 29 kB 00:00 2024-03-30T01:37:43.946 INFO:teuthology.orchestra.run.smithi031.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2024-03-30T01:37:43.971 INFO:teuthology.orchestra.run.smithi031.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 4.3 MB/s | 255 kB 00:00 2024-03-30T01:37:44.005 INFO:teuthology.orchestra.run.smithi031.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2024-03-30T01:37:44.028 INFO:teuthology.orchestra.run.smithi059.stdout:(7/146): ceph-mon-18.2.2-1018.g470100a4.el8.x86 3.9 MB/s | 4.8 MB 00:01 2024-03-30T01:37:44.055 INFO:teuthology.orchestra.run.smithi031.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 733 kB/s | 61 kB 00:00 2024-03-30T01:37:44.080 INFO:teuthology.orchestra.run.smithi031.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2024-03-30T01:37:44.097 INFO:teuthology.orchestra.run.smithi031.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 544 kB/s | 23 kB 00:00 2024-03-30T01:37:44.123 INFO:teuthology.orchestra.run.smithi031.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-03-30T01:37:44.190 INFO:teuthology.orchestra.run.smithi031.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2024-03-30T01:37:44.240 INFO:teuthology.orchestra.run.smithi031.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 153 kB/s | 18 kB 00:00 2024-03-30T01:37:44.390 INFO:teuthology.orchestra.run.smithi031.stdout:(72/146): python3-jwt-1.6.1-2.el8.noarch.rpm 213 kB/s | 43 kB 00:00 2024-03-30T01:37:44.416 INFO:teuthology.orchestra.run.smithi031.stdout:(73/146): python3-oauthlib-2.1.0-1.el8.noarch.r 885 kB/s | 155 kB 00:00 2024-03-30T01:37:44.549 INFO:teuthology.orchestra.run.smithi031.stdout:(74/146): python3-pycparser-2.14-14.el8.noarch. 688 kB/s | 109 kB 00:00 2024-03-30T01:37:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 183 kB/s | 43 kB 00:00 2024-03-30T01:37:44.675 INFO:teuthology.orchestra.run.smithi031.stdout:(76/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 149 kB/s | 19 kB 00:00 2024-03-30T01:37:44.712 INFO:teuthology.orchestra.run.smithi059.stdout:(8/146): ceph-common-18.2.2-1018.g470100a4.el8. 5.1 MB/s | 22 MB 00:04 2024-03-30T01:37:44.792 INFO:teuthology.orchestra.run.smithi031.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 309 kB/s | 44 kB 00:00 2024-03-30T01:37:44.803 INFO:teuthology.orchestra.run.smithi059.stdout:(9/146): ceph-selinux-18.2.2-1018.g470100a4.el8 266 kB/s | 24 kB 00:00 2024-03-30T01:37:44.817 INFO:teuthology.orchestra.run.smithi031.stdout:(78/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.2 MB/s | 605 kB 00:00 2024-03-30T01:37:44.951 INFO:teuthology.orchestra.run.smithi031.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2024-03-30T01:37:44.984 INFO:teuthology.orchestra.run.smithi031.stdout:(80/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2024-03-30T01:37:45.001 INFO:teuthology.orchestra.run.smithi031.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2024-03-30T01:37:45.196 INFO:teuthology.orchestra.run.smithi059.stdout:(10/146): ceph-osd-18.2.2-1018.g470100a4.el8.x8 8.4 MB/s | 18 MB 00:02 2024-03-30T01:37:45.271 INFO:teuthology.orchestra.run.smithi059.stdout:(11/146): cephfs-mirror-18.2.2-1018.g470100a4.e 3.4 MB/s | 261 kB 00:00 2024-03-30T01:37:45.346 INFO:teuthology.orchestra.run.smithi059.stdout:(12/146): libcephfs-devel-18.2.2-1018.g470100a4 414 kB/s | 31 kB 00:00 2024-03-30T01:37:45.446 INFO:teuthology.orchestra.run.smithi059.stdout:(13/146): libcephfs2-18.2.2-1018.g470100a4.el8. 7.6 MB/s | 775 kB 00:00 2024-03-30T01:37:45.518 INFO:teuthology.orchestra.run.smithi031.stdout:(82/146): ceph-test-18.2.2-1018.g470100a4.el8.x 7.4 MB/s | 47 MB 00:06 2024-03-30T01:37:45.530 INFO:teuthology.orchestra.run.smithi059.stdout:(14/146): libcephsqlite-18.2.2-1018.g470100a4.e 1.7 MB/s | 142 kB 00:00 2024-03-30T01:37:45.543 INFO:teuthology.orchestra.run.smithi031.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 86 kB/s | 48 kB 00:00 2024-03-30T01:37:45.577 INFO:teuthology.orchestra.run.smithi031.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 151 kB/s | 87 kB 00:00 2024-03-30T01:37:45.644 INFO:teuthology.orchestra.run.smithi031.stdout:(85/146): fuse-2.9.7-17.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2024-03-30T01:37:45.705 INFO:teuthology.orchestra.run.smithi059.stdout:(15/146): librados-devel-18.2.2-1018.g470100a4. 796 kB/s | 139 kB 00:00 2024-03-30T01:37:45.719 INFO:teuthology.orchestra.run.smithi031.stdout:(86/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 288 kB/s | 58 kB 00:00 2024-03-30T01:37:45.777 INFO:teuthology.orchestra.run.smithi031.stdout:(87/146): libcom_err-devel-1.45.6-4.el8.x86_64. 172 kB/s | 39 kB 00:00 2024-03-30T01:37:45.788 INFO:teuthology.orchestra.run.smithi059.stdout:(16/146): libradosstriper1-18.2.2-1018.g470100a 5.6 MB/s | 475 kB 00:00 2024-03-30T01:37:45.828 INFO:teuthology.orchestra.run.smithi031.stdout:(88/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 645 kB 00:00 2024-03-30T01:37:45.854 INFO:teuthology.orchestra.run.smithi031.stdout:(89/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-03-30T01:37:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2024-03-30T01:37:45.921 INFO:teuthology.orchestra.run.smithi031.stdout:(91/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 694 kB/s | 100 kB 00:00 2024-03-30T01:37:45.996 INFO:teuthology.orchestra.run.smithi031.stdout:(92/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 511 kB/s | 73 kB 00:00 2024-03-30T01:37:46.046 INFO:teuthology.orchestra.run.smithi031.stdout:(93/146): elfutils-libelf-devel-0.189-3.el8.x86 495 kB/s | 62 kB 00:00 2024-03-30T01:37:46.080 INFO:teuthology.orchestra.run.smithi031.stdout:(94/146): NetworkManager-initscripts-updown-1.4 782 kB/s | 144 kB 00:00 2024-03-30T01:37:46.155 INFO:teuthology.orchestra.run.smithi031.stdout:(95/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.4 MB/s | 562 kB 00:00 2024-03-30T01:37:46.272 INFO:teuthology.orchestra.run.smithi031.stdout:(96/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 245 kB/s | 47 kB 00:00 2024-03-30T01:37:46.331 INFO:teuthology.orchestra.run.smithi059.stdout:(17/146): librgw2-18.2.2-1018.g470100a4.el8.x86 12 MB/s | 6.5 MB 00:00 2024-03-30T01:37:46.364 INFO:teuthology.orchestra.run.smithi031.stdout:(97/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 7.3 MB/s | 2.3 MB 00:00 2024-03-30T01:37:46.398 INFO:teuthology.orchestra.run.smithi031.stdout:(98/146): python3-cryptography-3.2.1-7.el8_9.x8 2.3 MB/s | 559 kB 00:00 2024-03-30T01:37:46.406 INFO:teuthology.orchestra.run.smithi059.stdout:(18/146): python3-ceph-argparse-18.2.2-1018.g47 598 kB/s | 45 kB 00:00 2024-03-30T01:37:46.431 INFO:teuthology.orchestra.run.smithi031.stdout:(99/146): python3-packaging-16.8-9.el8.noarch.r 304 kB/s | 48 kB 00:00 2024-03-30T01:37:46.489 INFO:teuthology.orchestra.run.smithi059.stdout:(19/146): python3-ceph-common-18.2.2-1018.g4701 1.5 MB/s | 129 kB 00:00 2024-03-30T01:37:46.498 INFO:teuthology.orchestra.run.smithi031.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 149 kB/s | 20 kB 00:00 2024-03-30T01:37:46.540 INFO:teuthology.orchestra.run.smithi031.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 309 kB/s | 44 kB 00:00 2024-03-30T01:37:46.573 INFO:teuthology.orchestra.run.smithi059.stdout:(20/146): python3-cephfs-18.2.2-1018.g470100a4. 2.3 MB/s | 196 kB 00:00 2024-03-30T01:37:46.601 INFO:teuthology.orchestra.run.smithi031.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 648 kB/s | 110 kB 00:00 2024-03-30T01:37:46.657 INFO:teuthology.orchestra.run.smithi059.stdout:(21/146): python3-rados-18.2.2-1018.g470100a4.e 4.4 MB/s | 371 kB 00:00 2024-03-30T01:37:46.674 INFO:teuthology.orchestra.run.smithi031.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 151 kB/s | 27 kB 00:00 2024-03-30T01:37:46.699 INFO:teuthology.orchestra.run.smithi031.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 796 kB/s | 126 kB 00:00 2024-03-30T01:37:46.733 INFO:teuthology.orchestra.run.smithi031.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 3.4 MB/s | 457 kB 00:00 2024-03-30T01:37:46.790 INFO:teuthology.orchestra.run.smithi059.stdout:(22/146): ceph-radosgw-18.2.2-1018.g470100a4.el 4.7 MB/s | 13 MB 00:02 2024-03-30T01:37:46.808 INFO:teuthology.orchestra.run.smithi031.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2024-03-30T01:37:46.815 INFO:teuthology.orchestra.run.smithi059.stdout:(23/146): python3-rbd-18.2.2-1018.g470100a4.el8 2.3 MB/s | 366 kB 00:00 2024-03-30T01:37:46.842 INFO:teuthology.orchestra.run.smithi031.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2024-03-30T01:37:46.874 INFO:teuthology.orchestra.run.smithi059.stdout:(24/146): python3-rgw-18.2.2-1018.g470100a4.el8 1.3 MB/s | 107 kB 00:00 2024-03-30T01:37:46.899 INFO:teuthology.orchestra.run.smithi059.stdout:(25/146): rbd-fuse-18.2.2-1018.g470100a4.el8.x8 1.1 MB/s | 95 kB 00:00 2024-03-30T01:37:46.909 INFO:teuthology.orchestra.run.smithi031.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2024-03-30T01:37:46.926 INFO:teuthology.orchestra.run.smithi031.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 331 kB/s | 39 kB 00:00 2024-03-30T01:37:46.966 INFO:teuthology.orchestra.run.smithi059.stdout:(26/146): rbd-nbd-18.2.2-1018.g470100a4.el8.x86 2.7 MB/s | 181 kB 00:00 2024-03-30T01:37:46.984 INFO:teuthology.orchestra.run.smithi031.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.8 MB/s | 259 kB 00:00 2024-03-30T01:37:47.041 INFO:teuthology.orchestra.run.smithi059.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1018.g 310 kB/s | 23 kB 00:00 2024-03-30T01:37:47.043 INFO:teuthology.orchestra.run.smithi031.stdout:(111/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 886 kB/s | 103 kB 00:00 2024-03-30T01:37:47.068 INFO:teuthology.orchestra.run.smithi031.stdout:(112/146): uuid-1.6.2-43.el8.x86_64.rpm 403 kB/s | 64 kB 00:00 2024-03-30T01:37:47.125 INFO:teuthology.orchestra.run.smithi059.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1018.g470100a 1.6 MB/s | 138 kB 00:00 2024-03-30T01:37:47.219 INFO:teuthology.orchestra.run.smithi031.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 3.0 MB/s | 538 kB 00:00 2024-03-30T01:37:47.310 INFO:teuthology.orchestra.run.smithi031.stdout:(114/146): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2024-03-30T01:37:47.344 INFO:teuthology.orchestra.run.smithi031.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2024-03-30T01:37:47.350 INFO:teuthology.orchestra.run.smithi059.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1018.g47010 7.9 MB/s | 1.8 MB 00:00 2024-03-30T01:37:47.461 INFO:teuthology.orchestra.run.smithi031.stdout:(116/146): python3-babel-2.5.1-7.el8.noarch.rpm 10 MB/s | 4.8 MB 00:00 2024-03-30T01:37:47.478 INFO:teuthology.orchestra.run.smithi031.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.3 MB/s | 225 kB 00:00 2024-03-30T01:37:47.503 INFO:teuthology.orchestra.run.smithi031.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2024-03-30T01:37:47.629 INFO:teuthology.orchestra.run.smithi031.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2024-03-30T01:37:47.676 INFO:teuthology.orchestra.run.smithi059.stdout:(30/146): rbd-mirror-18.2.2-1018.g470100a4.el8. 4.3 MB/s | 3.5 MB 00:00 2024-03-30T01:37:47.754 INFO:teuthology.orchestra.run.smithi031.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 18 MB/s | 5.2 MB 00:00 2024-03-30T01:37:47.801 INFO:teuthology.orchestra.run.smithi059.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1018.g47 1.9 MB/s | 245 kB 00:00 2024-03-30T01:37:47.812 INFO:teuthology.orchestra.run.smithi031.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 3.8 MB/s | 714 kB 00:00 2024-03-30T01:37:47.896 INFO:teuthology.orchestra.run.smithi031.stdout:(122/146): openblas-0.3.15-6.el8.x86_64.rpm 13 MB/s | 5.0 MB 00:00 2024-03-30T01:37:47.909 INFO:teuthology.orchestra.run.smithi059.stdout:(32/146): ceph-mgr-rook-18.2.2-1018.g470100a4.e 455 kB/s | 49 kB 00:00 2024-03-30T01:37:47.922 INFO:teuthology.orchestra.run.smithi031.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2024-03-30T01:37:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2024-03-30T01:37:48.026 INFO:teuthology.orchestra.run.smithi059.stdout:(33/146): ceph-mgr-diskprediction-local-18.2.2- 11 MB/s | 7.4 MB 00:00 2024-03-30T01:37:48.051 INFO:teuthology.orchestra.run.smithi059.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1018.g4 104 kB/s | 15 kB 00:00 2024-03-30T01:37:48.064 INFO:teuthology.orchestra.run.smithi031.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 324 kB/s | 54 kB 00:00 2024-03-30T01:37:48.122 INFO:teuthology.orchestra.run.smithi031.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-03-30T01:37:48.193 INFO:teuthology.orchestra.run.smithi059.stdout:(35/146): ceph-volume-18.2.2-1018.g470100a4.el8 1.5 MB/s | 263 kB 00:00 2024-03-30T01:37:48.219 INFO:teuthology.orchestra.run.smithi059.stdout:(36/146): cephadm-18.2.2-1018.g470100a4.el8.noa 1.0 MB/s | 173 kB 00:00 2024-03-30T01:37:48.264 INFO:teuthology.orchestra.run.smithi031.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 7.5 MB/s | 1.5 MB 00:00 2024-03-30T01:37:48.286 INFO:teuthology.orchestra.run.smithi059.stdout:(37/146): cephfs-top-18.2.2-1018.g470100a4.el8. 197 kB/s | 18 kB 00:00 2024-03-30T01:37:48.306 INFO:teuthology.orchestra.run.smithi031.stdout:(128/146): postgresql-contrib-10.23-4.module+el 4.3 MB/s | 812 kB 00:00 2024-03-30T01:37:48.457 INFO:teuthology.orchestra.run.smithi031.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 26 MB/s | 14 MB 00:00 2024-03-30T01:37:48.511 INFO:teuthology.orchestra.run.smithi059.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 261 kB/s | 59 kB 00:00 2024-03-30T01:37:48.536 INFO:teuthology.orchestra.run.smithi059.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 965 kB/s | 306 kB 00:00 2024-03-30T01:37:48.582 INFO:teuthology.orchestra.run.smithi031.stdout:(130/146): postgresql-server-10.23-4.module+el8 16 MB/s | 5.1 MB 00:00 2024-03-30T01:37:48.957 INFO:teuthology.orchestra.run.smithi031.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 619 kB/s | 232 kB 00:00 2024-03-30T01:37:48.979 INFO:teuthology.orchestra.run.smithi059.stdout:(40/146): ceph-test-18.2.2-1018.g470100a4.el8.x 11 MB/s | 47 MB 00:04 2024-03-30T01:37:49.004 INFO:teuthology.orchestra.run.smithi059.stdout:(41/146): libunwind-1.3.1-3.el8.x86_64.rpm 152 kB/s | 75 kB 00:00 2024-03-30T01:37:49.037 INFO:teuthology.orchestra.run.smithi059.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 950 kB/s | 476 kB 00:00 2024-03-30T01:37:49.104 INFO:teuthology.orchestra.run.smithi059.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2024-03-30T01:37:49.129 INFO:teuthology.orchestra.run.smithi059.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 381 kB/s | 33 kB 00:00 2024-03-30T01:37:49.135 INFO:teuthology.orchestra.run.smithi031.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.7 MB/s | 303 kB 00:00 2024-03-30T01:37:49.163 INFO:teuthology.orchestra.run.smithi059.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 281 kB/s | 16 kB 00:00 2024-03-30T01:37:49.188 INFO:teuthology.orchestra.run.smithi059.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2024-03-30T01:37:49.230 INFO:teuthology.orchestra.run.smithi059.stdout:(47/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 174 kB/s | 44 kB 00:00 2024-03-30T01:37:49.249 INFO:teuthology.orchestra.run.smithi031.stdout:(133/146): librados2-18.2.2-1018.g470100a4.el8. 3.9 MB/s | 3.7 MB 00:00 2024-03-30T01:37:49.255 INFO:teuthology.orchestra.run.smithi059.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2024-03-30T01:37:49.281 INFO:teuthology.orchestra.run.smithi059.stdout:(49/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2024-03-30T01:37:49.298 INFO:teuthology.orchestra.run.smithi059.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 165 kB/s | 11 kB 00:00 2024-03-30T01:37:49.299 INFO:teuthology.orchestra.run.smithi031.stdout:(134/146): libmount-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 237 kB 00:00 2024-03-30T01:37:49.323 INFO:teuthology.orchestra.run.smithi059.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2024-03-30T01:37:49.356 INFO:teuthology.orchestra.run.smithi059.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 833 kB/s | 49 kB 00:00 2024-03-30T01:37:49.358 INFO:teuthology.orchestra.run.smithi031.stdout:(135/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.6 MB/s | 179 kB 00:00 2024-03-30T01:37:49.382 INFO:teuthology.orchestra.run.smithi059.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2024-03-30T01:37:49.415 INFO:teuthology.orchestra.run.smithi059.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2024-03-30T01:37:49.442 INFO:teuthology.orchestra.run.smithi031.stdout:(136/146): libuuid-2.32.1-43.el8.x86_64.rpm 697 kB/s | 99 kB 00:00 2024-03-30T01:37:49.449 INFO:teuthology.orchestra.run.smithi059.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2024-03-30T01:37:49.474 INFO:teuthology.orchestra.run.smithi059.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2024-03-30T01:37:49.500 INFO:teuthology.orchestra.run.smithi059.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 670 kB/s | 34 kB 00:00 2024-03-30T01:37:49.533 INFO:teuthology.orchestra.run.smithi059.stdout:(58/146): python3-kubernetes-11.0.0-6.el8.noarc 4.6 MB/s | 1.2 MB 00:00 2024-03-30T01:37:49.558 INFO:teuthology.orchestra.run.smithi059.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2024-03-30T01:37:49.592 INFO:teuthology.orchestra.run.smithi059.stdout:(60/146): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2024-03-30T01:37:49.609 INFO:teuthology.orchestra.run.smithi059.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2024-03-30T01:37:49.617 INFO:teuthology.orchestra.run.smithi031.stdout:(137/146): util-linux-2.32.1-43.el8.x86_64.rpm 9.6 MB/s | 2.5 MB 00:00 2024-03-30T01:37:49.642 INFO:teuthology.orchestra.run.smithi031.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 513 kB/s | 103 kB 00:00 2024-03-30T01:37:49.667 INFO:teuthology.orchestra.run.smithi059.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2024-03-30T01:37:49.684 INFO:teuthology.orchestra.run.smithi059.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 317 kB/s | 29 kB 00:00 2024-03-30T01:37:49.701 INFO:teuthology.orchestra.run.smithi059.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 300 kB/s | 27 kB 00:00 2024-03-30T01:37:49.727 INFO:teuthology.orchestra.run.smithi059.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 4.2 MB/s | 255 kB 00:00 2024-03-30T01:37:49.760 INFO:teuthology.orchestra.run.smithi059.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2024-03-30T01:37:49.768 INFO:teuthology.orchestra.run.smithi031.stdout:(139/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.6 MB/s | 854 kB 00:00 2024-03-30T01:37:49.785 INFO:teuthology.orchestra.run.smithi059.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2024-03-30T01:37:49.810 INFO:teuthology.orchestra.run.smithi031.stdout:(140/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2024-03-30T01:37:49.810 INFO:teuthology.orchestra.run.smithi059.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2024-03-30T01:37:49.836 INFO:teuthology.orchestra.run.smithi059.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 303 kB/s | 23 kB 00:00 2024-03-30T01:37:49.861 INFO:teuthology.orchestra.run.smithi059.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-03-30T01:37:49.868 INFO:teuthology.orchestra.run.smithi031.stdout:(141/146): librbd1-18.2.2-1018.g470100a4.el8.x8 2.5 MB/s | 3.6 MB 00:01 2024-03-30T01:37:49.893 INFO:teuthology.orchestra.run.smithi031.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 254 kB 00:00 2024-03-30T01:37:49.978 INFO:teuthology.orchestra.run.smithi059.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 126 kB/s | 18 kB 00:00 2024-03-30T01:37:49.985 INFO:teuthology.orchestra.run.smithi031.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.0 MB/s | 711 kB 00:00 2024-03-30T01:37:50.019 INFO:teuthology.orchestra.run.smithi031.stdout:(144/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.6 MB/s | 333 kB 00:00 2024-03-30T01:37:50.054 INFO:teuthology.orchestra.run.smithi059.stdout:(72/146): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2024-03-30T01:37:50.069 INFO:teuthology.orchestra.run.smithi031.stdout:(145/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.3 MB/s | 1.5 MB 00:00 2024-03-30T01:37:50.071 INFO:teuthology.orchestra.run.smithi059.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 204 kB/s | 43 kB 00:00 2024-03-30T01:37:50.144 INFO:teuthology.orchestra.run.smithi031.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.7 MB/s | 1.1 MB 00:00 2024-03-30T01:37:50.148 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-03-30T01:37:50.148 INFO:teuthology.orchestra.run.smithi031.stdout:Total 16 MB/s | 213 MB 00:13 2024-03-30T01:37:50.154 INFO:teuthology.orchestra.run.smithi059.stdout:(74/146): python3-oauthlib-2.1.0-1.el8.noarch.r 883 kB/s | 155 kB 00:00 2024-03-30T01:37:50.204 INFO:teuthology.orchestra.run.smithi059.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 320 kB/s | 43 kB 00:00 2024-03-30T01:37:50.230 INFO:teuthology.orchestra.run.smithi059.stdout:(76/146): python3-pycparser-2.14-14.el8.noarch. 621 kB/s | 109 kB 00:00 2024-03-30T01:37:50.297 INFO:teuthology.orchestra.run.smithi059.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 131 kB/s | 19 kB 00:00 2024-03-30T01:37:50.355 INFO:teuthology.orchestra.run.smithi059.stdout:(78/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 291 kB/s | 44 kB 00:00 2024-03-30T01:37:50.397 INFO:teuthology.orchestra.run.smithi059.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2024-03-30T01:37:50.431 INFO:teuthology.orchestra.run.smithi059.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2024-03-30T01:37:50.506 INFO:teuthology.orchestra.run.smithi059.stdout:(81/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-30T01:37:50.539 INFO:teuthology.orchestra.run.smithi059.stdout:(82/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 443 kB/s | 48 kB 00:00 2024-03-30T01:37:50.590 INFO:teuthology.orchestra.run.smithi059.stdout:(83/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.0 MB/s | 200 kB 00:00 2024-03-30T01:37:50.615 INFO:teuthology.orchestra.run.smithi059.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 800 kB/s | 87 kB 00:00 2024-03-30T01:37:50.657 INFO:teuthology.orchestra.run.smithi059.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 492 kB/s | 58 kB 00:00 2024-03-30T01:37:50.799 INFO:teuthology.orchestra.run.smithi059.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 185 kB/s | 39 kB 00:00 2024-03-30T01:37:50.999 INFO:teuthology.orchestra.run.smithi059.stdout:(87/146): fuse-2.9.7-17.el8.x86_64.rpm 216 kB/s | 83 kB 00:00 2024-03-30T01:37:51.024 INFO:teuthology.orchestra.run.smithi059.stdout:(88/146): libquadmath-8.5.0-20.el8.x86_64.rpm 765 kB/s | 172 kB 00:00 2024-03-30T01:37:51.058 INFO:teuthology.orchestra.run.smithi059.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 1.6 MB/s | 645 kB 00:00 2024-03-30T01:37:51.117 INFO:teuthology.orchestra.run.smithi059.stdout:(90/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 851 kB/s | 100 kB 00:00 2024-03-30T01:37:51.175 INFO:teuthology.orchestra.run.smithi059.stdout:(91/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-03-30T01:37:51.292 INFO:teuthology.orchestra.run.smithi059.stdout:(92/146): elfutils-libelf-devel-0.189-3.el8.x86 529 kB/s | 62 kB 00:00 2024-03-30T01:37:51.334 INFO:teuthology.orchestra.run.smithi059.stdout:(93/146): NetworkManager-initscripts-updown-1.4 663 kB/s | 144 kB 00:00 2024-03-30T01:37:51.348 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-03-30T01:37:51.359 INFO:teuthology.orchestra.run.smithi059.stdout:(94/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 241 kB/s | 73 kB 00:00 2024-03-30T01:37:51.552 INFO:teuthology.orchestra.run.smithi059.stdout:(95/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-03-30T01:37:51.585 INFO:teuthology.orchestra.run.smithi059.stdout:(96/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 1.9 MB/s | 562 kB 00:00 2024-03-30T01:37:51.627 INFO:teuthology.orchestra.run.smithi059.stdout:(97/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 176 kB/s | 47 kB 00:00 2024-03-30T01:37:51.669 INFO:teuthology.orchestra.run.smithi059.stdout:(98/146): python3-cryptography-3.2.1-7.el8_9.x8 4.7 MB/s | 559 kB 00:00 2024-03-30T01:37:51.694 INFO:teuthology.orchestra.run.smithi059.stdout:(99/146): python3-packaging-16.8-9.el8.noarch.r 443 kB/s | 48 kB 00:00 2024-03-30T01:37:51.761 INFO:teuthology.orchestra.run.smithi059.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 149 kB/s | 20 kB 00:00 2024-03-30T01:37:51.798 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-03-30T01:37:51.798 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-03-30T01:37:51.811 INFO:teuthology.orchestra.run.smithi059.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 309 kB/s | 44 kB 00:00 2024-03-30T01:37:51.870 INFO:teuthology.orchestra.run.smithi059.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 626 kB/s | 110 kB 00:00 2024-03-30T01:37:51.954 INFO:teuthology.orchestra.run.smithi059.stdout:(103/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 890 kB/s | 126 kB 00:00 2024-03-30T01:37:51.979 INFO:teuthology.orchestra.run.smithi059.stdout:(104/146): python3-jsonpatch-1.21-2.el8.noarch. 122 kB/s | 27 kB 00:00 2024-03-30T01:37:52.046 INFO:teuthology.orchestra.run.smithi059.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 2.5 MB/s | 457 kB 00:00 2024-03-30T01:37:52.121 INFO:teuthology.orchestra.run.smithi059.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2024-03-30T01:37:52.155 INFO:teuthology.orchestra.run.smithi059.stdout:(107/146): bison-3.0.4-10.el8.x86_64.rpm 3.4 MB/s | 688 kB 00:00 2024-03-30T01:37:52.205 INFO:teuthology.orchestra.run.smithi059.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2024-03-30T01:37:52.230 INFO:teuthology.orchestra.run.smithi059.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 356 kB/s | 39 kB 00:00 2024-03-30T01:37:52.255 INFO:teuthology.orchestra.run.smithi059.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.5 MB/s | 259 kB 00:00 2024-03-30T01:37:52.297 INFO:teuthology.orchestra.run.smithi059.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2024-03-30T01:37:52.398 INFO:teuthology.orchestra.run.smithi059.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 619 kB/s | 103 kB 00:00 2024-03-30T01:37:52.464 INFO:teuthology.orchestra.run.smithi059.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 3.1 MB/s | 538 kB 00:00 2024-03-30T01:37:52.607 INFO:teuthology.orchestra.run.smithi059.stdout:(114/146): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2024-03-30T01:37:52.623 INFO:teuthology.orchestra.run.smithi059.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2024-03-30T01:37:52.682 INFO:teuthology.orchestra.run.smithi059.stdout:(116/146): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2024-03-30T01:37:52.724 INFO:teuthology.orchestra.run.smithi059.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.9 MB/s | 225 kB 00:00 2024-03-30T01:37:52.757 INFO:teuthology.orchestra.run.smithi059.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2024-03-30T01:37:52.891 INFO:teuthology.orchestra.run.smithi059.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 25 MB/s | 5.2 MB 00:00 2024-03-30T01:37:52.916 INFO:teuthology.orchestra.run.smithi059.stdout:(120/146): python3-mako-1.0.6-14.el8.noarch.rpm 817 kB/s | 157 kB 00:00 2024-03-30T01:37:53.033 INFO:teuthology.orchestra.run.smithi059.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 18 MB/s | 5.0 MB 00:00 2024-03-30T01:37:53.067 INFO:teuthology.orchestra.run.smithi059.stdout:(122/146): automake-1.16.1-8.el8.noarch.rpm 4.0 MB/s | 714 kB 00:00 2024-03-30T01:37:53.092 INFO:teuthology.orchestra.run.smithi059.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-03-30T01:37:53.107 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-03-30T01:37:53.119 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-03-30T01:37:53.151 INFO:teuthology.orchestra.run.smithi059.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2024-03-30T01:37:53.209 INFO:teuthology.orchestra.run.smithi059.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 381 kB/s | 54 kB 00:00 2024-03-30T01:37:53.251 INFO:teuthology.orchestra.run.smithi059.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-03-30T01:37:53.401 INFO:teuthology.orchestra.run.smithi059.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 7.9 MB/s | 1.5 MB 00:00 2024-03-30T01:37:53.443 INFO:teuthology.orchestra.run.smithi059.stdout:(128/146): postgresql-contrib-10.23-4.module+el 4.1 MB/s | 812 kB 00:00 2024-03-30T01:37:53.594 INFO:teuthology.orchestra.run.smithi059.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 27 MB/s | 14 MB 00:00 2024-03-30T01:37:53.727 INFO:teuthology.orchestra.run.smithi059.stdout:(130/146): postgresql-server-10.23-4.module+el8 16 MB/s | 5.1 MB 00:00 2024-03-30T01:37:53.919 INFO:teuthology.orchestra.run.smithi059.stdout:(131/146): librados2-18.2.2-1018.g470100a4.el8. 7.7 MB/s | 3.7 MB 00:00 2024-03-30T01:37:53.945 INFO:teuthology.orchestra.run.smithi059.stdout:(132/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.0 MB/s | 232 kB 00:00 2024-03-30T01:37:54.044 INFO:teuthology.orchestra.run.smithi059.stdout:(133/146): libmount-2.32.1-43.el8.x86_64.rpm 2.3 MB/s | 237 kB 00:00 2024-03-30T01:37:54.078 INFO:teuthology.orchestra.run.smithi059.stdout:(134/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.9 MB/s | 303 kB 00:00 2024-03-30T01:37:54.170 INFO:teuthology.orchestra.run.smithi059.stdout:(135/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.4 MB/s | 179 kB 00:00 2024-03-30T01:37:54.195 INFO:teuthology.orchestra.run.smithi059.stdout:(136/146): libuuid-2.32.1-43.el8.x86_64.rpm 847 kB/s | 99 kB 00:00 2024-03-30T01:37:54.371 INFO:teuthology.orchestra.run.smithi059.stdout:(137/146): util-linux-2.32.1-43.el8.x86_64.rpm 12 MB/s | 2.5 MB 00:00 2024-03-30T01:37:54.421 INFO:teuthology.orchestra.run.smithi059.stdout:(138/146): librbd1-18.2.2-1018.g470100a4.el8.x8 4.3 MB/s | 3.6 MB 00:00 2024-03-30T01:37:54.471 INFO:teuthology.orchestra.run.smithi059.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 372 kB/s | 103 kB 00:00 2024-03-30T01:37:54.513 INFO:teuthology.orchestra.run.smithi059.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2024-03-30T01:37:54.563 INFO:teuthology.orchestra.run.smithi059.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2024-03-30T01:37:54.597 INFO:teuthology.orchestra.run.smithi059.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 254 kB 00:00 2024-03-30T01:37:54.681 INFO:teuthology.orchestra.run.smithi059.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.2 MB/s | 711 kB 00:00 2024-03-30T01:37:54.731 INFO:teuthology.orchestra.run.smithi059.stdout:(144/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.8 MB/s | 1.5 MB 00:00 2024-03-30T01:37:54.773 INFO:teuthology.orchestra.run.smithi059.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2024-03-30T01:37:54.865 INFO:teuthology.orchestra.run.smithi059.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-03-30T01:37:54.869 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-03-30T01:37:54.869 INFO:teuthology.orchestra.run.smithi059.stdout:Total 15 MB/s | 213 MB 00:14 2024-03-30T01:37:56.004 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-03-30T01:37:56.453 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-03-30T01:37:56.453 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-03-30T01:37:57.768 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-03-30T01:37:57.780 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-03-30T01:37:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-03-30T01:37:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-30T01:38:00.097 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T01:38:00.361 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T01:38:00.565 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-30T01:38:00.729 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T01:38:00.814 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T01:38:01.005 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T01:38:01.084 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T01:38:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-30T01:38:01.482 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-03-30T01:38:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-30T01:38:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-30T01:38:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T01:38:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T01:38:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T01:38:03.032 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T01:38:03.246 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T01:38:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T01:38:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-03-30T01:38:03.554 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T01:38:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-30T01:38:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T01:38:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T01:38:03.965 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T01:38:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T01:38:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T01:38:04.304 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T01:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-30T01:38:04.472 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-30T01:38:04.490 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T01:38:04.568 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T01:38:04.694 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-30T01:38:04.757 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T01:38:04.845 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T01:38:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-30T01:38:05.029 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-30T01:38:05.234 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-03-30T01:38:05.399 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-30T01:38:05.506 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-30T01:38:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-30T01:38:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T01:38:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-30T01:38:05.896 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T01:38:06.036 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T01:38:06.168 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T01:38:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T01:38:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T01:38:06.356 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T01:38:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-03-30T01:38:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-03-30T01:38:06.784 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T01:38:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-03-30T01:38:07.115 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T01:38:07.145 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-30T01:38:07.250 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T01:38:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-30T01:38:07.423 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T01:38:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-30T01:38:07.526 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T01:38:07.809 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T01:38:08.056 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T01:38:08.149 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T01:38:08.199 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-30T01:38:08.342 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-30T01:38:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-30T01:38:08.563 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-30T01:38:08.887 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-30T01:38:09.061 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-30T01:38:09.245 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-03-30T01:38:09.275 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-30T01:38:09.539 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-30T01:38:09.706 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T01:38:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T01:38:10.032 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-03-30T01:38:10.098 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T01:38:10.179 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T01:38:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-03-30T01:38:10.363 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-03-30T01:38:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-03-30T01:38:10.691 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-03-30T01:38:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-30T01:38:10.896 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-03-30T01:38:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 31/162 2024-03-30T01:38:11.089 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-30T01:38:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 32/162 2024-03-30T01:38:11.236 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-30T01:38:11.414 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-30T01:38:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 33/162 2024-03-30T01:38:11.886 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-30T01:38:12.091 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-30T01:38:12.689 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-30T01:38:13.388 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-30T01:38:13.622 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-03-30T01:38:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-30T01:38:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-03-30T01:38:14.397 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-03-30T01:38:14.655 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-03-30T01:38:14.864 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-03-30T01:38:15.119 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-30T01:38:15.305 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 31/162 2024-03-30T01:38:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-03-30T01:38:15.436 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 32/162 2024-03-30T01:38:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-03-30T01:38:15.733 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 33/162 2024-03-30T01:38:15.930 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-30T01:38:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-03-30T01:38:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-30T01:38:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-30T01:38:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 41/162 2024-03-30T01:38:16.844 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 42/162 2024-03-30T01:38:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T01:38:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T01:38:17.344 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T01:38:17.420 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T01:38:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-30T01:38:17.900 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-30T01:38:17.903 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-30T01:38:18.163 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T01:38:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-03-30T01:38:18.711 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T01:38:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T01:38:19.101 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T01:38:19.293 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-30T01:38:19.373 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-03-30T01:38:19.376 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-30T01:38:19.678 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-03-30T01:38:19.716 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-03-30T01:38:19.902 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-03-30T01:38:19.971 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-03-30T01:38:20.093 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-03-30T01:38:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-03-30T01:38:20.191 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-30T01:38:20.329 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-03-30T01:38:20.379 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-30T01:38:20.552 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-03-30T01:38:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 41/162 2024-03-30T01:38:20.773 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-03-30T01:38:20.830 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 42/162 2024-03-30T01:38:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-03-30T01:38:21.038 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T01:38:21.120 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T01:38:21.233 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-03-30T01:38:21.330 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T01:38:21.404 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T01:38:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-03-30T01:38:21.625 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-03-30T01:38:21.656 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-30T01:38:21.809 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-03-30T01:38:21.875 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-30T01:38:21.982 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-03-30T01:38:22.087 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T01:38:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-03-30T01:38:22.570 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T01:38:22.605 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-03-30T01:38:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-03-30T01:38:22.889 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T01:38:22.920 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T01:38:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-03-30T01:38:23.146 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-30T01:38:23.228 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-03-30T01:38:23.234 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-30T01:38:23.532 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-03-30T01:38:23.644 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-03-30T01:38:23.750 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-03-30T01:38:23.980 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-03-30T01:38:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-03-30T01:38:24.052 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-03-30T01:38:24.230 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-03-30T01:38:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-03-30T01:38:24.430 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-03-30T01:38:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-03-30T01:38:24.618 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-03-30T01:38:24.715 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-03-30T01:38:24.833 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-03-30T01:38:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-03-30T01:38:25.060 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-03-30T01:38:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-03-30T01:38:25.289 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-03-30T01:38:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-03-30T01:38:25.486 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-03-30T01:38:25.676 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-03-30T01:38:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-03-30T01:38:25.882 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-03-30T01:38:25.885 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-03-30T01:38:26.142 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-03-30T01:38:26.206 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-03-30T01:38:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-30T01:38:26.514 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-30T01:38:26.516 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-03-30T01:38:26.735 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-03-30T01:38:26.779 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-30T01:38:26.915 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-03-30T01:38:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-30T01:38:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-03-30T01:38:27.205 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-30T01:38:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-30T01:38:27.541 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-03-30T01:38:27.647 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-30T01:38:27.887 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-03-30T01:38:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-30T01:38:28.139 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-30T01:38:28.274 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-03-30T01:38:28.331 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-30T01:38:28.488 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-03-30T01:38:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-30T01:38:28.677 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-03-30T01:38:28.692 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-30T01:38:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-30T01:38:28.938 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-03-30T01:38:29.149 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-30T01:38:29.181 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-03-30T01:38:29.390 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-03-30T01:38:29.437 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-30T01:38:29.624 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-03-30T01:38:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-30T01:38:29.810 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-03-30T01:38:29.926 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-30T01:38:30.040 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-03-30T01:38:30.179 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-30T01:38:30.214 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-30T01:38:30.395 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-30T01:38:30.684 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-30T01:38:30.878 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-30T01:38:31.084 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-30T01:38:31.291 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-30T01:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-30T01:38:31.517 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-30T01:38:31.735 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-30T01:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-30T01:38:31.951 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-30T01:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-30T01:38:32.141 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-30T01:38:32.285 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-30T01:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-30T01:38:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-30T01:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-30T01:38:32.750 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-30T01:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-30T01:38:32.982 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-30T01:38:33.258 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-30T01:38:33.479 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-30T01:38:33.679 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-30T01:38:33.685 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-30T01:38:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T01:38:33.903 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T01:38:33.916 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-30T01:38:34.161 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 104/162 2024-03-30T01:38:35.086 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-30T01:38:35.423 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-30T01:38:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-30T01:38:36.021 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-30T01:38:36.323 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-30T01:38:36.516 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-30T01:38:37.357 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-30T01:38:37.502 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T01:38:37.584 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T01:38:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 104/162 2024-03-30T01:38:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T01:38:38.674 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T01:38:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T01:38:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T01:38:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T01:38:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T01:38:42.318 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T01:38:42.561 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T01:38:43.220 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T01:38:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T01:38:43.591 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T01:38:43.752 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /run 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2024-03-30T01:39:02.759 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:39:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T01:39:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T01:39:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T01:39:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T01:39:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T01:39:06.102 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T01:39:06.102 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-03-30T01:39:06.102 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-03-30T01:39:06.102 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2024-03-30T01:39:06.102 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /run 2024-03-30T01:39:06.102 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-03-30T01:39:06.103 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-03-30T01:39:06.103 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-03-30T01:39:06.103 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-03-30T01:39:06.103 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2024-03-30T01:39:06.103 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:39:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T01:39:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T01:39:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T01:39:06.450 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T01:39:08.249 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T01:39:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T01:39:08.445 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T01:39:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T01:39:08.676 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 113/162 2024-03-30T01:39:08.690 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T01:39:08.841 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T01:39:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 114/162 2024-03-30T01:39:09.161 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T01:39:09.366 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T01:39:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T01:39:09.524 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T01:39:09.546 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T01:39:09.693 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T01:39:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T01:39:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T01:39:11.039 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T01:39:11.438 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 118/162 2024-03-30T01:39:11.520 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T01:39:11.619 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T01:39:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T01:39:11.731 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T01:39:11.843 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T01:39:11.859 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T01:39:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 113/162 2024-03-30T01:39:12.078 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 114/162 2024-03-30T01:39:12.337 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T01:39:12.584 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T01:39:12.885 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T01:39:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 121/162 2024-03-30T01:39:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T01:39:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T01:39:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T01:39:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T01:39:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T01:39:13.959 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T01:39:14.114 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T01:39:14.226 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T01:39:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T01:39:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T01:39:14.588 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 118/162 2024-03-30T01:39:14.753 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T01:39:14.798 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T01:39:14.956 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T01:39:14.971 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T01:39:16.354 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 121/162 2024-03-30T01:39:16.552 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T01:39:16.593 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T01:39:16.757 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T01:39:16.807 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T01:39:16.973 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T01:39:17.359 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T01:39:17.523 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T01:39:19.358 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T01:39:20.454 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 126/162 2024-03-30T01:39:20.766 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-30T01:39:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-03-30T01:39:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 129/162 2024-03-30T01:39:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-03-30T01:39:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-03-30T01:39:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-03-30T01:39:22.316 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-30T01:39:22.466 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T01:39:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-30T01:39:22.692 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 134/162 2024-03-30T01:39:23.549 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 126/162 2024-03-30T01:39:23.820 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-30T01:39:24.138 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-03-30T01:39:24.554 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 129/162 2024-03-30T01:39:24.900 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-03-30T01:39:25.080 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-03-30T01:39:25.097 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-03-30T01:39:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T01:39:25.419 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-30T01:39:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T01:39:25.577 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T01:39:25.619 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-30T01:39:25.789 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 134/162 2024-03-30T01:39:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-30T01:39:26.430 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-30T01:39:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-30T01:39:26.817 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 139/162 2024-03-30T01:39:27.051 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 140/162 2024-03-30T01:39:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 141/162 2024-03-30T01:39:27.708 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-30T01:39:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-03-30T01:39:28.053 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-03-30T01:39:28.257 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-03-30T01:39:28.268 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-03-30T01:39:28.451 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-30T01:39:28.463 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-30T01:39:28.518 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T01:39:28.593 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-03-30T01:39:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-30T01:39:28.806 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T01:39:28.858 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T01:39:28.995 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-30T01:39:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-30T01:39:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T01:39:29.286 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-30T01:39:29.404 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T01:39:29.404 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T01:39:29.625 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-30T01:39:29.627 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T01:39:29.627 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T01:39:29.860 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-30T01:39:29.913 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T01:39:30.048 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 139/162 2024-03-30T01:39:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-30T01:39:30.265 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 140/162 2024-03-30T01:39:30.288 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-30T01:39:30.527 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-30T01:39:30.528 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T01:39:30.551 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 141/162 2024-03-30T01:39:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T01:39:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T01:39:30.812 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-30T01:39:30.993 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T01:39:30.993 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T01:39:31.012 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-03-30T01:39:31.100 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-03-30T01:39:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T01:39:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T01:39:31.329 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-03-30T01:39:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-03-30T01:39:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T01:39:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T01:39:31.529 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-30T01:39:31.541 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-30T01:39:31.699 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-03-30T01:39:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T01:39:31.872 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-30T01:39:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-30T01:39:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T01:39:32.090 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-30T01:39:32.233 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-30T01:39:32.234 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T01:39:32.491 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T01:39:32.491 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T01:39:32.700 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T01:39:32.700 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T01:39:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 1/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 3/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 4/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1018.g47010 5/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 6/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 7/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 8/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T01:39:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 13/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 14/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 15/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 16/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 17/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 18/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 20/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 21/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 22/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 23/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 24/162 2024-03-30T01:39:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 25/162 2024-03-30T01:39:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 26/162 2024-03-30T01:39:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 27/162 2024-03-30T01:39:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 28/162 2024-03-30T01:39:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 29/162 2024-03-30T01:39:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 30/162 2024-03-30T01:39:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 31/162 2024-03-30T01:39:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 32/162 2024-03-30T01:39:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 33/162 2024-03-30T01:39:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 34/162 2024-03-30T01:39:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 35/162 2024-03-30T01:39:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 36/162 2024-03-30T01:39:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 37/162 2024-03-30T01:39:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 38/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-30T01:39:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-30T01:39:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-03-30T01:39:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 93/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 94/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 95/162 2024-03-30T01:39:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 96/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 97/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 98/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-03-30T01:39:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-03-30T01:39:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 133/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-30T01:39:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-30T01:39:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-30T01:39:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-30T01:39:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-30T01:39:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-30T01:39:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-30T01:39:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-30T01:39:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-30T01:39:32.970 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T01:39:33.189 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-30T01:39:33.375 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-30T01:39:33.598 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-30T01:39:33.598 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T01:39:33.874 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T01:39:33.874 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T01:39:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T01:39:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T01:39:34.323 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T01:39:34.324 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T01:39:34.559 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T01:39:34.559 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T01:39:34.790 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T01:39:34.932 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-30T01:39:34.932 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T01:39:35.937 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T01:39:35.937 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 1/162 2024-03-30T01:39:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2/162 2024-03-30T01:39:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 3/162 2024-03-30T01:39:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 4/162 2024-03-30T01:39:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1018.g47010 5/162 2024-03-30T01:39:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 6/162 2024-03-30T01:39:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 7/162 2024-03-30T01:39:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 8/162 2024-03-30T01:39:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T01:39:35.938 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 13/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 14/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 15/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 16/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 17/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 18/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 20/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 21/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 22/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 23/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 24/162 2024-03-30T01:39:35.939 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 25/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 26/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 27/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 28/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 29/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 30/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 31/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 32/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 33/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 34/162 2024-03-30T01:39:35.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 35/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 36/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 37/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 38/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-30T01:39:35.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-30T01:39:35.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-03-30T01:39:35.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-03-30T01:39:35.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 93/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 94/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 95/162 2024-03-30T01:39:35.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 96/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 97/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 98/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-03-30T01:39:35.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-03-30T01:39:35.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 133/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-30T01:39:35.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-30T01:39:35.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-30T01:39:35.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-30T01:39:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-30T01:39:50.356 INFO:teuthology.orchestra.run.smithi031.stdout:Installed products updated. 2024-03-30T01:39:51.053 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:39:51.053 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-03-30T01:39:51.053 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-30T01:39:51.053 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-30T01:39:51.053 INFO:teuthology.orchestra.run.smithi031.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-30T01:39:51.053 INFO:teuthology.orchestra.run.smithi031.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-30T01:39:51.054 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.1-8.el8.noarch 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.0.4-10.el8.x86_64 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.055 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.1-9.el8.x86_64 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-30T01:39:51.056 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: jq-1.6-7.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: libpq-13.11-1.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.057 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-30T01:39:51.058 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-30T01:39:51.059 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-30T01:39:51.060 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-2.6-1.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.061 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-30T01:39:51.062 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:39:51.063 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-03-30T01:39:51.446 DEBUG:teuthology.parallel:result is None 2024-03-30T01:39:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-30T01:39:53.780 INFO:teuthology.orchestra.run.smithi059.stdout:Installed products updated. 2024-03-30T01:39:54.972 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:39:54.972 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-03-30T01:39:54.972 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-30T01:39:54.972 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-30T01:39:54.972 INFO:teuthology.orchestra.run.smithi059.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-30T01:39:54.972 INFO:teuthology.orchestra.run.smithi059.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-30T01:39:54.972 INFO:teuthology.orchestra.run.smithi059.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-30T01:39:54.972 INFO:teuthology.orchestra.run.smithi059.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-30T01:39:54.972 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.1-8.el8.noarch 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.0.4-10.el8.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.973 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.974 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.1-9.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: jq-1.6-7.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-30T01:39:54.975 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: libpq-13.11-1.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-30T01:39:54.977 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-30T01:39:54.978 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-30T01:39:54.979 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-2.6-1.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-30T01:39:54.980 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.981 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T01:39:54.982 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-30T01:39:54.982 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-30T01:39:54.982 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-30T01:39:54.982 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-30T01:39:54.982 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-30T01:39:54.982 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-30T01:39:54.982 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:39:54.982 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-03-30T01:39:56.149 DEBUG:teuthology.parallel:result is None 2024-03-30T01:39:56.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T01:39:56.289 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T01:39:56.320 INFO:teuthology.orchestra.run.smithi031.stdout:18.2.2-1018.g470100a4.el8 2024-03-30T01:39:56.320 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018.g470100a4.el8 2024-03-30T01:39:56.320 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018.g470100a4 is installed. 2024-03-30T01:39:56.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T01:39:56.467 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T01:39:56.499 INFO:teuthology.orchestra.run.smithi059.stdout:18.2.2-1018.g470100a4.el8 2024-03-30T01:39:56.499 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018.g470100a4.el8 2024-03-30T01:39:56.499 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018.g470100a4 is installed. 2024-03-30T01:39:56.501 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-30T01:39:56.501 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:39:56.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T01:39:56.555 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:39:56.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T01:39:56.608 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-30T01:39:56.608 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:39:56.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T01:39:56.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T01:39:56.718 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:39:56.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T01:39:56.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T01:39:56.828 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-30T01:39:56.829 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:39:56.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T01:39:56.863 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T01:39:56.937 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:39:56.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T01:39:56.971 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T01:39:57.046 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-30T01:39:57.046 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:39:57.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T01:39:57.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T01:39:57.155 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:39:57.155 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T01:39:57.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T01:39:57.265 INFO:teuthology.run_tasks:Running task ceph... 2024-03-30T01:39:57.343 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-30T01:39:57.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-03-30T01:39:57.347 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /var/log/ceph 2024-03-30T01:39:57.381 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-30T01:39:57.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-30T01:39:57.420 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-30T01:39:57.455 INFO:tasks.ceph:Creating extra log directories... 2024-03-30T01:39:57.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-30T01:39:57.495 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-30T01:39:57.536 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-30T01:39:57.536 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-30T01:39:57.537 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-30T01:39:57.537 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:39:57.537 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-30T01:39:57.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-30T01:39:57.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-30T01:39:57.746 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:39:57.746 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-30T01:39:57.767 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-30T01:39:57.891 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-30T01:39:57.930 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-30T01:39:57.930 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-30T01:39:57.931 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629316', 'branch': 'wip-rishabh-testing-20240327.051042-reef', 'description': 'fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}', 'email': None, 'first_in_suite': False, 'job_id': '7629316', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240327.051042-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240327.051042-reef', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 8052, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'sleep_before_teardown': 0, 'subset': '497/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240327.051042-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBk7UkPPupy5hguDJvEkH8EUjYZrclRc4Pzj7u0lFqnlJ0AxhsJzhyHneA6dkbFuycMKrdvxhr+4QslNGpeHyng=', 'smithi059.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBACPKumA6RhDjM0xJarFiqwgHezSGIpTCARRsLgrPgL7MtPPobgvaUhhNBuaTuTx+ubiaFwiJHnStvdh03dHPlY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_multifs_auth']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-29_18:05:24', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-30T01:39:57.931 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-30T01:39:57.933 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-30T01:39:57.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-03-30T01:39:57.995 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/run/ceph 2024-03-30T01:39:58.062 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:39:58.062 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T01:39:58.088 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T01:39:58.089 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-03-30T01:39:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T01:39:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T01:39:58.207 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 179894 Links: 1 2024-03-30T01:39:58.207 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T01:39:58.207 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T01:39:58.207 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-03-30 01:39:32.469300246 +0000 2024-03-30T01:39:58.207 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-03-30 01:35:56.370104226 +0000 2024-03-30T01:39:58.207 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-03-30 01:35:56.370104226 +0000 2024-03-30T01:39:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-03-30T01:39:58.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T01:39:58.269 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-03-30T01:39:58.269 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-03-30T01:39:58.269 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000282418 s, 1.8 MB/s 2024-03-30T01:39:58.270 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T01:39:58.293 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-03-30T01:39:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T01:39:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T01:39:58.355 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 181370 Links: 1 2024-03-30T01:39:58.355 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T01:39:58.355 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T01:39:58.355 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-03-30 01:39:32.469300246 +0000 2024-03-30T01:39:58.355 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-03-30 01:35:56.761097975 +0000 2024-03-30T01:39:58.355 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-03-30 01:35:56.761097975 +0000 2024-03-30T01:39:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-03-30T01:39:58.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T01:39:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-03-30T01:39:58.528 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-03-30T01:39:58.528 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000231585 s, 2.2 MB/s 2024-03-30T01:39:58.529 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T01:39:58.594 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-03-30T01:39:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T01:39:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T01:39:58.657 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 177919 Links: 1 2024-03-30T01:39:58.657 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T01:39:58.657 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T01:39:58.657 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-03-30 01:39:32.469300246 +0000 2024-03-30T01:39:58.657 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-03-30 01:35:57.136091979 +0000 2024-03-30T01:39:58.657 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-03-30 01:35:57.136091979 +0000 2024-03-30T01:39:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-03-30T01:39:58.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T01:39:58.730 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-03-30T01:39:58.730 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-03-30T01:39:58.730 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000212022 s, 2.4 MB/s 2024-03-30T01:39:58.731 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T01:39:58.794 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-03-30T01:39:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T01:39:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T01:39:58.857 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 181452 Links: 1 2024-03-30T01:39:58.857 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T01:39:58.858 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T01:39:58.858 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-03-30 01:39:32.469300246 +0000 2024-03-30T01:39:58.858 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-03-30 01:35:57.511085984 +0000 2024-03-30T01:39:58.858 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-03-30 01:35:57.511085984 +0000 2024-03-30T01:39:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-03-30T01:39:58.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T01:39:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-03-30T01:39:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-03-30T01:39:58.930 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000232652 s, 2.2 MB/s 2024-03-30T01:39:58.931 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T01:39:58.994 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T01:39:58.995 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:39:58.995 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T01:39:59.016 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T01:39:59.017 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-03-30T01:39:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T01:39:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T01:39:59.079 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 178452 Links: 1 2024-03-30T01:39:59.079 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T01:39:59.079 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T01:39:59.079 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-03-30 01:39:35.478289306 +0000 2024-03-30T01:39:59.079 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-03-30 01:36:03.166675382 +0000 2024-03-30T01:39:59.079 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-03-30 01:36:03.166675382 +0000 2024-03-30T01:39:59.079 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2024-03-30T01:39:59.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T01:39:59.153 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-03-30T01:39:59.153 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-03-30T01:39:59.153 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000310033 s, 1.7 MB/s 2024-03-30T01:39:59.154 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T01:39:59.218 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-03-30T01:39:59.281 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T01:39:59.281 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T01:39:59.281 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 174911 Links: 1 2024-03-30T01:39:59.281 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T01:39:59.281 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T01:39:59.281 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-03-30 01:39:35.478289306 +0000 2024-03-30T01:39:59.281 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-03-30 01:36:03.549670390 +0000 2024-03-30T01:39:59.281 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-03-30 01:36:03.549670390 +0000 2024-03-30T01:39:59.281 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2024-03-30T01:39:59.282 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T01:39:59.353 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-03-30T01:39:59.353 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-03-30T01:39:59.353 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000230888 s, 2.2 MB/s 2024-03-30T01:39:59.355 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T01:39:59.418 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-03-30T01:39:59.480 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T01:39:59.480 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T01:39:59.480 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 183404 Links: 1 2024-03-30T01:39:59.480 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T01:39:59.480 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T01:39:59.480 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-03-30 01:39:35.478289306 +0000 2024-03-30T01:39:59.480 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-03-30 01:36:03.914665632 +0000 2024-03-30T01:39:59.480 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-03-30 01:36:03.914665632 +0000 2024-03-30T01:39:59.480 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2024-03-30T01:39:59.481 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T01:39:59.553 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-03-30T01:39:59.553 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-03-30T01:39:59.553 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000208922 s, 2.5 MB/s 2024-03-30T01:39:59.555 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T01:39:59.617 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-03-30T01:39:59.679 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T01:39:59.680 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T01:39:59.680 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 179528 Links: 1 2024-03-30T01:39:59.680 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T01:39:59.680 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T01:39:59.680 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-03-30 01:39:35.478289306 +0000 2024-03-30T01:39:59.680 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-03-30 01:36:04.288660757 +0000 2024-03-30T01:39:59.680 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-03-30 01:36:04.288660757 +0000 2024-03-30T01:39:59.680 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2024-03-30T01:39:59.680 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T01:39:59.752 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-03-30T01:39:59.752 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-03-30T01:39:59.753 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000279068 s, 1.8 MB/s 2024-03-30T01:39:59.754 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T01:39:59.816 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T01:39:59.817 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-30T01:39:59.817 INFO:tasks.ceph:Generating config... 2024-03-30T01:39:59.818 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-30T01:39:59.818 INFO:tasks.ceph:[client] debug client = 20 2024-03-30T01:39:59.818 INFO:tasks.ceph:[client] debug ms = 1 2024-03-30T01:39:59.818 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-30T01:39:59.818 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-30T01:39:59.818 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-30T01:39:59.818 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-30T01:39:59.818 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-30T01:39:59.819 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-30T01:39:59.820 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-30T01:39:59.820 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-30T01:39:59.820 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-30T01:39:59.820 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-30T01:39:59.821 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-30T01:39:59.821 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-30T01:39:59.821 INFO:tasks.ceph:Setting up mon.a... 2024-03-30T01:39:59.821 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-30T01:39:59.879 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2024-03-30T01:39:59.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-30T01:39:59.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T01:39:59.967 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.b', '172.21.15.59'), ('mon.c', '[v2:172.21.15.59:3301,v1:172.21.15.59:6790]')] 2024-03-30T01:39:59.968 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-30T01:39:59.968 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:39:59.969 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-30T01:40:00.029 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --add b 172.21.15.59 --addv c '[v2:172.21.15.59:3301,v1:172.21.15.59:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:00.128 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:00.128 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid 9739890c-57ec-4503-9fd1-d4b0145023f4 2024-03-30T01:40:00.128 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = pacific 2024-03-30T01:40:00.128 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2024-03-30T01:40:00.128 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 9739890c-57ec-4503-9fd1-d4b0145023f4 2024-03-30T01:40:00.128 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2024-03-30T01:40:00.123331+0000 2024-03-30T01:40:00.128 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-03-30T01:40:00.123331+0000 2024-03-30T01:40:00.128 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 16 (pacific) 2024-03-30T01:40:00.128 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2024-03-30T01:40:00.129 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-03-30T01:40:00.129 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-03-30T01:40:00.129 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] mon.c 2024-03-30T01:40:00.129 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-30T01:40:00.130 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-30T01:40:00.150 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9739890c-57ec-4503-9fd1-d4b0145023f4... 2024-03-30T01:40:00.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T01:40:00.193 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T01:40:00.249 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-03-30T01:40:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-03-30T01:40:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-03-30T01:40:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790]" 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on skipped message = False 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 9739890c-57ec-4503-9fd1-d4b0145023f4 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-03-30T01:40:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-03-30T01:40:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 20 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 20 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 10 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2024-03-30T01:40:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-03-30T01:40:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: mon op complaint time = 120 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: client mount timeout = 600 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2024-03-30T01:40:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout:[mds] 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds = 20 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds balancer = 20 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug frag = True 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug scatterstat = True 2024-03-30T01:40:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: mds op complaint time = 180 2024-03-30T01:40:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: mds verify scatter = True 2024-03-30T01:40:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2024-03-30T01:40:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2024-03-30T01:40:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2024-03-30T01:40:00.260 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-03-30T01:40:00.260 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2024-03-30T01:40:00.260 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T01:40:00.260 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2024-03-30T01:40:00.260 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T01:40:00.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T01:40:00.262 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.31,172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790]" 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on skipped message = False 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = 9739890c-57ec-4503-9fd1-d4b0145023f4 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2024-03-30T01:40:00.263 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2024-03-30T01:40:00.264 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 20 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 20 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 10 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2024-03-30T01:40:00.265 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2024-03-30T01:40:00.266 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T01:40:00.267 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2024-03-30T01:40:00.268 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2024-03-30T01:40:00.269 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = True 2024-03-30T01:40:00.270 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2024-03-30T01:40:00.270 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = True 2024-03-30T01:40:00.270 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2024-03-30T01:40:00.270 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2024-03-30T01:40:00.270 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2024-03-30T01:40:00.271 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-30T01:40:00.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-30T01:40:00.382 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-30T01:40:00.382 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:00.383 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-30T01:40:00.402 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:00.402 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-30T01:40:00.463 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-03-30T01:40:00.463 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:00.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-30T01:40:00.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T01:40:00.550 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:00.550 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:00.612 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:40:00.612 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:00.612 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-30T01:40:00.612 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T01:40:00.660 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:00.660 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:00.722 INFO:tasks.ceph:Setting up mon nodes... 2024-03-30T01:40:00.722 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-30T01:40:00.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-30T01:40:00.789 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-30T01:40:00.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-30T01:40:00.876 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-30T01:40:00.879 INFO:tasks.ceph:Setting up mds nodes... 2024-03-30T01:40:00.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-30T01:40:00.946 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-30T01:40:00.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-30T01:40:00.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-30T01:40:01.087 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-30T01:40:01.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-30T01:40:01.121 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-30T01:40:01.188 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-30T01:40:01.192 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-30T01:40:01.223 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-30T01:40:01.436 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-30T01:40:01.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-30T01:40:01.471 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-30T01:40:01.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-30T01:40:01.526 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-30T01:40:01.542 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-30T01:40:01.597 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-30T01:40:01.613 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-30T01:40:01.613 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-30T01:40:01.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-30T01:40:01.648 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T01:40:01.648 INFO:tasks.ceph:role: osd.0 2024-03-30T01:40:01.648 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-03-30T01:40:01.648 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-30T01:40:01.729 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T01:40:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T01:40:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T01:40:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T01:40:01.730 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T01:40:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-03-30T01:40:01.730 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T01:40:01.730 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T01:40:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T01:40:01.730 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T01:40:02.161 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-03-30T01:40:02.163 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-03-30T01:40:02.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-30T01:40:02.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-30T01:40:02.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-30T01:40:02.457 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T01:40:02.457 INFO:tasks.ceph:role: osd.1 2024-03-30T01:40:02.457 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-03-30T01:40:02.457 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-30T01:40:02.538 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T01:40:02.539 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T01:40:02.539 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T01:40:02.539 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T01:40:02.539 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T01:40:02.539 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-03-30T01:40:02.539 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T01:40:02.539 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T01:40:02.539 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T01:40:02.539 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T01:40:02.955 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-03-30T01:40:02.958 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-03-30T01:40:02.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-30T01:40:03.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-30T01:40:03.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-30T01:40:03.193 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T01:40:03.193 INFO:tasks.ceph:role: osd.2 2024-03-30T01:40:03.193 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2024-03-30T01:40:03.194 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T01:40:03.277 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T01:40:03.729 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-03-30T01:40:03.731 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-03-30T01:40:03.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-30T01:40:03.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-30T01:40:03.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-30T01:40:03.994 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T01:40:03.994 INFO:tasks.ceph:role: osd.3 2024-03-30T01:40:03.994 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2024-03-30T01:40:03.994 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T01:40:04.077 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T01:40:04.518 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-03-30T01:40:04.520 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-03-30T01:40:04.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-30T01:40:04.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-30T01:40:04.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:04.805+0000 7fed4f9eb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-30T01:40:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:04.805+0000 7fed4f9eb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-30T01:40:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:04.805+0000 7fed4f9eb640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T01:40:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:04.805+0000 7fed4f9eb640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T01:40:04.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:04.805+0000 7fed4f9eb640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T01:40:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:04.806+0000 7fed4f9eb640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-30T01:40:07.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-30T01:40:07.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:07.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:07.797+0000 7fd161194640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-30T01:40:07.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:07.797+0000 7fd161194640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-30T01:40:07.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:07.797+0000 7fd161194640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T01:40:07.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:07.797+0000 7fd161194640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T01:40:07.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:07.797+0000 7fd161194640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T01:40:07.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:07.797+0000 7fd161194640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-30T01:40:10.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-30T01:40:10.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:10.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:10.804+0000 7fee6e3fd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-30T01:40:10.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:10.805+0000 7fee6e3fd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-30T01:40:10.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:10.805+0000 7fee6e3fd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T01:40:10.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:10.805+0000 7fee6e3fd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T01:40:10.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:10.805+0000 7fee6e3fd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T01:40:10.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:10.805+0000 7fee6e3fd640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-30T01:40:13.666 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-30T01:40:13.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:13.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:13.794+0000 7f73dc4fd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-30T01:40:13.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:13.794+0000 7f73dc4fd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-30T01:40:13.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:13.794+0000 7f73dc4fd640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T01:40:13.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:13.794+0000 7f73dc4fd640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T01:40:13.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:13.794+0000 7f73dc4fd640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T01:40:13.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:13.795+0000 7f73dc4fd640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-30T01:40:16.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-30T01:40:16.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-30T01:40:16.731 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T01:40:16.731 INFO:tasks.ceph:role: osd.4 2024-03-30T01:40:16.731 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:40:16.731 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-30T01:40:16.814 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T01:40:16.814 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T01:40:16.814 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T01:40:16.814 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T01:40:16.814 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T01:40:16.815 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-03-30T01:40:16.815 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T01:40:16.815 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T01:40:16.815 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T01:40:16.815 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T01:40:17.245 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-03-30T01:40:17.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-03-30T01:40:17.248 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-30T01:40:17.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-30T01:40:17.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-30T01:40:17.559 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T01:40:17.559 INFO:tasks.ceph:role: osd.5 2024-03-30T01:40:17.559 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:40:17.560 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T01:40:17.642 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T01:40:18.050 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-03-30T01:40:18.052 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-03-30T01:40:18.052 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-30T01:40:18.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-30T01:40:18.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-30T01:40:18.288 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T01:40:18.288 INFO:tasks.ceph:role: osd.6 2024-03-30T01:40:18.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:40:18.288 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-30T01:40:18.372 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T01:40:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T01:40:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T01:40:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T01:40:18.372 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T01:40:18.372 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-03-30T01:40:18.372 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T01:40:18.372 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T01:40:18.373 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T01:40:18.373 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T01:40:18.820 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-03-30T01:40:18.822 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-03-30T01:40:18.823 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-30T01:40:19.006 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-30T01:40:19.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-30T01:40:19.124 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T01:40:19.124 INFO:tasks.ceph:role: osd.7 2024-03-30T01:40:19.124 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:40:19.124 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-30T01:40:19.207 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T01:40:19.207 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T01:40:19.207 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T01:40:19.207 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T01:40:19.208 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T01:40:19.208 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2024-03-30T01:40:19.208 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T01:40:19.208 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T01:40:19.208 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T01:40:19.208 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T01:40:19.644 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2024-03-30T01:40:19.646 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2024-03-30T01:40:19.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-30T01:40:19.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-30T01:40:19.826 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:19.921+0000 7f46f116e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-30T01:40:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:19.921+0000 7f46f116e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-30T01:40:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:19.921+0000 7f46f116e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T01:40:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:19.921+0000 7f46f116e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T01:40:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:19.921+0000 7f46f116e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T01:40:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:19.922+0000 7f46f116e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-30T01:40:22.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-30T01:40:22.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:22.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:22.919+0000 7f0c3abb5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-30T01:40:22.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:22.919+0000 7f0c3abb5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-30T01:40:22.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:22.919+0000 7f0c3abb5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T01:40:22.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:22.919+0000 7f0c3abb5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T01:40:22.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:22.919+0000 7f0c3abb5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T01:40:22.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:22.920+0000 7f0c3abb5640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-30T01:40:25.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-30T01:40:25.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:25.912+0000 7f5a3b2cc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-30T01:40:25.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:25.912+0000 7f5a3b2cc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-30T01:40:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:25.912+0000 7f5a3b2cc640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T01:40:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:25.912+0000 7f5a3b2cc640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T01:40:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:25.912+0000 7f5a3b2cc640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T01:40:25.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:25.913+0000 7f5a3b2cc640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-30T01:40:27.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:40:27.935 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:40:28.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-30T01:40:28.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:28.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:28.923+0000 7ff7d5def640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-30T01:40:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:28.923+0000 7ff7d5def640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-30T01:40:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:28.923+0000 7ff7d5def640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T01:40:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:28.924+0000 7ff7d5def640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T01:40:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:28.924+0000 7ff7d5def640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T01:40:28.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:28.924+0000 7ff7d5def640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-30T01:40:31.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-30T01:40:31.815 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-30T01:40:31.816 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:31.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-30T01:40:31.850 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:31.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-30T01:40:31.925 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:31.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-30T01:40:31.999 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:31.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-30T01:40:32.074 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:32.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-30T01:40:32.149 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:32.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-30T01:40:32.224 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:32.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-30T01:40:32.298 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:32.298 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-30T01:40:32.333 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:32.333 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-30T01:40:32.407 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:32.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-30T01:40:32.482 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:32.482 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-30T01:40:32.558 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:32.558 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-30T01:40:32.632 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:32.632 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-30T01:40:32.707 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:32.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-30T01:40:32.780 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:32.780 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-30T01:40:32.802 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:32.802 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-30T01:40:32.842 INFO:tasks.ceph:Adding keys to all mons... 2024-03-30T01:40:32.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-30T01:40:32.845 DEBUG:teuthology.orchestra.run.smithi059:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-30T01:40:32.886 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2024-03-30T01:40:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBwbQdmSyDMLhAAIk3a6lqgYJDDIrak1TE3Og== 2024-03-30T01:40:32.886 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2024-03-30T01:40:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBwbQdmKS4dOBAAoFJ9d9VJCzRRA6dvzqlvdA== 2024-03-30T01:40:32.886 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2024-03-30T01:40:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBxbQdmy2PsBBAAmQgxB6hsqmzkMOlsiyA2DQ== 2024-03-30T01:40:32.886 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-03-30T01:40:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB0bQdmv2kRMBAA+Z/39yEKUrYr3hMcRw6UtQ== 2024-03-30T01:40:32.886 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB3bQdmgRSYLxAA9DUas+VdQgEmd2nF77YNmw== 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB6bQdmjqwOMBAASDQNMVbnD3xzqGdxNYtxTg== 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB9bQdmM61yLxAAAeBjdPHisSW+6+y6yUMYfw== 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBwbQdmVEftMxAA1lDaA5SAWgLRPojMSwq70g== 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBxbQdmttL5ChAABt7Prk29JiHASSSErpnSgg== 2024-03-30T01:40:32.887 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2024-03-30T01:40:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBxbQdm5o/FGRAAuUz1WBZZuhrB8lrBXkgrzg== 2024-03-30T01:40:32.888 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2024-03-30T01:40:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCDbQdmAeb7NhAAo/xdDmOPniSMNAWGqXprzg== 2024-03-30T01:40:32.888 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2024-03-30T01:40:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCGbQdmsibcNhAAYJXjCRMNskc5suFV3ylOqw== 2024-03-30T01:40:32.888 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2024-03-30T01:40:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCJbQdmCi5wNhAAoFVc2HDSZQULOOhYt7lBDA== 2024-03-30T01:40:32.888 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2024-03-30T01:40:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCMbQdm2DocNxAAUPu6LwRv6gIMMr801Jsjhw== 2024-03-30T01:40:32.889 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-03-30T01:40:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBxbQdmickSHxAAowVfYqM7JYAqRJqYG0fQBw== 2024-03-30T01:40:32.889 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2024-03-30T01:40:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBxbQdmijhSIxAAbia5PslTYSh6u9Xvg/PF2g== 2024-03-30T01:40:32.915 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.y] 2024-03-30T01:40:32.915 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBwbQdmSyDMLhAAIk3a6lqgYJDDIrak1TE3Og== 2024-03-30T01:40:32.915 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2024-03-30T01:40:32.915 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBwbQdmKS4dOBAAoFJ9d9VJCzRRA6dvzqlvdA== 2024-03-30T01:40:32.915 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBxbQdmy2PsBBAAmQgxB6hsqmzkMOlsiyA2DQ== 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.0] 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQB0bQdmv2kRMBAA+Z/39yEKUrYr3hMcRw6UtQ== 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.1] 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQB3bQdmgRSYLxAA9DUas+VdQgEmd2nF77YNmw== 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.2] 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQB6bQdmjqwOMBAASDQNMVbnD3xzqGdxNYtxTg== 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.3] 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQB9bQdmM61yLxAAAeBjdPHisSW+6+y6yUMYfw== 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.x] 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBwbQdmVEftMxAA1lDaA5SAWgLRPojMSwq70g== 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBxbQdmttL5ChAABt7Prk29JiHASSSErpnSgg== 2024-03-30T01:40:32.916 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBxbQdm5o/FGRAAuUz1WBZZuhrB8lrBXkgrzg== 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.4] 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQCDbQdmAeb7NhAAo/xdDmOPniSMNAWGqXprzg== 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.5] 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQCGbQdmsibcNhAAYJXjCRMNskc5suFV3ylOqw== 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.6] 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQCJbQdmCi5wNhAAoFVc2HDSZQULOOhYt7lBDA== 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.7] 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQCMbQdm2DocNxAAUPu6LwRv6gIMMr801Jsjhw== 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBxbQdmickSHxAAowVfYqM7JYAqRJqYG0fQBw== 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout:[client.1] 2024-03-30T01:40:32.917 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQBxbQdmijhSIxAAbia5PslTYSh6u9Xvg/PF2g== 2024-03-30T01:40:32.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T01:40:32.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T01:40:33.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T01:40:33.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T01:40:33.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T01:40:33.094 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T01:40:33.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.223 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.285 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.348 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T01:40:33.409 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T01:40:33.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T01:40:33.471 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T01:40:33.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T01:40:33.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T01:40:33.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.726 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.787 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T01:40:33.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T01:40:33.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T01:40:33.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T01:40:33.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T01:40:33.978 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-30T01:40:33.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-30T01:40:34.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T01:40:34.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-30T01:40:34.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-30T01:40:34.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T01:40:34.295 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-30T01:40:34.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-30T01:40:34.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T01:40:34.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-30T01:40:34.547 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:34.549 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-30T01:40:34.608 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-30T01:40:34.609 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-30T01:40:34.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-30T01:40:34.611 INFO:tasks.ceph.mon.a:Started 2024-03-30T01:40:34.611 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-30T01:40:34.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-30T01:40:34.651 INFO:tasks.ceph.mon.b:Started 2024-03-30T01:40:34.651 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-30T01:40:34.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-30T01:40:34.653 INFO:tasks.ceph.mon.c:Started 2024-03-30T01:40:34.653 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-30T01:40:34.653 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-30T01:40:34.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-30T01:40:34.656 INFO:tasks.ceph.mgr.y:Started 2024-03-30T01:40:34.656 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-30T01:40:34.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-30T01:40:34.658 INFO:tasks.ceph.mgr.x:Started 2024-03-30T01:40:34.658 INFO:tasks.ceph:Setting crush tunables to default 2024-03-30T01:40:34.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2024-03-30T01:40:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.792+0000 7f827f394700 1 Processor -- start 2024-03-30T01:40:34.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.792+0000 7f827f394700 1 -- start start 2024-03-30T01:40:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827f394700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f827814fa20 0x7f827806f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827f394700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82780724e0 0x7f82781561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827f394700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8278153160 con 0x7f82780c1e00 2024-03-30T01:40:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827f394700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8278152830 con 0x7f82780724e0 2024-03-30T01:40:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827f394700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8278152b10 con 0x7f827814fa20 2024-03-30T01:40:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d931700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82780724e0 0x7f82781561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d931700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82780724e0 0x7f82781561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43532/0 (socket says 172.21.15.31:43532) 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d931700 1 -- 172.21.15.31:0/2294869143 learned_addr learned my addr 172.21.15.31:0/2294869143 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d931700 1 -- 172.21.15.31:0/2294869143 >> v2:172.21.15.31:3300/0 conn(0x7f82780724e0 msgr2=0x7f82781561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d931700 1 -- 172.21.15.31:0/2294869143 >> v2:172.21.15.31:3300/0 conn(0x7f82780724e0 msgr2=0x7f82781561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d931700 1 --2- 172.21.15.31:0/2294869143 >> v2:172.21.15.31:3300/0 conn(0x7f82780724e0 0x7f82781561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d931700 1 --2- 172.21.15.31:0/2294869143 >> v2:172.21.15.31:3300/0 conn(0x7f82780724e0 0x7f82781561e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d130700 1 --2- 172.21.15.31:0/2294869143 >> v2:172.21.15.59:3300/0 conn(0x7f827814fa20 0x7f827806f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d130700 1 -- 172.21.15.31:0/2294869143 >> v2:172.21.15.59:3300/0 conn(0x7f827814fa20 msgr2=0x7f827806f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d130700 1 -- 172.21.15.31:0/2294869143 >> v2:172.21.15.59:3300/0 conn(0x7f827814fa20 msgr2=0x7f827806f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d130700 1 --2- 172.21.15.31:0/2294869143 >> v2:172.21.15.59:3300/0 conn(0x7f827814fa20 0x7f827806f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T01:40:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.793+0000 7f827d130700 1 --2- 172.21.15.31:0/2294869143 >> v2:172.21.15.59:3300/0 conn(0x7f827814fa20 0x7f827806f310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T01:40:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.975+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879799654 0 0) 0x7f8278153160 con 0x7f82780c1e00 2024-03-30T01:40:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.975+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82640035b0 con 0x7f82780c1e00 2024-03-30T01:40:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.981+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 <== mon.1 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 518288284 0 0) 0x7f82640035b0 con 0x7f82780c1e00 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.981+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 >> v2:172.21.15.59:3300/0 conn(0x7f827814fa20 msgr2=0x7f827806f310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.981+0000 7f826ffff700 1 --2- 172.21.15.31:0/2294869143 >> v2:172.21.15.59:3300/0 conn(0x7f827814fa20 0x7f827806f310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 >> v2:172.21.15.31:3300/0 conn(0x7f82780724e0 msgr2=0x7f82781561e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 --2- 172.21.15.31:0/2294869143 >> v2:172.21.15.31:3300/0 conn(0x7f82780724e0 0x7f82781561e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8278152df0 con 0x7f82780c1e00 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 <== mon.1 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f8268001670 con 0x7f82780c1e00 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 >> v1:172.21.15.59:6789/0 conn(0x7f82780c1e00 legacy=0x7f8278155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f8264003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8264004850 0x7f8264004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82640053d0 0x7f826400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8278152830 con 0x7f8264003b30 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8278152b10 con 0x7f8264004850 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826400c370 con 0x7f82640053d0 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f827d130700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f8264003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f827c92f700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8264004850 0x7f8264004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f827d931700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82640053d0 0x7f826400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f827d130700 1 -- 172.21.15.31:0/2294869143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82640053d0 msgr2=0x7f826400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f827d130700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82640053d0 0x7f826400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f827d130700 1 -- 172.21.15.31:0/2294869143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8264004850 msgr2=0x7f8264004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f827d130700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8264004850 0x7f8264004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f827d130700 1 -- 172.21.15.31:0/2294869143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640035b0 con 0x7f8264003b30 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.982+0000 7f827d931700 1 -- 172.21.15.31:0/2294869143 reap_dead start 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.983+0000 7f827d130700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f8264003f50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8274009890 tx=0x7f8264005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf6ec39fa44c5c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.983+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400c070 con 0x7f8264003b30 2024-03-30T01:40:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.983+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8274004020 con 0x7f8264003b30 2024-03-30T01:40:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.983+0000 7f826ffff700 1 -- 172.21.15.31:0/2294869143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740043b0 con 0x7f8264003b30 2024-03-30T01:40:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.984+0000 7f827f394700 1 -- 172.21.15.31:0/2294869143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 msgr2=0x7f8264003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.984+0000 7f827f394700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f8264003f50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8274009890 tx=0x7f8264005880 comp rx=0 tx=0).stop 2024-03-30T01:40:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.984+0000 7f827f394700 1 -- 172.21.15.31:0/2294869143 shutdown_connections 2024-03-30T01:40:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.984+0000 7f827f394700 1 --2- 172.21.15.31:0/2294869143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f8264003f50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.984+0000 7f827f394700 1 -- 172.21.15.31:0/2294869143 >> 172.21.15.31:0/2294869143 conn(0x7f82780c3980 msgr2=0x7f82780c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.984+0000 7f827f394700 1 -- 172.21.15.31:0/2294869143 shutdown_connections 2024-03-30T01:40:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.984+0000 7f827f394700 1 -- 172.21.15.31:0/2294869143 wait complete. 2024-03-30T01:40:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.986+0000 7f827f394700 1 Processor -- start 2024-03-30T01:40:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.986+0000 7f827f394700 1 -- start start 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827f394700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f82781553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827f394700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82780c12b0 0x7f827814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827f394700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f827814e400 0x7f8278149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827f394700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82781e5f80 con 0x7f8264003b30 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827f394700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82781e64b0 con 0x7f827814e400 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827f394700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827814c0f0 con 0x7f82780c12b0 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827d130700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f82781553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827c92f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82780c12b0 0x7f827814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827d130700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f82781553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43586/0 (socket says 172.21.15.31:43586) 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827d130700 1 -- 172.21.15.31:0/1986720489 learned_addr learned my addr 172.21.15.31:0/1986720489 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.987+0000 7f827d931700 1 --2- 172.21.15.31:0/1986720489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f827814e400 0x7f8278149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:34.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.988+0000 7f827d130700 1 -- 172.21.15.31:0/1986720489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82780c12b0 msgr2=0x7f827814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:34.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.988+0000 7f827d130700 1 --2- 172.21.15.31:0/1986720489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82780c12b0 0x7f827814c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:34.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.988+0000 7f827d130700 1 -- 172.21.15.31:0/1986720489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f827814e400 msgr2=0x7f8278149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:34.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.988+0000 7f827d130700 1 --2- 172.21.15.31:0/1986720489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f827814e400 0x7f8278149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:34.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.988+0000 7f827d130700 1 -- 172.21.15.31:0/1986720489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82781e61d0 con 0x7f8264003b30 2024-03-30T01:40:34.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.988+0000 7f827c92f700 1 --2- 172.21.15.31:0/1986720489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82780c12b0 0x7f827814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:34.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.988+0000 7f827d130700 1 --2- 172.21.15.31:0/1986720489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f82781553d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8274000c00 tx=0x7f82740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:34.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.990+0000 7f826dffb700 1 -- 172.21.15.31:0/1986720489 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740045a0 con 0x7f8264003b30 2024-03-30T01:40:34.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.990+0000 7f826dffb700 1 -- 172.21.15.31:0/1986720489 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8274003d60 con 0x7f8264003b30 2024-03-30T01:40:34.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.990+0000 7f826dffb700 1 -- 172.21.15.31:0/1986720489 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274026680 con 0x7f8264003b30 2024-03-30T01:40:34.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.990+0000 7f827f394700 1 -- 172.21.15.31:0/1986720489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827814bd80 con 0x7f8264003b30 2024-03-30T01:40:34.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.990+0000 7f827f394700 1 -- 172.21.15.31:0/1986720489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8278155af0 con 0x7f8264003b30 2024-03-30T01:40:34.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.991+0000 7f826dffb700 1 -- 172.21.15.31:0/1986720489 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8274026ae0 con 0x7f8264003b30 2024-03-30T01:40:34.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.991+0000 7f826dffb700 1 -- 172.21.15.31:0/1986720489 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f827402fe10 con 0x7f8264003b30 2024-03-30T01:40:34.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.992+0000 7f827f394700 1 -- 172.21.15.31:0/1986720489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827814bbc0 con 0x7f8264003b30 2024-03-30T01:40:35.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:34.995+0000 7f826dffb700 1 -- 172.21.15.31:0/1986720489 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f8274017030 con 0x7f8264003b30 2024-03-30T01:40:35.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.077+0000 7f827f394700 1 -- 172.21.15.31:0/1986720489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f82780747d0 con 0x7f8264003b30 2024-03-30T01:40:35.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.079+0000 7f826dffb700 1 -- 172.21.15.31:0/1986720489 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f827400f040 con 0x7f8264003b30 2024-03-30T01:40:35.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.081+0000 7f827f394700 1 -- 172.21.15.31:0/1986720489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 msgr2=0x7f82781553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:35.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.081+0000 7f827f394700 1 --2- 172.21.15.31:0/1986720489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f82781553d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8274000c00 tx=0x7f82740047c0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.081+0000 7f827f394700 1 -- 172.21.15.31:0/1986720489 shutdown_connections 2024-03-30T01:40:35.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.081+0000 7f827f394700 1 --2- 172.21.15.31:0/1986720489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8264003b30 0x7f82781553d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.081+0000 7f827f394700 1 --2- 172.21.15.31:0/1986720489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f827814e400 0x7f8278149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.081+0000 7f827f394700 1 --2- 172.21.15.31:0/1986720489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f82780c12b0 0x7f827814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.081+0000 7f827f394700 1 -- 172.21.15.31:0/1986720489 >> 172.21.15.31:0/1986720489 conn(0x7f82780c3980 msgr2=0x7f82780a8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:35.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.081+0000 7f827f394700 1 -- 172.21.15.31:0/1986720489 shutdown_connections 2024-03-30T01:40:35.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:35.081+0000 7f827f394700 1 -- 172.21.15.31:0/1986720489 wait complete. 2024-03-30T01:40:35.087 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-03-30T01:40:35.100 INFO:tasks.ceph:check_enable_crimson: False 2024-03-30T01:40:35.100 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-30T01:40:35.101 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:35.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-30T01:40:35.130 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:35.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-30T01:40:35.202 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:35.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-30T01:40:35.272 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:40:35.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-30T01:40:35.343 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:35.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-30T01:40:35.373 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:35.373 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-30T01:40:35.443 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:35.444 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-30T01:40:35.462 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:35.457+0000 7f1fa53cc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T01:40:35.488 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:35.482+0000 7f1deb859200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T01:40:35.492 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:40:35.492 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-30T01:40:35.557 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:35.551+0000 7f1fa53cc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T01:40:35.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 4d49ae23-471f-4a34-9479-9676567f3867 0 2024-03-30T01:40:35.577 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:35.572+0000 7f1deb859200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T01:40:35.642 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:35.637+0000 7f1fa53cc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T01:40:35.660 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:35.655+0000 7f1deb859200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T01:40:35.724 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:35.718+0000 7f1fa53cc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T01:40:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.733+0000 7fd9769a0700 1 Processor -- start 2024-03-30T01:40:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.733+0000 7fd9769a0700 1 -- start start 2024-03-30T01:40:35.740 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:35.734+0000 7f1deb859200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T01:40:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.734+0000 7fd9769a0700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9700a7600 0x7fd970152f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.734+0000 7fd9769a0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd97014ff20 0x7fd9700c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd9769a0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9700c4620 con 0x7fd97014fb40 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd9769a0700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd970152570 con 0x7fd9700a7600 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd9769a0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd970152ad0 con 0x7fd97014ff20 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd974f3d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd97014ff20 0x7fd9700c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd974f3d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd97014ff20 0x7fd9700c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52266/0 (socket says 172.21.15.59:52266) 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd974f3d700 1 -- 172.21.15.59:0/3061959330 learned_addr learned my addr 172.21.15.59:0/3061959330 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd96f7fe700 1 --2- 172.21.15.59:0/3061959330 >> v2:172.21.15.31:3300/0 conn(0x7fd9700a7600 0x7fd970152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd974f3d700 1 -- 172.21.15.59:0/3061959330 >> v2:172.21.15.31:3300/0 conn(0x7fd9700a7600 msgr2=0x7fd970152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd974f3d700 1 --2- 172.21.15.59:0/3061959330 >> v2:172.21.15.31:3300/0 conn(0x7fd9700a7600 0x7fd970152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd974f3d700 1 -- 172.21.15.59:0/3061959330 >> v1:172.21.15.59:6789/0 conn(0x7fd97014fb40 legacy=0x7fd9700c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd974f3d700 1 -- 172.21.15.59:0/3061959330 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9701534a0 con 0x7fd97014ff20 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd974f3d700 1 --2- 172.21.15.59:0/3061959330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97014ff20 0x7fd9700c47f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9700c8e00 tx=0x7fd96800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42b1deba4c64dac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.735+0000 7fd96effd700 1 -- 172.21.15.59:0/3061959330 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96800e070 con 0x7fd97014ff20 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.736+0000 7fd96effd700 1 -- 172.21.15.59:0/3061959330 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96800b910 con 0x7fd97014ff20 2024-03-30T01:40:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.736+0000 7fd96effd700 1 -- 172.21.15.59:0/3061959330 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96800bcc0 con 0x7fd97014ff20 2024-03-30T01:40:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.737+0000 7fd9769a0700 1 -- 172.21.15.59:0/3061959330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97014ff20 msgr2=0x7fd9700c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.737+0000 7fd9769a0700 1 --2- 172.21.15.59:0/3061959330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97014ff20 0x7fd9700c47f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9700c8e00 tx=0x7fd96800b330 comp rx=0 tx=0).stop 2024-03-30T01:40:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.738+0000 7fd9769a0700 1 -- 172.21.15.59:0/3061959330 shutdown_connections 2024-03-30T01:40:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.738+0000 7fd9769a0700 1 --2- 172.21.15.59:0/3061959330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97014ff20 0x7fd9700c47f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.738+0000 7fd9769a0700 1 --2- 172.21.15.59:0/3061959330 >> v2:172.21.15.31:3300/0 conn(0x7fd9700a7600 0x7fd970152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.738+0000 7fd9769a0700 1 -- 172.21.15.59:0/3061959330 >> 172.21.15.59:0/3061959330 conn(0x7fd9700c7d60 msgr2=0x7fd9700c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.738+0000 7fd9769a0700 1 -- 172.21.15.59:0/3061959330 shutdown_connections 2024-03-30T01:40:35.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.738+0000 7fd9769a0700 1 -- 172.21.15.59:0/3061959330 wait complete. 2024-03-30T01:40:35.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.739+0000 7fd9769a0700 1 Processor -- start 2024-03-30T01:40:35.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.740+0000 7fd9769a0700 1 -- start start 2024-03-30T01:40:35.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd9769a0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9700a7600 0x7fd9701ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:35.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd9769a0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97014fb40 0x7fd9700aa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:35.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd9769a0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd97014ff20 0x7fd9701f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd9769a0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700a2210 con 0x7fd97014fb40 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd9769a0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700a2460 con 0x7fd9700a7600 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd9769a0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700a2740 con 0x7fd97014ff20 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd974f3d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd97014ff20 0x7fd9701f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd96ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9700a7600 0x7fd9701ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd974f3d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd97014ff20 0x7fd9701f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60326/0 (socket says 172.21.15.59:60326) 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd974f3d700 1 -- 172.21.15.59:0/970827944 learned_addr learned my addr 172.21.15.59:0/970827944 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd96ffff700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9700a7600 0x7fd9701ed9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52270/0 (socket says 172.21.15.59:52270) 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd96f7fe700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97014fb40 0x7fd9700aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd974f3d700 1 -- 172.21.15.59:0/970827944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9700a7600 msgr2=0x7fd9701ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd974f3d700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9700a7600 0x7fd9701ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd974f3d700 1 -- 172.21.15.59:0/970827944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97014fb40 msgr2=0x7fd9700aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd974f3d700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97014fb40 0x7fd9700aa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd974f3d700 1 -- 172.21.15.59:0/970827944 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd968009b30 con 0x7fd97014ff20 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd96ffff700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9700a7600 0x7fd9701ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.741+0000 7fd96f7fe700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97014fb40 0x7fd9700aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.742+0000 7fd974f3d700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd97014ff20 0x7fd9701f1650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd970152c10 tx=0x7fd968004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.742+0000 7fd96cff9700 1 -- 172.21.15.59:0/970827944 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968004880 con 0x7fd97014ff20 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.742+0000 7fd96cff9700 1 -- 172.21.15.59:0/970827944 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9680099b0 con 0x7fd97014ff20 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.742+0000 7fd9769a0700 1 -- 172.21.15.59:0/970827944 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9701f1130 con 0x7fd97014ff20 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.742+0000 7fd96cff9700 1 -- 172.21.15.59:0/970827944 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96801c680 con 0x7fd97014ff20 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.742+0000 7fd9769a0700 1 -- 172.21.15.59:0/970827944 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9700aa560 con 0x7fd97014ff20 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.742+0000 7fd96cff9700 1 -- 172.21.15.59:0/970827944 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd96801cae0 con 0x7fd97014ff20 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.742+0000 7fd96cff9700 1 -- 172.21.15.59:0/970827944 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd968025b90 con 0x7fd97014ff20 2024-03-30T01:40:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.743+0000 7fd9769a0700 1 -- 172.21.15.59:0/970827944 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9701f0f90 con 0x7fd97014ff20 2024-03-30T01:40:35.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.745+0000 7fd96cff9700 1 -- 172.21.15.59:0/970827944 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fd9701f0f90 con 0x7fd97014ff20 2024-03-30T01:40:35.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.836+0000 7fd9769a0700 1 -- 172.21.15.59:0/970827944 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4d49ae23-471f-4a34-9479-9676567f3867", "id": 0} v 0) v1 -- 0x7fd9700747d0 con 0x7fd97014ff20 2024-03-30T01:40:35.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.842+0000 7fd96cff9700 1 -- 172.21.15.59:0/970827944 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4d49ae23-471f-4a34-9479-9676567f3867", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd9680350b0 con 0x7fd97014ff20 2024-03-30T01:40:35.848 INFO:teuthology.orchestra.run.smithi059.stdout:0 2024-03-30T01:40:35.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.843+0000 7fd9769a0700 1 -- 172.21.15.59:0/970827944 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd97014ff20 msgr2=0x7fd9701f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:35.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.843+0000 7fd9769a0700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd97014ff20 0x7fd9701f1650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd970152c10 tx=0x7fd968004ad0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.843+0000 7fd9769a0700 1 -- 172.21.15.59:0/970827944 shutdown_connections 2024-03-30T01:40:35.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.843+0000 7fd9769a0700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd97014ff20 0x7fd9701f1650 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.843+0000 7fd9769a0700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9700a7600 0x7fd9701ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.843+0000 7fd9769a0700 1 --2- 172.21.15.59:0/970827944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd97014fb40 0x7fd9700aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:35.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.843+0000 7fd9769a0700 1 -- 172.21.15.59:0/970827944 >> 172.21.15.59:0/970827944 conn(0x7fd9700c7d60 msgr2=0x7fd970068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:35.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.843+0000 7fd9769a0700 1 -- 172.21.15.59:0/970827944 shutdown_connections 2024-03-30T01:40:35.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:35.843+0000 7fd9769a0700 1 -- 172.21.15.59:0/970827944 wait complete. 2024-03-30T01:40:35.862 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 19544e17-30cf-428e-9196-f982994de0b4 1 2024-03-30T01:40:35.925 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:35.919+0000 7f1fa53cc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T01:40:35.934 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:35.929+0000 7f1deb859200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T01:40:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.033+0000 7f72e180d700 1 Processor -- start 2024-03-30T01:40:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.033+0000 7f72e180d700 1 -- start start 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72e180d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f72dc155530 0x7f72dc0a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72e180d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c7810 0x7f72dc0c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72e180d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc0c31c0 con 0x7f72dc0c4a00 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72e180d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc0c41a0 con 0x7f72dc155530 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72e180d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0c34a0 con 0x7f72dc0c7810 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72da7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c7810 0x7f72dc0c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72daffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f72dc155530 0x7f72dc0a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72daffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f72dc155530 0x7f72dc0a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52274/0 (socket says 172.21.15.59:52274) 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72da7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c7810 0x7f72dc0c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60338/0 (socket says 172.21.15.59:60338) 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.034+0000 7f72daffd700 1 -- 172.21.15.59:0/1365215898 learned_addr learned my addr 172.21.15.59:0/1365215898 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72da7fc700 1 -- 172.21.15.59:0/1365215898 >> v2:172.21.15.59:3300/0 conn(0x7f72dc155530 msgr2=0x7f72dc0a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72da7fc700 1 --2- 172.21.15.59:0/1365215898 >> v2:172.21.15.59:3300/0 conn(0x7f72dc155530 0x7f72dc0a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72da7fc700 1 -- 172.21.15.59:0/1365215898 >> v1:172.21.15.59:6789/0 conn(0x7f72dc0c4a00 legacy=0x7f72dc157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72da7fc700 1 -- 172.21.15.59:0/1365215898 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc0c2890 con 0x7f72dc0c7810 2024-03-30T01:40:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72daffd700 1 --2- 172.21.15.59:0/1365215898 >> v2:172.21.15.59:3300/0 conn(0x7f72dc155530 0x7f72dc0a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72da7fc700 1 --2- 172.21.15.59:0/1365215898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c7810 0x7f72dc0c44c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009840 tx=0x7f72d0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea51b117895814c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72d9ffb700 1 -- 172.21.15.59:0/1365215898 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0011830 con 0x7f72dc0c7810 2024-03-30T01:40:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72d9ffb700 1 -- 172.21.15.59:0/1365215898 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0011990 con 0x7f72dc0c7810 2024-03-30T01:40:36.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72d9ffb700 1 -- 172.21.15.59:0/1365215898 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0011d20 con 0x7f72dc0c7810 2024-03-30T01:40:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72e180d700 1 -- 172.21.15.59:0/1365215898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c7810 msgr2=0x7f72dc0c44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.035+0000 7f72e180d700 1 --2- 172.21.15.59:0/1365215898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c7810 0x7f72dc0c44c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009840 tx=0x7f72d0009b50 comp rx=0 tx=0).stop 2024-03-30T01:40:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.036+0000 7f72e180d700 1 -- 172.21.15.59:0/1365215898 shutdown_connections 2024-03-30T01:40:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.036+0000 7f72e180d700 1 --2- 172.21.15.59:0/1365215898 >> v2:172.21.15.59:3300/0 conn(0x7f72dc155530 0x7f72dc0a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.036+0000 7f72e180d700 1 --2- 172.21.15.59:0/1365215898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c7810 0x7f72dc0c44c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.036+0000 7f72e180d700 1 -- 172.21.15.59:0/1365215898 >> 172.21.15.59:0/1365215898 conn(0x7f72dc0c7d60 msgr2=0x7f72dc0c8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.036+0000 7f72e180d700 1 -- 172.21.15.59:0/1365215898 shutdown_connections 2024-03-30T01:40:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.036+0000 7f72e180d700 1 -- 172.21.15.59:0/1365215898 wait complete. 2024-03-30T01:40:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.037+0000 7f72e180d700 1 Processor -- start 2024-03-30T01:40:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.037+0000 7f72e180d700 1 -- start start 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72e180d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c4a00 0x7f72dc1ed100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72e180d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0c7810 0x7f72dc150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72e180d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc155530 0x7f72dc1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72e180d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc09a6e0 con 0x7f72dc0c7810 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72e180d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc099a00 con 0x7f72dc155530 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72e180d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc1edb80 con 0x7f72dc0c4a00 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72db7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc155530 0x7f72dc1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72daffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c4a00 0x7f72dc1ed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72db7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc155530 0x7f72dc1512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52278/0 (socket says 172.21.15.59:52278) 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72db7fe700 1 -- 172.21.15.59:0/822699544 learned_addr learned my addr 172.21.15.59:0/822699544 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72da7fc700 1 --2- 172.21.15.59:0/822699544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0c7810 0x7f72dc150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72db7fe700 1 -- 172.21.15.59:0/822699544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c4a00 msgr2=0x7f72dc1ed100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72db7fe700 1 --2- 172.21.15.59:0/822699544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c4a00 0x7f72dc1ed100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72db7fe700 1 -- 172.21.15.59:0/822699544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0c7810 msgr2=0x7f72dc150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72db7fe700 1 --2- 172.21.15.59:0/822699544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0c7810 0x7f72dc150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72db7fe700 1 -- 172.21.15.59:0/822699544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc009b30 con 0x7f72dc155530 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.038+0000 7f72daffd700 1 --2- 172.21.15.59:0/822699544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c4a00 0x7f72dc1ed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.039+0000 7f72db7fe700 1 --2- 172.21.15.59:0/822699544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc155530 0x7f72dc1512e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f72d400c3c0 tx=0x7f72d400c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.039+0000 7f72c3fff700 1 -- 172.21.15.59:0/822699544 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d40052d0 con 0x7f72dc155530 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.039+0000 7f72c3fff700 1 -- 172.21.15.59:0/822699544 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d4005430 con 0x7f72dc155530 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.039+0000 7f72e180d700 1 -- 172.21.15.59:0/822699544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d00094a0 con 0x7f72dc155530 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.039+0000 7f72c3fff700 1 -- 172.21.15.59:0/822699544 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d40038a0 con 0x7f72dc155530 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.039+0000 7f72e180d700 1 -- 172.21.15.59:0/822699544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc0a6850 con 0x7f72dc155530 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.039+0000 7f72c3fff700 1 -- 172.21.15.59:0/822699544 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f72d4003d00 con 0x7f72dc155530 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.039+0000 7f72c3fff700 1 -- 172.21.15.59:0/822699544 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f72d401cd80 con 0x7f72dc155530 2024-03-30T01:40:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.040+0000 7f72e180d700 1 -- 172.21.15.59:0/822699544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc0747d0 con 0x7f72dc155530 2024-03-30T01:40:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.042+0000 7f72c3fff700 1 -- 172.21.15.59:0/822699544 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f72d4017070 con 0x7f72dc155530 2024-03-30T01:40:36.118 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:36.113+0000 7f1fa53cc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T01:40:36.120 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:36.115+0000 7f1deb859200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T01:40:36.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.133+0000 7f72e180d700 1 -- 172.21.15.59:0/822699544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd new", "uuid": "19544e17-30cf-428e-9196-f982994de0b4", "id": 1} v 0) v1 -- 0x7f72dc1530f0 con 0x7f72dc155530 2024-03-30T01:40:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.141+0000 7f72c3fff700 1 -- 172.21.15.59:0/822699544 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19544e17-30cf-428e-9196-f982994de0b4", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f72d401c440 con 0x7f72dc155530 2024-03-30T01:40:36.147 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-03-30T01:40:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.142+0000 7f72e180d700 1 -- 172.21.15.59:0/822699544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc155530 msgr2=0x7f72dc1512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.142+0000 7f72e180d700 1 --2- 172.21.15.59:0/822699544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc155530 0x7f72dc1512e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f72d400c3c0 tx=0x7f72d400c6d0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.142+0000 7f72e180d700 1 -- 172.21.15.59:0/822699544 shutdown_connections 2024-03-30T01:40:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.142+0000 7f72e180d700 1 --2- 172.21.15.59:0/822699544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc155530 0x7f72dc1512e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.142+0000 7f72e180d700 1 --2- 172.21.15.59:0/822699544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72dc0c4a00 0x7f72dc1ed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.142+0000 7f72e180d700 1 --2- 172.21.15.59:0/822699544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0c7810 0x7f72dc150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.142+0000 7f72e180d700 1 -- 172.21.15.59:0/822699544 >> 172.21.15.59:0/822699544 conn(0x7f72dc0c7d60 msgr2=0x7f72dc0c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:36.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.142+0000 7f72e180d700 1 -- 172.21.15.59:0/822699544 shutdown_connections 2024-03-30T01:40:36.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.142+0000 7f72e180d700 1 -- 172.21.15.59:0/822699544 wait complete. 2024-03-30T01:40:36.163 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 7412e4be-3d32-4222-9caf-e60156b87fd7 2 2024-03-30T01:40:36.245 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:36.240+0000 7f1deb859200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T01:40:36.246 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:36.241+0000 7f1fa53cc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T01:40:36.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.331+0000 7f5e3e165700 1 Processor -- start 2024-03-30T01:40:36.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.332+0000 7f5e3e165700 1 -- start start 2024-03-30T01:40:36.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.332+0000 7f5e3e165700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e380c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.332+0000 7f5e3e165700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e380c07f0 0x7f5e3806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.332+0000 7f5e3e165700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e38155000 con 0x7f5e380721f0 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.332+0000 7f5e3e165700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3806e930 con 0x7f5e380c07f0 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.332+0000 7f5e3e165700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3806df40 con 0x7f5e38151ea0 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e37fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e380c07f0 0x7f5e3806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e377fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e380c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e37fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e380c07f0 0x7f5e3806ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52294/0 (socket says 172.21.15.59:52294) 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e37fff700 1 -- 172.21.15.59:0/4212966262 learned_addr learned my addr 172.21.15.59:0/4212966262 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e377fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e380c02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60358/0 (socket says 172.21.15.59:60358) 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e377fe700 1 -- 172.21.15.59:0/4212966262 >> v2:172.21.15.59:3300/0 conn(0x7f5e380c07f0 msgr2=0x7f5e3806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e377fe700 1 --2- 172.21.15.59:0/4212966262 >> v2:172.21.15.59:3300/0 conn(0x7f5e380c07f0 0x7f5e3806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e377fe700 1 -- 172.21.15.59:0/4212966262 >> v1:172.21.15.31:6789/0 conn(0x7f5e380721f0 legacy=0x7f5e3806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e377fe700 1 -- 172.21.15.59:0/4212966262 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3806e650 con 0x7f5e38151ea0 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e36ffd700 1 -- 172.21.15.59:0/4212966262 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621270529 0 0) 0x7f5e38155000 con 0x7f5e380721f0 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e37fff700 1 --2- 172.21.15.59:0/4212966262 >> v2:172.21.15.59:3300/0 conn(0x7f5e380c07f0 0x7f5e3806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:36.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e377fe700 1 --2- 172.21.15.59:0/4212966262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e380c02b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009840 tx=0x7f5e28009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a447c52eb6a880a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e36ffd700 1 -- 172.21.15.59:0/4212966262 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28011830 con 0x7f5e38151ea0 2024-03-30T01:40:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e36ffd700 1 -- 172.21.15.59:0/4212966262 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e28011990 con 0x7f5e38151ea0 2024-03-30T01:40:36.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.333+0000 7f5e36ffd700 1 -- 172.21.15.59:0/4212966262 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28011d20 con 0x7f5e38151ea0 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.334+0000 7f5e3e165700 1 -- 172.21.15.59:0/4212966262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 msgr2=0x7f5e380c02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.334+0000 7f5e3e165700 1 --2- 172.21.15.59:0/4212966262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e380c02b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009840 tx=0x7f5e28009b50 comp rx=0 tx=0).stop 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.334+0000 7f5e3e165700 1 -- 172.21.15.59:0/4212966262 shutdown_connections 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.334+0000 7f5e3e165700 1 --2- 172.21.15.59:0/4212966262 >> v2:172.21.15.59:3300/0 conn(0x7f5e380c07f0 0x7f5e3806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.334+0000 7f5e3e165700 1 --2- 172.21.15.59:0/4212966262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e380c02b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.334+0000 7f5e3e165700 1 -- 172.21.15.59:0/4212966262 >> 172.21.15.59:0/4212966262 conn(0x7f5e380c3980 msgr2=0x7f5e380c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.334+0000 7f5e3e165700 1 -- 172.21.15.59:0/4212966262 shutdown_connections 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.334+0000 7f5e3e165700 1 -- 172.21.15.59:0/4212966262 wait complete. 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.335+0000 7f5e3e165700 1 Processor -- start 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.336+0000 7f5e3e165700 1 -- start start 2024-03-30T01:40:36.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.336+0000 7f5e3e165700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380721f0 0x7f5e381578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.336+0000 7f5e3e165700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e380c07f0 0x7f5e38157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.336+0000 7f5e3e165700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e381ecdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.336+0000 7f5e3e165700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38158a20 con 0x7f5e380721f0 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.336+0000 7f5e3e165700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38158cc0 con 0x7f5e380c07f0 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.336+0000 7f5e3e165700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3814c9b0 con 0x7f5e38151ea0 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e37fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e381ecdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e2effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e380c07f0 0x7f5e38157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e37fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e381ecdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60372/0 (socket says 172.21.15.59:60372) 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e37fff700 1 -- 172.21.15.59:0/3404500684 learned_addr learned my addr 172.21.15.59:0/3404500684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e377fe700 1 --2- 172.21.15.59:0/3404500684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380721f0 0x7f5e381578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e37fff700 1 -- 172.21.15.59:0/3404500684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e380c07f0 msgr2=0x7f5e38157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e37fff700 1 --2- 172.21.15.59:0/3404500684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e380c07f0 0x7f5e38157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e37fff700 1 -- 172.21.15.59:0/3404500684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380721f0 msgr2=0x7f5e381578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e37fff700 1 --2- 172.21.15.59:0/3404500684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380721f0 0x7f5e381578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e37fff700 1 -- 172.21.15.59:0/3404500684 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e30009b30 con 0x7f5e38151ea0 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e2effd700 1 --2- 172.21.15.59:0/3404500684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e380c07f0 0x7f5e38157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e37fff700 1 --2- 172.21.15.59:0/3404500684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e381ecdc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000efc0 tx=0x7f5e3000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e34ff9700 1 -- 172.21.15.59:0/3404500684 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30018070 con 0x7f5e38151ea0 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e34ff9700 1 -- 172.21.15.59:0/3404500684 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3000f040 con 0x7f5e38151ea0 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e34ff9700 1 -- 172.21.15.59:0/3404500684 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30003c20 con 0x7f5e38151ea0 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e3e165700 1 -- 172.21.15.59:0/3404500684 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e280094a0 con 0x7f5e38151ea0 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.337+0000 7f5e3e165700 1 -- 172.21.15.59:0/3404500684 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e381ea6b0 con 0x7f5e38151ea0 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.338+0000 7f5e34ff9700 1 -- 172.21.15.59:0/3404500684 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5e3000cb80 con 0x7f5e38151ea0 2024-03-30T01:40:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.338+0000 7f5e34ff9700 1 -- 172.21.15.59:0/3404500684 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f5e30014480 con 0x7f5e38151ea0 2024-03-30T01:40:36.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.338+0000 7f5e3e165700 1 -- 172.21.15.59:0/3404500684 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e380747d0 con 0x7f5e38151ea0 2024-03-30T01:40:36.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.341+0000 7f5e34ff9700 1 -- 172.21.15.59:0/3404500684 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f5e30003d80 con 0x7f5e38151ea0 2024-03-30T01:40:36.352 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:36.347+0000 7f1deb859200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T01:40:36.357 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:36.352+0000 7f1fa53cc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T01:40:36.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.431+0000 7f5e3e165700 1 -- 172.21.15.59:0/3404500684 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7412e4be-3d32-4222-9caf-e60156b87fd7", "id": 2} v 0) v1 -- 0x7f5e380c1520 con 0x7f5e38151ea0 2024-03-30T01:40:36.437 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:36.432+0000 7f1deb859200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T01:40:36.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.439+0000 7f5e34ff9700 1 -- 172.21.15.59:0/3404500684 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7412e4be-3d32-4222-9caf-e60156b87fd7", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5e3002c0b0 con 0x7f5e38151ea0 2024-03-30T01:40:36.445 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:36.439+0000 7f1fa53cc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T01:40:36.445 INFO:teuthology.orchestra.run.smithi059.stdout:2 2024-03-30T01:40:36.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.440+0000 7f5e3e165700 1 -- 172.21.15.59:0/3404500684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 msgr2=0x7f5e381ecdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.440+0000 7f5e3e165700 1 --2- 172.21.15.59:0/3404500684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e381ecdc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000efc0 tx=0x7f5e3000c3a0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.440+0000 7f5e3e165700 1 -- 172.21.15.59:0/3404500684 shutdown_connections 2024-03-30T01:40:36.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.440+0000 7f5e3e165700 1 --2- 172.21.15.59:0/3404500684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380721f0 0x7f5e381578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.440+0000 7f5e3e165700 1 --2- 172.21.15.59:0/3404500684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e38151ea0 0x7f5e381ecdc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.440+0000 7f5e3e165700 1 --2- 172.21.15.59:0/3404500684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e380c07f0 0x7f5e38157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.440+0000 7f5e3e165700 1 -- 172.21.15.59:0/3404500684 >> 172.21.15.59:0/3404500684 conn(0x7f5e380c3980 msgr2=0x7f5e380c5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:36.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.440+0000 7f5e3e165700 1 -- 172.21.15.59:0/3404500684 shutdown_connections 2024-03-30T01:40:36.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.440+0000 7f5e3e165700 1 -- 172.21.15.59:0/3404500684 wait complete. 2024-03-30T01:40:36.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 91097e90-f74b-4bd9-a1a7-0a2cce9c9df9 3 2024-03-30T01:40:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.591+0000 7f29c91fc700 1 Processor -- start 2024-03-30T01:40:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.591+0000 7f29c91fc700 1 -- start start 2024-03-30T01:40:36.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.592+0000 7f29c91fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29c4071df0 0x7f29c406a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.592+0000 7f29c91fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c40c2ad0 0x7f29c406ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.592+0000 7f29c91fc700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c4151e70 con 0x7f29c4071300 2024-03-30T01:40:36.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.592+0000 7f29c91fc700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c41523a0 con 0x7f29c4071df0 2024-03-30T01:40:36.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.592+0000 7f29c91fc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4070d90 con 0x7f29c40c2ad0 2024-03-30T01:40:36.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.592+0000 7f29c359e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c40c2ad0 0x7f29c406ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c359e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c40c2ad0 0x7f29c406ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60378/0 (socket says 172.21.15.59:60378) 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c359e700 1 -- 172.21.15.59:0/1880745289 learned_addr learned my addr 172.21.15.59:0/1880745289 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29bbfff700 1 --2- 172.21.15.59:0/1880745289 >> v2:172.21.15.31:3300/0 conn(0x7f29c4071df0 0x7f29c406a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c359e700 1 -- 172.21.15.59:0/1880745289 >> v2:172.21.15.31:3300/0 conn(0x7f29c4071df0 msgr2=0x7f29c406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c359e700 1 --2- 172.21.15.59:0/1880745289 >> v2:172.21.15.31:3300/0 conn(0x7f29c4071df0 0x7f29c406a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c359e700 1 -- 172.21.15.59:0/1880745289 >> v1:172.21.15.31:6789/0 conn(0x7f29c4071300 legacy=0x7f29c40716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c359e700 1 -- 172.21.15.59:0/1880745289 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c406c050 con 0x7f29c40c2ad0 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c359e700 1 --2- 172.21.15.59:0/1880745289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c40c2ad0 0x7f29c406ac40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009e50 tx=0x7f29bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a4cb3dd5a53ad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c259c700 1 -- 172.21.15.59:0/1880745289 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00e070 con 0x7f29c40c2ad0 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c259c700 1 -- 172.21.15.59:0/1880745289 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc00b910 con 0x7f29c40c2ad0 2024-03-30T01:40:36.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.593+0000 7f29c259c700 1 -- 172.21.15.59:0/1880745289 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00bcc0 con 0x7f29c40c2ad0 2024-03-30T01:40:36.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.594+0000 7f29c91fc700 1 -- 172.21.15.59:0/1880745289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c40c2ad0 msgr2=0x7f29c406ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.594+0000 7f29c91fc700 1 --2- 172.21.15.59:0/1880745289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c40c2ad0 0x7f29c406ac40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009e50 tx=0x7f29bc00b330 comp rx=0 tx=0).stop 2024-03-30T01:40:36.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.594+0000 7f29c91fc700 1 -- 172.21.15.59:0/1880745289 shutdown_connections 2024-03-30T01:40:36.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.594+0000 7f29c91fc700 1 --2- 172.21.15.59:0/1880745289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c40c2ad0 0x7f29c406ac40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.594+0000 7f29c91fc700 1 --2- 172.21.15.59:0/1880745289 >> v2:172.21.15.31:3300/0 conn(0x7f29c4071df0 0x7f29c406a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.594+0000 7f29c91fc700 1 -- 172.21.15.59:0/1880745289 >> 172.21.15.59:0/1880745289 conn(0x7f29c40c65c0 msgr2=0x7f29c40c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:36.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.594+0000 7f29c91fc700 1 -- 172.21.15.59:0/1880745289 shutdown_connections 2024-03-30T01:40:36.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.594+0000 7f29c91fc700 1 -- 172.21.15.59:0/1880745289 wait complete. 2024-03-30T01:40:36.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c91fc700 1 Processor -- start 2024-03-30T01:40:36.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c91fc700 1 -- start start 2024-03-30T01:40:36.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c91fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29c4071300 0x7f29c41dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c91fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c4071df0 0x7f29c41548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c91fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29c40c2ad0 0x7f29c41ded70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c91fc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41e7b60 con 0x7f29c4071300 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c91fc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41ddd80 con 0x7f29c40c2ad0 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c91fc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41ddfb0 con 0x7f29c4071df0 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29bbfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c4071df0 0x7f29c41548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29c40c2ad0 0x7f29c41ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29c359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29c40c2ad0 0x7f29c41ded70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52302/0 (socket says 172.21.15.59:52302) 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.596+0000 7f29bbfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c4071df0 0x7f29c41548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60384/0 (socket says 172.21.15.59:60384) 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c359e700 1 -- 172.21.15.59:0/3368992626 learned_addr learned my addr 172.21.15.59:0/3368992626 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c2d9d700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29c4071300 0x7f29c41dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c359e700 1 -- 172.21.15.59:0/3368992626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c4071df0 msgr2=0x7f29c41548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c359e700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c4071df0 0x7f29c41548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c359e700 1 -- 172.21.15.59:0/3368992626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29c4071300 msgr2=0x7f29c41dc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c359e700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29c4071300 0x7f29c41dc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c359e700 1 -- 172.21.15.59:0/3368992626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc009b30 con 0x7f29c40c2ad0 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29bbfff700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c4071df0 0x7f29c41548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c2d9d700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29c4071300 0x7f29c41dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c359e700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29c40c2ad0 0x7f29c41ded70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f29c40c2a80 tx=0x7f29bc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29bb7fe700 1 -- 172.21.15.59:0/3368992626 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc004800 con 0x7f29c40c2ad0 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29bb7fe700 1 -- 172.21.15.59:0/3368992626 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc004960 con 0x7f29c40c2ad0 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c91fc700 1 -- 172.21.15.59:0/3368992626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c41de290 con 0x7f29c40c2ad0 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29bb7fe700 1 -- 172.21.15.59:0/3368992626 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc01c680 con 0x7f29c40c2ad0 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.597+0000 7f29c91fc700 1 -- 172.21.15.59:0/3368992626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c41de850 con 0x7f29c40c2ad0 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.598+0000 7f29bb7fe700 1 -- 172.21.15.59:0/3368992626 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f29bc01cae0 con 0x7f29c40c2ad0 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.598+0000 7f29bb7fe700 1 -- 172.21.15.59:0/3368992626 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f29bc025ca0 con 0x7f29c40c2ad0 2024-03-30T01:40:36.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.598+0000 7f29c91fc700 1 -- 172.21.15.59:0/3368992626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c40c12d0 con 0x7f29c40c2ad0 2024-03-30T01:40:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.600+0000 7f29bb7fe700 1 -- 172.21.15.59:0/3368992626 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f29bc044b50 con 0x7f29c40c2ad0 2024-03-30T01:40:36.697 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:36.692+0000 7f1deb859200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T01:40:36.715 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:36.710+0000 7f1fa53cc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T01:40:36.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.734+0000 7f29c91fc700 1 -- 172.21.15.59:0/3368992626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd new", "uuid": "91097e90-f74b-4bd9-a1a7-0a2cce9c9df9", "id": 3} v 0) v1 -- 0x7f29c40747d0 con 0x7f29c40c2ad0 2024-03-30T01:40:36.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.742+0000 7f29bb7fe700 1 -- 172.21.15.59:0/3368992626 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91097e90-f74b-4bd9-a1a7-0a2cce9c9df9", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f29bc033070 con 0x7f29c40c2ad0 2024-03-30T01:40:36.748 INFO:teuthology.orchestra.run.smithi059.stdout:3 2024-03-30T01:40:36.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.743+0000 7f29c91fc700 1 -- 172.21.15.59:0/3368992626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29c40c2ad0 msgr2=0x7f29c41ded70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.743+0000 7f29c91fc700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29c40c2ad0 0x7f29c41ded70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f29c40c2a80 tx=0x7f29bc004ad0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.743+0000 7f29c91fc700 1 -- 172.21.15.59:0/3368992626 shutdown_connections 2024-03-30T01:40:36.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.743+0000 7f29c91fc700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29c4071300 0x7f29c41dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.743+0000 7f29c91fc700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29c40c2ad0 0x7f29c41ded70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.743+0000 7f29c91fc700 1 --2- 172.21.15.59:0/3368992626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29c4071df0 0x7f29c41548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.743+0000 7f29c91fc700 1 -- 172.21.15.59:0/3368992626 >> 172.21.15.59:0/3368992626 conn(0x7f29c40c65c0 msgr2=0x7f29c40c69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.743+0000 7f29c91fc700 1 -- 172.21.15.59:0/3368992626 shutdown_connections 2024-03-30T01:40:36.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.743+0000 7f29c91fc700 1 -- 172.21.15.59:0/3368992626 wait complete. 2024-03-30T01:40:36.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new ee161410-21c0-40e1-b7d2-54663eb413d7 4 2024-03-30T01:40:36.777 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:36.772+0000 7f1deb859200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T01:40:36.799 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:36.793+0000 7f1fa53cc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T01:40:36.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.891+0000 7f207278c700 1 Processor -- start 2024-03-30T01:40:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.892+0000 7f207278c700 1 -- start start 2024-03-30T01:40:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.892+0000 7f207278c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f206c0c5d40 0x7f206c0c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.892+0000 7f207278c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151f40 0x7f206c0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.892+0000 7f207278c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206c154990 con 0x7f206c151b60 2024-03-30T01:40:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.892+0000 7f207278c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206c155610 con 0x7f206c0c5d40 2024-03-30T01:40:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.892+0000 7f207278c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1547a0 con 0x7f206c151f40 2024-03-30T01:40:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.892+0000 7f2070d29700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151f40 0x7f206c0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f206c0c5d40 0x7f206c0c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f206c0c5d40 0x7f206c0c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52308/0 (socket says 172.21.15.59:52308) 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206b7fe700 1 -- 172.21.15.59:0/2292216582 learned_addr learned my addr 172.21.15.59:0/2292216582 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f2070d29700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151f40 0x7f206c0c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60398/0 (socket says 172.21.15.59:60398) 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206b7fe700 1 -- 172.21.15.59:0/2292216582 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151f40 msgr2=0x7f206c0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206b7fe700 1 --2- 172.21.15.59:0/2292216582 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151f40 0x7f206c0c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206b7fe700 1 -- 172.21.15.59:0/2292216582 >> v1:172.21.15.59:6789/0 conn(0x7f206c151b60 legacy=0x7f206c070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206b7fe700 1 -- 172.21.15.59:0/2292216582 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c154c70 con 0x7f206c0c5d40 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f2070d29700 1 --2- 172.21.15.59:0/2292216582 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151f40 0x7f206c0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206b7fe700 1 --2- 172.21.15.59:0/2292216582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 0x7f206c0c2760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f20600098f0 tx=0x7f2060009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b21bfa57cd46a01 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206affd700 1 -- 172.21.15.59:0/2292216582 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060011830 con 0x7f206c0c5d40 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206affd700 1 -- 172.21.15.59:0/2292216582 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2060011990 con 0x7f206c0c5d40 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.893+0000 7f206affd700 1 -- 172.21.15.59:0/2292216582 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060011d20 con 0x7f206c0c5d40 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.894+0000 7f207278c700 1 -- 172.21.15.59:0/2292216582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 msgr2=0x7f206c0c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.894+0000 7f207278c700 1 --2- 172.21.15.59:0/2292216582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 0x7f206c0c2760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f20600098f0 tx=0x7f2060009c00 comp rx=0 tx=0).stop 2024-03-30T01:40:36.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.894+0000 7f207278c700 1 -- 172.21.15.59:0/2292216582 shutdown_connections 2024-03-30T01:40:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.894+0000 7f207278c700 1 --2- 172.21.15.59:0/2292216582 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151f40 0x7f206c0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.894+0000 7f207278c700 1 --2- 172.21.15.59:0/2292216582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 0x7f206c0c2760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.894+0000 7f207278c700 1 -- 172.21.15.59:0/2292216582 >> 172.21.15.59:0/2292216582 conn(0x7f206c0c7d60 msgr2=0x7f206c0c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.894+0000 7f207278c700 1 -- 172.21.15.59:0/2292216582 shutdown_connections 2024-03-30T01:40:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.894+0000 7f207278c700 1 -- 172.21.15.59:0/2292216582 wait complete. 2024-03-30T01:40:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.895+0000 7f207278c700 1 Processor -- start 2024-03-30T01:40:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f207278c700 1 -- start start 2024-03-30T01:40:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f207278c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 0x7f206c158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f207278c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151b60 0x7f206c153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f207278c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c151f40 0x7f206c14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f207278c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1588e0 con 0x7f206c151f40 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f207278c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1ecd90 con 0x7f206c0c5d40 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f207278c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1ed070 con 0x7f206c151b60 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f206b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151b60 0x7f206c153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f206bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 0x7f206c158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f206b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151b60 0x7f206c153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60414/0 (socket says 172.21.15.59:60414) 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.896+0000 7f206b7fe700 1 -- 172.21.15.59:0/3588980454 learned_addr learned my addr 172.21.15.59:0/3588980454 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f2070d29700 1 --2- 172.21.15.59:0/3588980454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c151f40 0x7f206c14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f206b7fe700 1 -- 172.21.15.59:0/3588980454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 msgr2=0x7f206c158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f206b7fe700 1 --2- 172.21.15.59:0/3588980454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 0x7f206c158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f206b7fe700 1 -- 172.21.15.59:0/3588980454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c151f40 msgr2=0x7f206c14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f206b7fe700 1 --2- 172.21.15.59:0/3588980454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c151f40 0x7f206c14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f206b7fe700 1 -- 172.21.15.59:0/3588980454 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2064009a70 con 0x7f206c151b60 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f206bfff700 1 --2- 172.21.15.59:0/3588980454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 0x7f206c158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f206b7fe700 1 --2- 172.21.15.59:0/3588980454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151b60 0x7f206c153600 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2060009c40 tx=0x7f2060004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f2068ff9700 1 -- 172.21.15.59:0/3588980454 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20600044d0 con 0x7f206c151b60 2024-03-30T01:40:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f2068ff9700 1 -- 172.21.15.59:0/3588980454 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206001e070 con 0x7f206c151b60 2024-03-30T01:40:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f207278c700 1 -- 172.21.15.59:0/3588980454 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2060009550 con 0x7f206c151b60 2024-03-30T01:40:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f2068ff9700 1 -- 172.21.15.59:0/3588980454 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20600038e0 con 0x7f206c151b60 2024-03-30T01:40:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f207278c700 1 -- 172.21.15.59:0/3588980454 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c1ed630 con 0x7f206c151b60 2024-03-30T01:40:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f2068ff9700 1 -- 172.21.15.59:0/3588980454 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2060003d40 con 0x7f206c151b60 2024-03-30T01:40:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.897+0000 7f2068ff9700 1 -- 172.21.15.59:0/3588980454 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f206002dc10 con 0x7f206c151b60 2024-03-30T01:40:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.898+0000 7f207278c700 1 -- 172.21.15.59:0/3588980454 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206c0747d0 con 0x7f206c151b60 2024-03-30T01:40:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.901+0000 7f2068ff9700 1 -- 172.21.15.59:0/3588980454 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f2060020070 con 0x7f206c151b60 2024-03-30T01:40:36.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.990+0000 7f207278c700 1 -- 172.21.15.59:0/3588980454 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ee161410-21c0-40e1-b7d2-54663eb413d7", "id": 4} v 0) v1 -- 0x7f206c0c53b0 con 0x7f206c151b60 2024-03-30T01:40:37.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.998+0000 7f2068ff9700 1 -- 172.21.15.59:0/3588980454 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ee161410-21c0-40e1-b7d2-54663eb413d7", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2060015020 con 0x7f206c151b60 2024-03-30T01:40:37.004 INFO:teuthology.orchestra.run.smithi059.stdout:4 2024-03-30T01:40:37.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.999+0000 7f207278c700 1 -- 172.21.15.59:0/3588980454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151b60 msgr2=0x7f206c153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.999+0000 7f207278c700 1 --2- 172.21.15.59:0/3588980454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151b60 0x7f206c153600 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2060009c40 tx=0x7f2060004750 comp rx=0 tx=0).stop 2024-03-30T01:40:37.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.999+0000 7f207278c700 1 -- 172.21.15.59:0/3588980454 shutdown_connections 2024-03-30T01:40:37.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.999+0000 7f207278c700 1 --2- 172.21.15.59:0/3588980454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c151f40 0x7f206c14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.999+0000 7f207278c700 1 --2- 172.21.15.59:0/3588980454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f206c0c5d40 0x7f206c158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.999+0000 7f207278c700 1 --2- 172.21.15.59:0/3588980454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f206c151b60 0x7f206c153600 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.999+0000 7f207278c700 1 -- 172.21.15.59:0/3588980454 >> 172.21.15.59:0/3588980454 conn(0x7f206c0c7d60 msgr2=0x7f206c06c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:37.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.999+0000 7f207278c700 1 -- 172.21.15.59:0/3588980454 shutdown_connections 2024-03-30T01:40:37.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:36.999+0000 7f207278c700 1 -- 172.21.15.59:0/3588980454 wait complete. 2024-03-30T01:40:37.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 5e41e79a-4ea9-4753-9895-ffeaa0e14f7a 5 2024-03-30T01:40:37.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.191+0000 7f779fbbf700 1 Processor -- start 2024-03-30T01:40:37.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.191+0000 7f779fbbf700 1 -- start start 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779fbbf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f77980ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779fbbf700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77980c7f10 0x7f77980c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779fbbf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7798155240 con 0x7f77980ca890 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779fbbf700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779806cb20 con 0x7f77980c7f10 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779fbbf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77981556d0 con 0x7f7798151d80 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779d95b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f77980ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779e15c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f77980ca890 0x7f77980c6f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52150/0 (socket says 172.21.15.59:52150) 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779e15c700 1 -- 172.21.15.59:0/1808210402 learned_addr learned my addr 172.21.15.59:0/1808210402 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779d15a700 1 --2- 172.21.15.59:0/1808210402 >> v2:172.21.15.31:3300/0 conn(0x7f77980c7f10 0x7f77980c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779c959700 1 -- 172.21.15.59:0/1808210402 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227667984 0 0) 0x7f7798155240 con 0x7f77980ca890 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779c959700 1 -- 172.21.15.59:0/1808210402 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77840035b0 con 0x7f77980ca890 2024-03-30T01:40:37.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779d95b700 1 -- 172.21.15.59:0/1808210402 >> v2:172.21.15.31:3300/0 conn(0x7f77980c7f10 msgr2=0x7f77980c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:37.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779d95b700 1 --2- 172.21.15.59:0/1808210402 >> v2:172.21.15.31:3300/0 conn(0x7f77980c7f10 0x7f77980c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779d95b700 1 -- 172.21.15.59:0/1808210402 >> v1:172.21.15.59:6789/0 conn(0x7f77980ca890 legacy=0x7f77980c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.192+0000 7f779d95b700 1 -- 172.21.15.59:0/1808210402 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798154910 con 0x7f7798151d80 2024-03-30T01:40:37.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.193+0000 7f779d95b700 1 --2- 172.21.15.59:0/1808210402 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f77980ca350 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7794009840 tx=0x7f7794009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc8c8581092a60b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:37.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.193+0000 7f779c959700 1 -- 172.21.15.59:0/1808210402 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794011830 con 0x7f7798151d80 2024-03-30T01:40:37.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.193+0000 7f779c959700 1 -- 172.21.15.59:0/1808210402 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794011990 con 0x7f7798151d80 2024-03-30T01:40:37.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.193+0000 7f779c959700 1 -- 172.21.15.59:0/1808210402 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794011d20 con 0x7f7798151d80 2024-03-30T01:40:37.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.193+0000 7f779fbbf700 1 -- 172.21.15.59:0/1808210402 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 msgr2=0x7f77980ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.193+0000 7f779fbbf700 1 --2- 172.21.15.59:0/1808210402 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f77980ca350 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7794009840 tx=0x7f7794009b50 comp rx=0 tx=0).stop 2024-03-30T01:40:37.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.194+0000 7f779fbbf700 1 -- 172.21.15.59:0/1808210402 shutdown_connections 2024-03-30T01:40:37.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.194+0000 7f779fbbf700 1 --2- 172.21.15.59:0/1808210402 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f77980ca350 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.194+0000 7f779fbbf700 1 --2- 172.21.15.59:0/1808210402 >> v2:172.21.15.31:3300/0 conn(0x7f77980c7f10 0x7f77980c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.194+0000 7f779fbbf700 1 -- 172.21.15.59:0/1808210402 >> 172.21.15.59:0/1808210402 conn(0x7f7798072010 msgr2=0x7f7798072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.194+0000 7f779fbbf700 1 -- 172.21.15.59:0/1808210402 shutdown_connections 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.194+0000 7f779fbbf700 1 -- 172.21.15.59:0/1808210402 wait complete. 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.195+0000 7f779fbbf700 1 Processor -- start 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.195+0000 7f779fbbf700 1 -- start start 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779fbbf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77980c7f10 0x7f77980aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779fbbf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77980ca890 0x7f77980bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779fbbf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f7798158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779fbbf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77980be350 con 0x7f77980ca890 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779fbbf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779814af70 con 0x7f77980c7f10 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779fbbf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77981edea0 con 0x7f7798151d80 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d95b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77980c7f10 0x7f77980aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779e15c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f7798158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d95b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77980c7f10 0x7f77980aa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52326/0 (socket says 172.21.15.59:52326) 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779e15c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f7798158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60430/0 (socket says 172.21.15.59:60430) 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d95b700 1 -- 172.21.15.59:0/1811547054 learned_addr learned my addr 172.21.15.59:0/1811547054 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d15a700 1 --2- 172.21.15.59:0/1811547054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77980ca890 0x7f77980bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d95b700 1 -- 172.21.15.59:0/1811547054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 msgr2=0x7f7798158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d95b700 1 --2- 172.21.15.59:0/1811547054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f7798158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d95b700 1 -- 172.21.15.59:0/1811547054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77980ca890 msgr2=0x7f77980bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d95b700 1 --2- 172.21.15.59:0/1811547054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77980ca890 0x7f77980bdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d95b700 1 -- 172.21.15.59:0/1811547054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940094a0 con 0x7f77980c7f10 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d15a700 1 --2- 172.21.15.59:0/1811547054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77980ca890 0x7f77980bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.196+0000 7f779d95b700 1 --2- 172.21.15.59:0/1811547054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77980c7f10 0x7f77980aa5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7794009b90 tx=0x7f7794004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.197+0000 7f778e7fc700 1 -- 172.21.15.59:0/1811547054 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794004510 con 0x7f77980c7f10 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.197+0000 7f778e7fc700 1 -- 172.21.15.59:0/1811547054 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77940036a0 con 0x7f77980c7f10 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.197+0000 7f779fbbf700 1 -- 172.21.15.59:0/1811547054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77981ed0e0 con 0x7f77980c7f10 2024-03-30T01:40:37.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.197+0000 7f778e7fc700 1 -- 172.21.15.59:0/1811547054 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794003a10 con 0x7f77980c7f10 2024-03-30T01:40:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.197+0000 7f779fbbf700 1 -- 172.21.15.59:0/1811547054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77981ed6a0 con 0x7f77980c7f10 2024-03-30T01:40:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.197+0000 7f778e7fc700 1 -- 172.21.15.59:0/1811547054 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f779402e450 con 0x7f77980c7f10 2024-03-30T01:40:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.197+0000 7f778e7fc700 1 -- 172.21.15.59:0/1811547054 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f779402ec80 con 0x7f77980c7f10 2024-03-30T01:40:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.198+0000 7f779fbbf700 1 -- 172.21.15.59:0/1811547054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77980747d0 con 0x7f77980c7f10 2024-03-30T01:40:37.206 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:37.198+0000 7f1deb859200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T01:40:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.200+0000 7f778e7fc700 1 -- 172.21.15.59:0/1811547054 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f7794015020 con 0x7f77980c7f10 2024-03-30T01:40:37.255 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:37.249+0000 7f1fa53cc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T01:40:37.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.290+0000 7f779fbbf700 1 -- 172.21.15.59:0/1811547054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5e41e79a-4ea9-4753-9895-ffeaa0e14f7a", "id": 5} v 0) v1 -- 0x7f77980ca4d0 con 0x7f77980c7f10 2024-03-30T01:40:37.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.298+0000 7f778e7fc700 1 -- 172.21.15.59:0/1811547054 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5e41e79a-4ea9-4753-9895-ffeaa0e14f7a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f779402e700 con 0x7f77980c7f10 2024-03-30T01:40:37.304 INFO:teuthology.orchestra.run.smithi059.stdout:5 2024-03-30T01:40:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.299+0000 7f779fbbf700 1 -- 172.21.15.59:0/1811547054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77980c7f10 msgr2=0x7f77980aa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.299+0000 7f779fbbf700 1 --2- 172.21.15.59:0/1811547054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77980c7f10 0x7f77980aa5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7794009b90 tx=0x7f7794004750 comp rx=0 tx=0).stop 2024-03-30T01:40:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.299+0000 7f779fbbf700 1 -- 172.21.15.59:0/1811547054 shutdown_connections 2024-03-30T01:40:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.299+0000 7f779fbbf700 1 --2- 172.21.15.59:0/1811547054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7798151d80 0x7f7798158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.299+0000 7f779fbbf700 1 --2- 172.21.15.59:0/1811547054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77980c7f10 0x7f77980aa5d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.299+0000 7f779fbbf700 1 --2- 172.21.15.59:0/1811547054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77980ca890 0x7f77980bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.299+0000 7f779fbbf700 1 -- 172.21.15.59:0/1811547054 >> 172.21.15.59:0/1811547054 conn(0x7f7798072010 msgr2=0x7f7798150e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.299+0000 7f779fbbf700 1 -- 172.21.15.59:0/1811547054 shutdown_connections 2024-03-30T01:40:37.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.299+0000 7f779fbbf700 1 -- 172.21.15.59:0/1811547054 wait complete. 2024-03-30T01:40:37.318 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new d84d4e71-1b5e-4c4f-b085-a80bd441131b 6 2024-03-30T01:40:37.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.447+0000 7ff6063fd700 1 Processor -- start 2024-03-30T01:40:37.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.447+0000 7ff6063fd700 1 -- start start 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff6063fd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6000adcb0 0x7ff6000c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff6063fd700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6000a71c0 con 0x7ff60014fb30 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff6063fd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6000a74a0 con 0x7ff60014ff10 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff6063fd700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6000a7780 con 0x7ff6000adcb0 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff60499a700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff60014ff10 0x7ff6000c7c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52160/0 (socket says 172.21.15.59:52160) 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff60499a700 1 -- 172.21.15.59:0/2105061110 learned_addr learned my addr 172.21.15.59:0/2105061110 (peer_addr_for_me v1:172.21.15.59:0/0) 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff5ff7fe700 1 --2- 172.21.15.59:0/2105061110 >> v2:172.21.15.31:3300/0 conn(0x7ff6000adcb0 0x7ff6000c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff5feffd700 1 -- 172.21.15.59:0/2105061110 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060894965 0 0) 0x7ff6000a74a0 con 0x7ff60014ff10 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff5feffd700 1 -- 172.21.15.59:0/2105061110 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5ec0035b0 con 0x7ff60014ff10 2024-03-30T01:40:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff5feffd700 1 -- 172.21.15.59:0/2105061110 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008605923 0 0) 0x7ff6000a71c0 con 0x7ff60014fb30 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.448+0000 7ff5feffd700 1 -- 172.21.15.59:0/2105061110 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6000a74a0 con 0x7ff60014fb30 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.449+0000 7ff5ff7fe700 1 -- 172.21.15.59:0/2105061110 >> v1:172.21.15.59:6789/0 conn(0x7ff60014ff10 legacy=0x7ff6000c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.449+0000 7ff5ff7fe700 1 -- 172.21.15.59:0/2105061110 >> v1:172.21.15.31:6789/0 conn(0x7ff60014fb30 legacy=0x7ff600155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.449+0000 7ff5ff7fe700 1 -- 172.21.15.59:0/2105061110 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000c6d50 con 0x7ff6000adcb0 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.449+0000 7ff5ff7fe700 1 --2- 172.21.15.59:0/2105061110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000adcb0 0x7ff6000c7550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009840 tx=0x7ff5e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd56ed2d76824b2e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.449+0000 7ff5feffd700 1 -- 172.21.15.59:0/2105061110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8011830 con 0x7ff6000adcb0 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.449+0000 7ff5feffd700 1 -- 172.21.15.59:0/2105061110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e8011990 con 0x7ff6000adcb0 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.449+0000 7ff5feffd700 1 -- 172.21.15.59:0/2105061110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8011d20 con 0x7ff6000adcb0 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.450+0000 7ff6063fd700 1 -- 172.21.15.59:0/2105061110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000adcb0 msgr2=0x7ff6000c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.450+0000 7ff6063fd700 1 --2- 172.21.15.59:0/2105061110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000adcb0 0x7ff6000c7550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009840 tx=0x7ff5e8009b50 comp rx=0 tx=0).stop 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.450+0000 7ff6063fd700 1 -- 172.21.15.59:0/2105061110 shutdown_connections 2024-03-30T01:40:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.450+0000 7ff6063fd700 1 --2- 172.21.15.59:0/2105061110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6000adcb0 0x7ff6000c7550 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.450+0000 7ff6063fd700 1 -- 172.21.15.59:0/2105061110 >> 172.21.15.59:0/2105061110 conn(0x7ff6000a9aa0 msgr2=0x7ff6000a9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.450+0000 7ff6063fd700 1 -- 172.21.15.59:0/2105061110 shutdown_connections 2024-03-30T01:40:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.450+0000 7ff6063fd700 1 -- 172.21.15.59:0/2105061110 wait complete. 2024-03-30T01:40:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.451+0000 7ff6063fd700 1 Processor -- start 2024-03-30T01:40:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.451+0000 7ff6063fd700 1 -- start start 2024-03-30T01:40:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff6063fd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6000adcb0 0x7ff6000bf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff6063fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60014fb30 0x7ff6000c1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff6063fd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff60014ff10 0x7ff6000c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff6063fd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000bfcc0 con 0x7ff60014fb30 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff6063fd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001ea690 con 0x7ff6000adcb0 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff6063fd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001ea110 con 0x7ff60014ff10 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6000adcb0 0x7ff6000bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6000adcb0 0x7ff6000bf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52334/0 (socket says 172.21.15.59:52334) 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff60499a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff60014ff10 0x7ff6000c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff5fffff700 1 -- 172.21.15.59:0/3614880217 learned_addr learned my addr 172.21.15.59:0/3614880217 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff5ff7fe700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60014fb30 0x7ff6000c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff60499a700 1 -- 172.21.15.59:0/3614880217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6000adcb0 msgr2=0x7ff6000bf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff60499a700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6000adcb0 0x7ff6000bf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff60499a700 1 -- 172.21.15.59:0/3614880217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60014fb30 msgr2=0x7ff6000c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff60499a700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60014fb30 0x7ff6000c1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.452+0000 7ff60499a700 1 -- 172.21.15.59:0/3614880217 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80094a0 con 0x7ff60014ff10 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff5fffff700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6000adcb0 0x7ff6000bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff5ff7fe700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60014fb30 0x7ff6000c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff60499a700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff60014ff10 0x7ff6000c1b70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800caa0 tx=0x7ff5f800cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff5fcff9700 1 -- 172.21.15.59:0/3614880217 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800a9a0 con 0x7ff60014ff10 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff5fcff9700 1 -- 172.21.15.59:0/3614880217 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f800ab00 con 0x7ff60014ff10 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff6063fd700 1 -- 172.21.15.59:0/3614880217 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60006d810 con 0x7ff60014ff10 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff5fcff9700 1 -- 172.21.15.59:0/3614880217 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800ae70 con 0x7ff60014ff10 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff6063fd700 1 -- 172.21.15.59:0/3614880217 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6001ea3b0 con 0x7ff60014ff10 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff5fcff9700 1 -- 172.21.15.59:0/3614880217 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff5f801c440 con 0x7ff60014ff10 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.453+0000 7ff5fcff9700 1 -- 172.21.15.59:0/3614880217 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff5f8012a40 con 0x7ff60014ff10 2024-03-30T01:40:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.454+0000 7ff6063fd700 1 -- 172.21.15.59:0/3614880217 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6000747d0 con 0x7ff60014ff10 2024-03-30T01:40:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.456+0000 7ff5fcff9700 1 -- 172.21.15.59:0/3614880217 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7ff5f8021060 con 0x7ff60014ff10 2024-03-30T01:40:37.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.546+0000 7ff6063fd700 1 -- 172.21.15.59:0/3614880217 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d84d4e71-1b5e-4c4f-b085-a80bd441131b", "id": 6} v 0) v1 -- 0x7ff6000bf000 con 0x7ff60014ff10 2024-03-30T01:40:37.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.554+0000 7ff5fcff9700 1 -- 172.21.15.59:0/3614880217 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d84d4e71-1b5e-4c4f-b085-a80bd441131b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff5f8017070 con 0x7ff60014ff10 2024-03-30T01:40:37.560 INFO:teuthology.orchestra.run.smithi059.stdout:6 2024-03-30T01:40:37.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.555+0000 7ff6063fd700 1 -- 172.21.15.59:0/3614880217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff60014ff10 msgr2=0x7ff6000c1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.555+0000 7ff6063fd700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff60014ff10 0x7ff6000c1b70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800caa0 tx=0x7ff5f800cdb0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.555+0000 7ff6063fd700 1 -- 172.21.15.59:0/3614880217 shutdown_connections 2024-03-30T01:40:37.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.555+0000 7ff6063fd700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff60014ff10 0x7ff6000c1b70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.555+0000 7ff6063fd700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6000adcb0 0x7ff6000bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.555+0000 7ff6063fd700 1 --2- 172.21.15.59:0/3614880217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff60014fb30 0x7ff6000c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.555+0000 7ff6063fd700 1 -- 172.21.15.59:0/3614880217 >> 172.21.15.59:0/3614880217 conn(0x7ff6000a9aa0 msgr2=0x7ff6000c21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:37.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.555+0000 7ff6063fd700 1 -- 172.21.15.59:0/3614880217 shutdown_connections 2024-03-30T01:40:37.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.555+0000 7ff6063fd700 1 -- 172.21.15.59:0/3614880217 wait complete. 2024-03-30T01:40:37.566 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:37.561+0000 7f1deb859200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T01:40:37.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 9ad43344-0b2d-4a25-87df-15217ad5a295 7 2024-03-30T01:40:37.634 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:37.628+0000 7f1fa53cc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T01:40:37.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.703+0000 7f9e1ea25700 1 Processor -- start 2024-03-30T01:40:37.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.703+0000 7f9e1ea25700 1 -- start start 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1ea25700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9e18151d80 0x7f9e180ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1ea25700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e180c7f10 0x7f9e180c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1ea25700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1ea25700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1806cb20 con 0x7f9e18151d80 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1ea25700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e18155240 con 0x7f9e180c7f10 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1ea25700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e181556d0 con 0x7f9e180ca890 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1cfc2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e177fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e180c7f10 0x7f9e180c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e177fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e180c7f10 0x7f9e180c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52348/0 (socket says 172.21.15.59:52348) 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1cfc2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60454/0 (socket says 172.21.15.59:60454) 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e177fe700 1 -- 172.21.15.59:0/1135694757 learned_addr learned my addr 172.21.15.59:0/1135694757 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:37.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e17fff700 1 --2- 172.21.15.59:0/1135694757 >> v2:172.21.15.31:3300/0 conn(0x7f9e18151d80 0x7f9e180ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1cfc2700 1 -- 172.21.15.59:0/1135694757 >> v2:172.21.15.59:3300/0 conn(0x7f9e180c7f10 msgr2=0x7f9e180c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1cfc2700 1 --2- 172.21.15.59:0/1135694757 >> v2:172.21.15.59:3300/0 conn(0x7f9e180c7f10 0x7f9e180c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1cfc2700 1 -- 172.21.15.59:0/1135694757 >> v2:172.21.15.31:3300/0 conn(0x7f9e18151d80 msgr2=0x7f9e180ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1cfc2700 1 --2- 172.21.15.59:0/1135694757 >> v2:172.21.15.31:3300/0 conn(0x7f9e18151d80 0x7f9e180ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e1cfc2700 1 -- 172.21.15.59:0/1135694757 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180c74a0 con 0x7f9e180ca890 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.704+0000 7f9e177fe700 1 --2- 172.21.15.59:0/1135694757 >> v2:172.21.15.59:3300/0 conn(0x7f9e180c7f10 0x7f9e180c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e1cfc2700 1 --2- 172.21.15.59:0/1135694757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c6f60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e180c6950 tx=0x7f9e1000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f14c51b5cded074 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e16ffd700 1 -- 172.21.15.59:0/1135694757 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000e070 con 0x7f9e180ca890 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e16ffd700 1 -- 172.21.15.59:0/1135694757 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1000bb20 con 0x7f9e180ca890 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e16ffd700 1 -- 172.21.15.59:0/1135694757 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10012430 con 0x7f9e180ca890 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e1ea25700 1 -- 172.21.15.59:0/1135694757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 msgr2=0x7f9e180c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e1ea25700 1 --2- 172.21.15.59:0/1135694757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c6f60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e180c6950 tx=0x7f9e1000b330 comp rx=0 tx=0).stop 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e1ea25700 1 -- 172.21.15.59:0/1135694757 shutdown_connections 2024-03-30T01:40:37.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e1ea25700 1 --2- 172.21.15.59:0/1135694757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c6f60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e1ea25700 1 --2- 172.21.15.59:0/1135694757 >> v2:172.21.15.31:3300/0 conn(0x7f9e18151d80 0x7f9e180ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.705+0000 7f9e1ea25700 1 --2- 172.21.15.59:0/1135694757 >> v2:172.21.15.59:3300/0 conn(0x7f9e180c7f10 0x7f9e180c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.706+0000 7f9e1ea25700 1 -- 172.21.15.59:0/1135694757 >> 172.21.15.59:0/1135694757 conn(0x7f9e18072010 msgr2=0x7f9e18072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.706+0000 7f9e1ea25700 1 -- 172.21.15.59:0/1135694757 shutdown_connections 2024-03-30T01:40:37.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.706+0000 7f9e1ea25700 1 -- 172.21.15.59:0/1135694757 wait complete. 2024-03-30T01:40:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.707+0000 7f9e1ea25700 1 Processor -- start 2024-03-30T01:40:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.707+0000 7f9e1ea25700 1 -- start start 2024-03-30T01:40:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1ea25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e180c7f10 0x7f9e181ed170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1ea25700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1ea25700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e18151d80 0x7f9e180c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1ea25700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18148e50 con 0x7f9e180c7f10 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1ea25700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180ac8f0 con 0x7f9e18151d80 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1ea25700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180ac3c0 con 0x7f9e180ca890 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e177fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1cfc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e18151d80 0x7f9e180c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e177fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60470/0 (socket says 172.21.15.59:60470) 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e177fe700 1 -- 172.21.15.59:0/2576380068 learned_addr learned my addr 172.21.15.59:0/2576380068 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e17fff700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e180c7f10 0x7f9e181ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1cfc2700 1 -- 172.21.15.59:0/2576380068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 msgr2=0x7f9e180c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1cfc2700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1cfc2700 1 -- 172.21.15.59:0/2576380068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e180c7f10 msgr2=0x7f9e181ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1cfc2700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e180c7f10 0x7f9e181ed170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e1cfc2700 1 -- 172.21.15.59:0/2576380068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c009550 con 0x7f9e18151d80 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e177fe700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.708+0000 7f9e17fff700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e180c7f10 0x7f9e181ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.709+0000 7f9e1cfc2700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e18151d80 0x7f9e180c3970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9e1806ca00 tx=0x7f9e10004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.709+0000 7f9e14ff9700 1 -- 172.21.15.59:0/2576380068 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e100048b0 con 0x7f9e18151d80 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.709+0000 7f9e14ff9700 1 -- 172.21.15.59:0/2576380068 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e10020070 con 0x7f9e18151d80 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.709+0000 7f9e1ea25700 1 -- 172.21.15.59:0/2576380068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e10009a70 con 0x7f9e18151d80 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.709+0000 7f9e14ff9700 1 -- 172.21.15.59:0/2576380068 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e100276c0 con 0x7f9e18151d80 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.709+0000 7f9e1ea25700 1 -- 172.21.15.59:0/2576380068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e180aa510 con 0x7f9e18151d80 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.709+0000 7f9e14ff9700 1 -- 172.21.15.59:0/2576380068 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9e10027b20 con 0x7f9e18151d80 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.709+0000 7f9e14ff9700 1 -- 172.21.15.59:0/2576380068 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f9e10022070 con 0x7f9e18151d80 2024-03-30T01:40:37.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.710+0000 7f9e1ea25700 1 -- 172.21.15.59:0/2576380068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e180aa6a0 con 0x7f9e18151d80 2024-03-30T01:40:37.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.712+0000 7f9e14ff9700 1 -- 172.21.15.59:0/2576380068 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f9e10003e30 con 0x7f9e18151d80 2024-03-30T01:40:37.745 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:37.739+0000 7f1deb859200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T01:40:37.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.801+0000 7f9e1ea25700 1 -- 172.21.15.59:0/2576380068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ad43344-0b2d-4a25-87df-15217ad5a295", "id": 7} v 0) v1 -- 0x7f9e180747d0 con 0x7f9e18151d80 2024-03-30T01:40:37.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.810+0000 7f9e14ff9700 1 -- 172.21.15.59:0/2576380068 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ad43344-0b2d-4a25-87df-15217ad5a295", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9e100127a0 con 0x7f9e18151d80 2024-03-30T01:40:37.816 INFO:teuthology.orchestra.run.smithi059.stdout:7 2024-03-30T01:40:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.811+0000 7f9e1ea25700 1 -- 172.21.15.59:0/2576380068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e18151d80 msgr2=0x7f9e180c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.811+0000 7f9e1ea25700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e18151d80 0x7f9e180c3970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9e1806ca00 tx=0x7f9e10004ad0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.811+0000 7f9e1ea25700 1 -- 172.21.15.59:0/2576380068 shutdown_connections 2024-03-30T01:40:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.811+0000 7f9e1ea25700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e180c7f10 0x7f9e181ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.811+0000 7f9e1ea25700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e18151d80 0x7f9e180c3970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.811+0000 7f9e1ea25700 1 --2- 172.21.15.59:0/2576380068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e180ca890 0x7f9e180c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.811+0000 7f9e1ea25700 1 -- 172.21.15.59:0/2576380068 >> 172.21.15.59:0/2576380068 conn(0x7f9e18072010 msgr2=0x7f9e1814fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.811+0000 7f9e1ea25700 1 -- 172.21.15.59:0/2576380068 shutdown_connections 2024-03-30T01:40:37.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-03-30T01:40:37.811+0000 7f9e1ea25700 1 -- 172.21.15.59:0/2576380068 wait complete. 2024-03-30T01:40:37.818 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:37.813+0000 7f1fa53cc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T01:40:37.830 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-30T01:40:37.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-30T01:40:37.833 INFO:tasks.ceph.osd.0:Started 2024-03-30T01:40:37.833 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-30T01:40:37.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-30T01:40:37.834 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:37.828+0000 7f1deb859200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T01:40:37.836 INFO:tasks.ceph.osd.1:Started 2024-03-30T01:40:37.836 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-30T01:40:37.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-30T01:40:37.838 INFO:tasks.ceph.osd.2:Started 2024-03-30T01:40:37.838 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-30T01:40:37.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-30T01:40:37.840 INFO:tasks.ceph.osd.3:Started 2024-03-30T01:40:37.840 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-30T01:40:37.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-30T01:40:37.842 INFO:tasks.ceph.osd.4:Started 2024-03-30T01:40:37.843 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-30T01:40:37.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-30T01:40:37.845 INFO:tasks.ceph.osd.5:Started 2024-03-30T01:40:37.845 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-30T01:40:37.845 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-30T01:40:37.847 INFO:tasks.ceph.osd.6:Started 2024-03-30T01:40:37.847 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-30T01:40:37.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-30T01:40:37.849 INFO:tasks.ceph.osd.7:Started 2024-03-30T01:40:37.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:40:37.923 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:37.918+0000 7f1fa53cc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T01:40:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.032+0000 7fcca3a12700 1 Processor -- start 2024-03-30T01:40:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.033+0000 7fcca3a12700 1 -- start start 2024-03-30T01:40:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.033+0000 7fcca3a12700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 0x7fcc9c1601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.033+0000 7fcca3a12700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9c0c7fd0 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.033+0000 7fcca3a12700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9c154a20 con 0x7fcc9c06b400 2024-03-30T01:40:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.033+0000 7fcca3a12700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c154830 con 0x7fcc9c0c6480 2024-03-30T01:40:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcca0fad700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcc9c0c4f60 0x7fcc9c0c5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42876/0 (socket says 172.21.15.31:42876) 2024-03-30T01:40:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcca0fad700 1 -- 172.21.15.31:0/1343493497 learned_addr learned my addr 172.21.15.31:0/1343493497 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcca1faf700 1 --2- 172.21.15.31:0/1343493497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 0x7fcc9c1601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcc93fff700 1 -- 172.21.15.31:0/1343493497 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154572249 0 0) 0x7fcc9c0c7fd0 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcc93fff700 1 -- 172.21.15.31:0/1343493497 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc880035b0 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcc93fff700 1 -- 172.21.15.31:0/1343493497 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121215052 0 0) 0x7fcc9c154a20 con 0x7fcc9c06b400 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcc93fff700 1 -- 172.21.15.31:0/1343493497 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc9c0c7fd0 con 0x7fcc9c06b400 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcca1faf700 1 -- 172.21.15.31:0/1343493497 >> v1:172.21.15.59:6789/0 conn(0x7fcc9c06b400 legacy=0x7fcc9c151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcca1faf700 1 -- 172.21.15.31:0/1343493497 >> v1:172.21.15.31:6789/0 conn(0x7fcc9c0c4f60 legacy=0x7fcc9c0c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcca1faf700 1 -- 172.21.15.31:0/1343493497 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c0c5590 con 0x7fcc9c0c6480 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.034+0000 7fcca1faf700 1 --2- 172.21.15.31:0/1343493497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 0x7fcc9c1601c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009dc0 tx=0x7fcc9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e118f020d42b341 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcc93fff700 1 -- 172.21.15.31:0/1343493497 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400e070 con 0x7fcc9c0c6480 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcc93fff700 1 -- 172.21.15.31:0/1343493497 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9400bb20 con 0x7fcc9c0c6480 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcc93fff700 1 -- 172.21.15.31:0/1343493497 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94012430 con 0x7fcc9c0c6480 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcca3a12700 1 -- 172.21.15.31:0/1343493497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 msgr2=0x7fcc9c1601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcca3a12700 1 --2- 172.21.15.31:0/1343493497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 0x7fcc9c1601c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009dc0 tx=0x7fcc9400b330 comp rx=0 tx=0).stop 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcca3a12700 1 -- 172.21.15.31:0/1343493497 shutdown_connections 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcca3a12700 1 --2- 172.21.15.31:0/1343493497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 0x7fcc9c1601c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcca3a12700 1 -- 172.21.15.31:0/1343493497 >> 172.21.15.31:0/1343493497 conn(0x7fcc9c0ca0d0 msgr2=0x7fcc9c0ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:38.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcca3a12700 1 -- 172.21.15.31:0/1343493497 shutdown_connections 2024-03-30T01:40:38.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.035+0000 7fcca3a12700 1 -- 172.21.15.31:0/1343493497 wait complete. 2024-03-30T01:40:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca3a12700 1 Processor -- start 2024-03-30T01:40:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca3a12700 1 -- start start 2024-03-30T01:40:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca3a12700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc9c06b400 0x7fcc9c0c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca3a12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc9c0c4f60 0x7fcc9c1ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca3a12700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 0x7fcc9c1ed500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca3a12700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c1567b0 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca3a12700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c156fa0 con 0x7fcc9c06b400 2024-03-30T01:40:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca3a12700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c158da0 con 0x7fcc9c0c6480 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca0fad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc9c0c4f60 0x7fcc9c1ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca0fad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc9c0c4f60 0x7fcc9c1ecfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50570/0 (socket says 172.21.15.31:50570) 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca0fad700 1 -- 172.21.15.31:0/191431843 learned_addr learned my addr 172.21.15.31:0/191431843 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca1faf700 1 --2- 172.21.15.31:0/191431843 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 0x7fcc9c1ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.037+0000 7fcca17ae700 1 --2- 172.21.15.31:0/191431843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc9c06b400 0x7fcc9c0c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcca0fad700 1 -- 172.21.15.31:0/191431843 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 msgr2=0x7fcc9c1ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcca0fad700 1 --2- 172.21.15.31:0/191431843 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 0x7fcc9c1ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcca0fad700 1 -- 172.21.15.31:0/191431843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc9c06b400 msgr2=0x7fcc9c0c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcca0fad700 1 --2- 172.21.15.31:0/191431843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc9c06b400 0x7fcc9c0c3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcca0fad700 1 -- 172.21.15.31:0/191431843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc94009a70 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcca0fad700 1 --2- 172.21.15.31:0/191431843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc9c0c4f60 0x7fcc9c1ecfc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00c520 tx=0x7fcc8c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcc91ffb700 1 -- 172.21.15.31:0/191431843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c005d70 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcc91ffb700 1 -- 172.21.15.31:0/191431843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c0070d0 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcc91ffb700 1 -- 172.21.15.31:0/191431843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c007440 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcca3a12700 1 -- 172.21.15.31:0/191431843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9c156c70 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcc91ffb700 1 -- 172.21.15.31:0/191431843 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcc8c012400 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.038+0000 7fcca3a12700 1 -- 172.21.15.31:0/191431843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9c1edb70 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.039+0000 7fcc91ffb700 1 -- 172.21.15.31:0/191431843 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fcc8c01ba80 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.039+0000 7fcca3a12700 1 -- 172.21.15.31:0/191431843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9c157150 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.042+0000 7fcc91ffb700 1 -- 172.21.15.31:0/191431843 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fcc9c157150 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.117+0000 7fcca3a12700 1 -- 172.21.15.31:0/191431843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc9c151000 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.117+0000 7fcc91ffb700 1 -- 172.21.15.31:0/191431843 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fcc8c017070 con 0x7fcc9c0c4f60 2024-03-30T01:40:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:38.123 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:40:37.803254+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:40:38.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.118+0000 7fcca3a12700 1 -- 172.21.15.31:0/191431843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc9c0c4f60 msgr2=0x7fcc9c1ecfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:38.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.118+0000 7fcca3a12700 1 --2- 172.21.15.31:0/191431843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc9c0c4f60 0x7fcc9c1ecfc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00c520 tx=0x7fcc8c00c830 comp rx=0 tx=0).stop 2024-03-30T01:40:38.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.118+0000 7fcca3a12700 1 -- 172.21.15.31:0/191431843 shutdown_connections 2024-03-30T01:40:38.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.118+0000 7fcca3a12700 1 --2- 172.21.15.31:0/191431843 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc9c0c6480 0x7fcc9c1ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:38.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.118+0000 7fcca3a12700 1 --2- 172.21.15.31:0/191431843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc9c06b400 0x7fcc9c0c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:38.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.118+0000 7fcca3a12700 1 --2- 172.21.15.31:0/191431843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc9c0c4f60 0x7fcc9c1ecfc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:38.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.118+0000 7fcca3a12700 1 -- 172.21.15.31:0/191431843 >> 172.21.15.31:0/191431843 conn(0x7fcc9c0ca0d0 msgr2=0x7fcc9c154110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:38.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.119+0000 7fcca3a12700 1 -- 172.21.15.31:0/191431843 shutdown_connections 2024-03-30T01:40:38.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:38.119+0000 7fcca3a12700 1 -- 172.21.15.31:0/191431843 wait complete. 2024-03-30T01:40:38.137 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-30T01:40:38.138 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-30T01:40:38.462 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-03-30T01:40:38.457+0000 7fbadf351640 -1 Falling back to public interface 2024-03-30T01:40:38.463 INFO:tasks.ceph.osd.5.smithi059.stderr:2024-03-30T01:40:38.457+0000 7f811421d640 -1 Falling back to public interface 2024-03-30T01:40:38.464 INFO:tasks.ceph.osd.7.smithi059.stderr:2024-03-30T01:40:38.459+0000 7fe07148f640 -1 Falling back to public interface 2024-03-30T01:40:38.470 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-03-30T01:40:38.464+0000 7f6d8188b640 -1 Falling back to public interface 2024-03-30T01:40:38.475 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-03-30T01:40:38.470+0000 7fa417533640 -1 Falling back to public interface 2024-03-30T01:40:38.482 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-03-30T01:40:38.476+0000 7fbf895e3640 -1 Falling back to public interface 2024-03-30T01:40:38.485 INFO:tasks.ceph.osd.4.smithi059.stderr:2024-03-30T01:40:38.479+0000 7fe331acf640 -1 Falling back to public interface 2024-03-30T01:40:38.490 INFO:tasks.ceph.osd.6.smithi059.stderr:2024-03-30T01:40:38.484+0000 7f5620a6c640 -1 Falling back to public interface 2024-03-30T01:40:38.697 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:38.692+0000 7f1deb859200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T01:40:38.728 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:38.723+0000 7f1fa53cc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T01:40:38.791 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:38.786+0000 7f1deb859200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T01:40:38.822 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:38.816+0000 7f1fa53cc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T01:40:38.941 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:38.935+0000 7f1deb859200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T01:40:38.943 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T01:40:38.973 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:38.968+0000 7f1fa53cc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T01:40:39.072 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.066+0000 7f3e40198700 1 Processor -- start 2024-03-30T01:40:39.072 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.066+0000 7f3e40198700 1 -- start start 2024-03-30T01:40:39.074 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.067+0000 7f3e40198700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e380c5330 0x7f3e380c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.067+0000 7f3e40198700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e381525d0 0x7f3e38152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.067+0000 7f3e40198700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e3806bf80 con 0x7f3e380c1370 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.067+0000 7f3e40198700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3806e640 con 0x7f3e380c5330 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.067+0000 7f3e40198700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e380c2560 con 0x7f3e381525d0 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.067+0000 7f3e3df34700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e380c5330 0x7f3e380c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.067+0000 7f3e3d733700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3e380c1370 0x7f3e380c19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42878/0 (socket says 172.21.15.31:42878) 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.067+0000 7f3e3d733700 1 -- 172.21.15.31:0/3038466949 learned_addr learned my addr 172.21.15.31:0/3038466949 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3e735700 1 --2- 172.21.15.31:0/3038466949 >> v2:172.21.15.59:3300/0 conn(0x7f3e381525d0 0x7f3e38152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3df34700 1 -- 172.21.15.31:0/3038466949 >> v2:172.21.15.59:3300/0 conn(0x7f3e381525d0 msgr2=0x7f3e38152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3df34700 1 --2- 172.21.15.31:0/3038466949 >> v2:172.21.15.59:3300/0 conn(0x7f3e381525d0 0x7f3e38152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3df34700 1 -- 172.21.15.31:0/3038466949 >> v1:172.21.15.31:6789/0 conn(0x7f3e380c1370 legacy=0x7f3e380c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3df34700 1 -- 172.21.15.31:0/3038466949 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e38152f60 con 0x7f3e380c5330 2024-03-30T01:40:39.075 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3cf32700 1 -- 172.21.15.31:0/3038466949 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262601073 0 0) 0x7f3e28001110 con 0x7f3e380c1370 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3df34700 1 --2- 172.21.15.31:0/3038466949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 0x7f3e380c5710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3e38066c60 tx=0x7f3e3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd3bcae04c55855 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3cf32700 1 -- 172.21.15.31:0/3038466949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400e070 con 0x7f3e380c5330 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3cf32700 1 -- 172.21.15.31:0/3038466949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3400bb20 con 0x7f3e380c5330 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.068+0000 7f3e3cf32700 1 -- 172.21.15.31:0/3038466949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34012430 con 0x7f3e380c5330 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.069+0000 7f3e40198700 1 -- 172.21.15.31:0/3038466949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 msgr2=0x7f3e380c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.069+0000 7f3e40198700 1 --2- 172.21.15.31:0/3038466949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 0x7f3e380c5710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3e38066c60 tx=0x7f3e3400b330 comp rx=0 tx=0).stop 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.069+0000 7f3e40198700 1 -- 172.21.15.31:0/3038466949 shutdown_connections 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.069+0000 7f3e40198700 1 --2- 172.21.15.31:0/3038466949 >> v2:172.21.15.59:3300/0 conn(0x7f3e381525d0 0x7f3e38152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.069+0000 7f3e40198700 1 --2- 172.21.15.31:0/3038466949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 0x7f3e380c5710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.069+0000 7f3e40198700 1 -- 172.21.15.31:0/3038466949 >> 172.21.15.31:0/3038466949 conn(0x7f3e380c3020 msgr2=0x7f3e380c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.069+0000 7f3e40198700 1 -- 172.21.15.31:0/3038466949 shutdown_connections 2024-03-30T01:40:39.076 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.069+0000 7f3e40198700 1 -- 172.21.15.31:0/3038466949 wait complete. 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.070+0000 7f3e40198700 1 Processor -- start 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e40198700 1 -- start start 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e40198700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e380c1370 0x7f3e381dec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e40198700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 0x7f3e38151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e40198700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e381525d0 0x7f3e380722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e40198700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381df1d0 con 0x7f3e380c5330 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e40198700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381ddba0 con 0x7f3e381525d0 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e40198700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380c0420 con 0x7f3e380c1370 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e3d733700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 0x7f3e38151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e3d733700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 0x7f3e38151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50580/0 (socket says 172.21.15.31:50580) 2024-03-30T01:40:39.079 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.071+0000 7f3e3d733700 1 -- 172.21.15.31:0/1852498142 learned_addr learned my addr 172.21.15.31:0/1852498142 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e3e735700 1 --2- 172.21.15.31:0/1852498142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e381525d0 0x7f3e380722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e3df34700 1 --2- 172.21.15.31:0/1852498142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e380c1370 0x7f3e381dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e3d733700 1 -- 172.21.15.31:0/1852498142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e380c1370 msgr2=0x7f3e381dec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e3d733700 1 --2- 172.21.15.31:0/1852498142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e380c1370 0x7f3e381dec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e3d733700 1 -- 172.21.15.31:0/1852498142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e381525d0 msgr2=0x7f3e380722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e3d733700 1 --2- 172.21.15.31:0/1852498142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e381525d0 0x7f3e380722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e3d733700 1 -- 172.21.15.31:0/1852498142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e34009a70 con 0x7f3e380c5330 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e3d733700 1 --2- 172.21.15.31:0/1852498142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 0x7f3e38151400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c550 tx=0x7f3e2800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e2e7fc700 1 -- 172.21.15.31:0/1852498142 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28004c80 con 0x7f3e380c5330 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e2e7fc700 1 -- 172.21.15.31:0/1852498142 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e28004e00 con 0x7f3e380c5330 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e40198700 1 -- 172.21.15.31:0/1852498142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3814d850 con 0x7f3e380c5330 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e2e7fc700 1 -- 172.21.15.31:0/1852498142 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28007310 con 0x7f3e380c5330 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.072+0000 7f3e40198700 1 -- 172.21.15.31:0/1852498142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e38154e40 con 0x7f3e380c5330 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.073+0000 7f3e2e7fc700 1 -- 172.21.15.31:0/1852498142 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3e2801d3f0 con 0x7f3e380c5330 2024-03-30T01:40:39.080 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.073+0000 7f3e2e7fc700 1 -- 172.21.15.31:0/1852498142 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3e28013a90 con 0x7f3e380c5330 2024-03-30T01:40:39.081 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.073+0000 7f3e40198700 1 -- 172.21.15.31:0/1852498142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e38071e20 con 0x7f3e380c5330 2024-03-30T01:40:39.081 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.076+0000 7f3e2e7fc700 1 -- 172.21.15.31:0/1852498142 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f3e3806bf80 con 0x7f3e380c5330 2024-03-30T01:40:39.157 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.152+0000 7f3e40198700 1 -- 172.21.15.31:0/1852498142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e38067bb0 con 0x7f3e380c5330 2024-03-30T01:40:39.158 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.153+0000 7f3e2e7fc700 1 -- 172.21.15.31:0/1852498142 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f3e280133c0 con 0x7f3e380c5330 2024-03-30T01:40:39.158 INFO:teuthology.misc.health.smithi031.stdout: 2024-03-30T01:40:39.158 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:40:37.803254+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:40:39.159 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.154+0000 7f3e40198700 1 -- 172.21.15.31:0/1852498142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 msgr2=0x7f3e38151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:39.159 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.154+0000 7f3e40198700 1 --2- 172.21.15.31:0/1852498142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 0x7f3e38151400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c550 tx=0x7f3e2800c860 comp rx=0 tx=0).stop 2024-03-30T01:40:39.159 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.154+0000 7f3e40198700 1 -- 172.21.15.31:0/1852498142 shutdown_connections 2024-03-30T01:40:39.159 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.154+0000 7f3e40198700 1 --2- 172.21.15.31:0/1852498142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e381525d0 0x7f3e380722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:39.159 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.154+0000 7f3e40198700 1 --2- 172.21.15.31:0/1852498142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e380c1370 0x7f3e381dec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:39.160 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.154+0000 7f3e40198700 1 --2- 172.21.15.31:0/1852498142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e380c5330 0x7f3e38151400 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:39.160 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.154+0000 7f3e40198700 1 -- 172.21.15.31:0/1852498142 >> 172.21.15.31:0/1852498142 conn(0x7f3e380c3020 msgr2=0x7f3e380c3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:39.160 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.154+0000 7f3e40198700 1 -- 172.21.15.31:0/1852498142 shutdown_connections 2024-03-30T01:40:39.160 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:39.154+0000 7f3e40198700 1 -- 172.21.15.31:0/1852498142 wait complete. 2024-03-30T01:40:39.171 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-30T01:40:39.578 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:39.572+0000 7f1deb859200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T01:40:39.605 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:39.599+0000 7f1fa53cc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T01:40:39.768 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:39.763+0000 7f1deb859200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T01:40:39.795 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:39.789+0000 7f1fa53cc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T01:40:39.955 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:39.950+0000 7f1deb859200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T01:40:39.981 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:39.976+0000 7f1fa53cc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T01:40:40.881 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:40.876+0000 7f1deb859200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T01:40:40.911 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:40.906+0000 7f1fa53cc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T01:40:41.164 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:41.159+0000 7f1deb859200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T01:40:41.188 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:41.183+0000 7f1fa53cc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T01:40:41.329 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:41.324+0000 7f1deb859200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T01:40:41.355 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:41.349+0000 7f1fa53cc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T01:40:41.782 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:41.777+0000 7f1deb859200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T01:40:41.793 INFO:tasks.ceph.mgr.x.smithi059.stderr:2024-03-30T01:40:41.787+0000 7f1fa53cc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T01:40:41.960 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-03-30T01:40:41.954+0000 7f6d8188b640 -1 osd.2 0 log_to_monitors true 2024-03-30T01:40:41.960 INFO:tasks.ceph.osd.7.smithi059.stderr:2024-03-30T01:40:41.954+0000 7fe07148f640 -1 osd.7 0 log_to_monitors true 2024-03-30T01:40:41.965 INFO:tasks.ceph.osd.5.smithi059.stderr:2024-03-30T01:40:41.960+0000 7f811421d640 -1 osd.5 0 log_to_monitors true 2024-03-30T01:40:41.991 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-03-30T01:40:41.986+0000 7fbadf351640 -1 osd.0 0 log_to_monitors true 2024-03-30T01:40:41.995 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-03-30T01:40:41.990+0000 7fa417533640 -1 osd.1 0 log_to_monitors true 2024-03-30T01:40:42.000 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-03-30T01:40:41.995+0000 7fbf895e3640 -1 osd.3 0 log_to_monitors true 2024-03-30T01:40:42.002 INFO:tasks.ceph.osd.4.smithi059.stderr:2024-03-30T01:40:41.997+0000 7fe331acf640 -1 osd.4 0 log_to_monitors true 2024-03-30T01:40:42.003 INFO:tasks.ceph.osd.6.smithi059.stderr:2024-03-30T01:40:41.997+0000 7f5620a6c640 -1 osd.6 0 log_to_monitors true 2024-03-30T01:40:42.808 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:42.803+0000 7f1da6af3700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-30T01:40:42.809 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-03-30T01:40:42.804+0000 7f1da6af3700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-30T01:40:45.885 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-03-30T01:40:45.879+0000 7f6d75c40700 -1 osd.2 0 waiting for initial osdmap 2024-03-30T01:40:45.905 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-03-30T01:40:45.900+0000 7f6d6d81a700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T01:40:45.922 INFO:tasks.ceph.osd.7.smithi059.stderr:2024-03-30T01:40:45.913+0000 7fe065844700 -1 osd.7 0 waiting for initial osdmap 2024-03-30T01:40:45.924 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-03-30T01:40:45.917+0000 7fbad3706700 -1 osd.0 0 waiting for initial osdmap 2024-03-30T01:40:45.933 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-03-30T01:40:45.928+0000 7fbacb2e0700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T01:40:45.949 INFO:tasks.ceph.osd.7.smithi059.stderr:2024-03-30T01:40:45.943+0000 7fe05d41e700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T01:40:45.959 INFO:tasks.ceph.osd.6.smithi059.stderr:2024-03-30T01:40:45.952+0000 7f5614e21700 -1 osd.6 0 waiting for initial osdmap 2024-03-30T01:40:45.968 INFO:tasks.ceph.osd.4.smithi059.stderr:2024-03-30T01:40:45.961+0000 7fe325e84700 -1 osd.4 0 waiting for initial osdmap 2024-03-30T01:40:45.969 INFO:tasks.ceph.osd.6.smithi059.stderr:2024-03-30T01:40:45.964+0000 7f560c9fb700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T01:40:45.973 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-03-30T01:40:45.968+0000 7fa40b8e8700 -1 osd.1 0 waiting for initial osdmap 2024-03-30T01:40:45.977 INFO:tasks.ceph.osd.5.smithi059.stderr:2024-03-30T01:40:45.971+0000 7f81085d2700 -1 osd.5 0 waiting for initial osdmap 2024-03-30T01:40:45.977 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T01:40:45.981 INFO:tasks.ceph.osd.4.smithi059.stderr:2024-03-30T01:40:45.976+0000 7fe31da5e700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T01:40:45.987 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-03-30T01:40:45.982+0000 7fa4034c2700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T01:40:45.990 INFO:tasks.ceph.osd.5.smithi059.stderr:2024-03-30T01:40:45.985+0000 7f81001ac700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T01:40:46.115 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.109+0000 7f3e7a93e700 1 Processor -- start 2024-03-30T01:40:46.115 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.110+0000 7f3e7a93e700 1 -- start start 2024-03-30T01:40:46.116 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.110+0000 7f3e7a93e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e7406cc10 0x7f3e74152090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:46.116 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.110+0000 7f3e7a93e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e740ac570 con 0x7f3e740c82d0 2024-03-30T01:40:46.116 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.110+0000 7f3e7a93e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e740abcb0 con 0x7f3e740c7210 2024-03-30T01:40:46.116 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.110+0000 7f3e7a93e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e740abf20 con 0x7f3e7406cc10 2024-03-30T01:40:46.116 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e78edb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e7406cc10 0x7f3e74152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:46.116 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e78edb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e7406cc10 0x7f3e74152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53948/0 (socket says 172.21.15.31:53948) 2024-03-30T01:40:46.116 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e78edb700 1 -- 172.21.15.31:0/3211705620 learned_addr learned my addr 172.21.15.31:0/3211705620 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:46.116 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e737fe700 1 --1- 172.21.15.31:0/3211705620 >> v1:172.21.15.31:6789/0 conn(0x7f3e740c82d0 0x7f3e74151980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51658/0 (socket says 172.21.15.31:51658) 2024-03-30T01:40:46.117 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e78edb700 1 -- 172.21.15.31:0/3211705620 >> v1:172.21.15.59:6789/0 conn(0x7f3e740c7210 legacy=0x7f3e741549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:46.117 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e78edb700 1 -- 172.21.15.31:0/3211705620 >> v1:172.21.15.31:6789/0 conn(0x7f3e740c82d0 legacy=0x7f3e74151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:46.117 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e78edb700 1 -- 172.21.15.31:0/3211705620 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e740ac200 con 0x7f3e7406cc10 2024-03-30T01:40:46.118 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e78edb700 1 --2- 172.21.15.31:0/3211705620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406cc10 0x7f3e74152090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009e80 tx=0x7f3e6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6da2f870167a15e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:46.118 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e72ffd700 1 -- 172.21.15.31:0/3211705620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c00e070 con 0x7f3e7406cc10 2024-03-30T01:40:46.118 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e72ffd700 1 -- 172.21.15.31:0/3211705620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c00b980 con 0x7f3e7406cc10 2024-03-30T01:40:46.118 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.111+0000 7f3e72ffd700 1 -- 172.21.15.31:0/3211705620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c00bd30 con 0x7f3e7406cc10 2024-03-30T01:40:46.118 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.112+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3211705620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406cc10 msgr2=0x7f3e74152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:46.118 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.112+0000 7f3e7a93e700 1 --2- 172.21.15.31:0/3211705620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406cc10 0x7f3e74152090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009e80 tx=0x7f3e6c00b330 comp rx=0 tx=0).stop 2024-03-30T01:40:46.118 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.112+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3211705620 shutdown_connections 2024-03-30T01:40:46.119 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.112+0000 7f3e7a93e700 1 --2- 172.21.15.31:0/3211705620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7406cc10 0x7f3e74152090 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:46.119 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.112+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3211705620 >> 172.21.15.31:0/3211705620 conn(0x7f3e74072010 msgr2=0x7f3e74072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:46.119 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.113+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3211705620 shutdown_connections 2024-03-30T01:40:46.119 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.113+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3211705620 wait complete. 2024-03-30T01:40:46.119 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.114+0000 7f3e7a93e700 1 Processor -- start 2024-03-30T01:40:46.119 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.114+0000 7f3e7a93e700 1 -- start start 2024-03-30T01:40:46.121 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e7a93e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e7406cc10 0x7f3e740c04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e7a93e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e740c7210 0x7f3e741ea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e7a93e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e740c82d0 0x7f3e740aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e7a93e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e741ea160 con 0x7f3e740c82d0 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e7a93e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740bf6d0 con 0x7f3e740c7210 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e7a93e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740bf9b0 con 0x7f3e7406cc10 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e78edb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e740c82d0 0x7f3e740aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e78edb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e740c82d0 0x7f3e740aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53950/0 (socket says 172.21.15.31:53950) 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e78edb700 1 -- 172.21.15.31:0/3235101874 learned_addr learned my addr 172.21.15.31:0/3235101874 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e737fe700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e740c7210 0x7f3e741ea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e73fff700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e7406cc10 0x7f3e740c04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e78edb700 1 -- 172.21.15.31:0/3235101874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e7406cc10 msgr2=0x7f3e740c04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e78edb700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e7406cc10 0x7f3e740c04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e78edb700 1 -- 172.21.15.31:0/3235101874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e740c7210 msgr2=0x7f3e741ea8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:46.122 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e78edb700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e740c7210 0x7f3e741ea8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:46.123 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e78edb700 1 -- 172.21.15.31:0/3235101874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c009b30 con 0x7f3e740c82d0 2024-03-30T01:40:46.123 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e737fe700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e740c7210 0x7f3e741ea8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:40:46.123 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.115+0000 7f3e78edb700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e740c82d0 0x7f3e740aefa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3e740acda0 tx=0x7f3e6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:46.123 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.116+0000 7f3e70ff9700 1 -- 172.21.15.31:0/3235101874 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c004960 con 0x7f3e740c82d0 2024-03-30T01:40:46.123 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.116+0000 7f3e70ff9700 1 -- 172.21.15.31:0/3235101874 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c0092e0 con 0x7f3e740c82d0 2024-03-30T01:40:46.123 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.116+0000 7f3e70ff9700 1 -- 172.21.15.31:0/3235101874 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c01c640 con 0x7f3e740c82d0 2024-03-30T01:40:46.123 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.116+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e740bfc90 con 0x7f3e740c82d0 2024-03-30T01:40:46.123 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.116+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e740c3770 con 0x7f3e740c82d0 2024-03-30T01:40:46.124 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.117+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e740c75f0 con 0x7f3e740c82d0 2024-03-30T01:40:46.124 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.118+0000 7f3e70ff9700 1 -- 172.21.15.31:0/3235101874 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f3e6c01c7a0 con 0x7f3e740c82d0 2024-03-30T01:40:46.124 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.119+0000 7f3e70ff9700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e6006ac00 0x7f3e6006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:46.124 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.119+0000 7f3e70ff9700 1 -- 172.21.15.31:0/3235101874 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f3e6c08a1d0 con 0x7f3e740c82d0 2024-03-30T01:40:46.124 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.119+0000 7f3e73fff700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e6006ac00 0x7f3e6006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:46.125 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.120+0000 7f3e73fff700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e6006ac00 0x7f3e6006d0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000f60 tx=0x7f3e64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:46.133 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.128+0000 7f3e70ff9700 1 -- 172.21.15.31:0/3235101874 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3e6c056390 con 0x7f3e740c82d0 2024-03-30T01:40:46.269 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-03-30T01:40:46.264+0000 7fbf7d998700 -1 osd.3 0 waiting for initial osdmap 2024-03-30T01:40:46.288 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-03-30T01:40:46.283+0000 7fbf75572700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T01:40:46.369 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.364+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e740c6300 con 0x7f3e740c82d0 2024-03-30T01:40:46.369 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.364+0000 7f3e70ff9700 1 -- 172.21.15.31:0/3235101874 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f3e6c0599b0 con 0x7f3e740c82d0 2024-03-30T01:40:46.370 INFO:teuthology.misc.health.smithi031.stdout: 2024-03-30T01:40:46.370 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":12,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:40:43.850735+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e6006ac00 msgr2=0x7f3e6006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e6006ac00 0x7f3e6006d0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000f60 tx=0x7f3e64009040 comp rx=0 tx=0).stop 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e740c82d0 msgr2=0x7f3e740aefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e740c82d0 0x7f3e740aefa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3e740acda0 tx=0x7f3e6c004ad0 comp rx=0 tx=0).stop 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 shutdown_connections 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e740c82d0 0x7f3e740aefa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e6006ac00 0x7f3e6006d0c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e7406cc10 0x7f3e740c04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 --2- 172.21.15.31:0/3235101874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e740c7210 0x7f3e741ea8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 >> 172.21.15.31:0/3235101874 conn(0x7f3e74072010 msgr2=0x7f3e74150cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 shutdown_connections 2024-03-30T01:40:46.373 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:46.367+0000 7f3e7a93e700 1 -- 172.21.15.31:0/3235101874 wait complete. 2024-03-30T01:40:46.384 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-30T01:40:53.194 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T01:40:53.328 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.322+0000 7f56722a7700 1 Processor -- start 2024-03-30T01:40:53.328 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.322+0000 7f56722a7700 1 -- start start 2024-03-30T01:40:53.330 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.322+0000 7f56722a7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f566c0c2000 0x7f566c0c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.322+0000 7f56722a7700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566c0c1190 con 0x7f566c0c78c0 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.322+0000 7f56722a7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566c0c13e0 con 0x7f566c0706f0 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.322+0000 7f56722a7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566c0c32f0 con 0x7f566c0c2000 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f566c0c2000 0x7f566c0c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f5670844700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f566c0c78c0 0x7f566c15f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51666/0 (socket says 172.21.15.31:51666) 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f5670844700 1 -- 172.21.15.31:0/1528033773 learned_addr learned my addr 172.21.15.31:0/1528033773 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566affd700 1 -- 172.21.15.31:0/1528033773 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885098328 0 0) 0x7f566c0c1190 con 0x7f566c0c78c0 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566affd700 1 -- 172.21.15.31:0/1528033773 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56580035b0 con 0x7f566c0c78c0 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566b7fe700 1 -- 172.21.15.31:0/1528033773 >> v1:172.21.15.59:6789/0 conn(0x7f566c0706f0 legacy=0x7f566c1552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566b7fe700 1 -- 172.21.15.31:0/1528033773 >> v1:172.21.15.31:6789/0 conn(0x7f566c0c78c0 legacy=0x7f566c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566b7fe700 1 -- 172.21.15.31:0/1528033773 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c0c16c0 con 0x7f566c0c2000 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566b7fe700 1 --2- 172.21.15.31:0/1528033773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0c2000 0x7f566c0c3740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5654009840 tx=0x7f5654009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520633cd624e7ca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:53.331 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566affd700 1 -- 172.21.15.31:0/1528033773 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654011830 con 0x7f566c0c2000 2024-03-30T01:40:53.332 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566affd700 1 -- 172.21.15.31:0/1528033773 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654011990 con 0x7f566c0c2000 2024-03-30T01:40:53.332 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.323+0000 7f566affd700 1 -- 172.21.15.31:0/1528033773 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654011d20 con 0x7f566c0c2000 2024-03-30T01:40:53.332 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.324+0000 7f56722a7700 1 -- 172.21.15.31:0/1528033773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0c2000 msgr2=0x7f566c0c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.332 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.324+0000 7f56722a7700 1 --2- 172.21.15.31:0/1528033773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0c2000 0x7f566c0c3740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5654009840 tx=0x7f5654009b50 comp rx=0 tx=0).stop 2024-03-30T01:40:53.332 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.324+0000 7f56722a7700 1 -- 172.21.15.31:0/1528033773 shutdown_connections 2024-03-30T01:40:53.332 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.324+0000 7f56722a7700 1 --2- 172.21.15.31:0/1528033773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0c2000 0x7f566c0c3740 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.332 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.324+0000 7f56722a7700 1 -- 172.21.15.31:0/1528033773 >> 172.21.15.31:0/1528033773 conn(0x7f566c0c48c0 msgr2=0x7f566c0c4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:53.332 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.324+0000 7f56722a7700 1 -- 172.21.15.31:0/1528033773 shutdown_connections 2024-03-30T01:40:53.332 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.324+0000 7f56722a7700 1 -- 172.21.15.31:0/1528033773 wait complete. 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.326+0000 7f56722a7700 1 Processor -- start 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.326+0000 7f56722a7700 1 -- start start 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f56722a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0706f0 0x7f566c1ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f56722a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566c0c2000 0x7f566c0be020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f56722a7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f566c0c78c0 0x7f566c0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f56722a7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0acd30 con 0x7f566c0706f0 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f56722a7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0c0440 con 0x7f566c0c2000 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f56722a7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566c0abe60 con 0x7f566c0c78c0 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f566bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0706f0 0x7f566c1ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f566bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0706f0 0x7f566c1ee0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54000/0 (socket says 172.21.15.31:54000) 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f566bfff700 1 -- 172.21.15.31:0/1801578436 learned_addr learned my addr 172.21.15.31:0/1801578436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:53.335 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f566b7fe700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566c0c2000 0x7f566c0be020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f5670844700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f566c0c78c0 0x7f566c0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f566bfff700 1 -- 172.21.15.31:0/1801578436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f566c0c78c0 msgr2=0x7f566c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f566bfff700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f566c0c78c0 0x7f566c0ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f566bfff700 1 -- 172.21.15.31:0/1801578436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566c0c2000 msgr2=0x7f566c0be020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f566bfff700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566c0c2000 0x7f566c0be020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.327+0000 7f566bfff700 1 -- 172.21.15.31:0/1801578436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56540094a0 con 0x7f566c0706f0 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.328+0000 7f566bfff700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0706f0 0x7f566c1ee0b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f565c00c4e0 tx=0x7f565c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.328+0000 7f5668ff9700 1 -- 172.21.15.31:0/1801578436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c0052d0 con 0x7f566c0706f0 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.328+0000 7f5668ff9700 1 -- 172.21.15.31:0/1801578436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c005430 con 0x7f566c0706f0 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.328+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566c155b50 con 0x7f566c0706f0 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.328+0000 7f5668ff9700 1 -- 172.21.15.31:0/1801578436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c0038c0 con 0x7f566c0706f0 2024-03-30T01:40:53.336 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.328+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566c0ac670 con 0x7f566c0706f0 2024-03-30T01:40:53.337 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.329+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566c04ea80 con 0x7f566c0706f0 2024-03-30T01:40:53.337 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.329+0000 7f5668ff9700 1 -- 172.21.15.31:0/1801578436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f565c003a20 con 0x7f566c0706f0 2024-03-30T01:40:53.337 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.330+0000 7f5668ff9700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f564c06ac90 0x7f564c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.337 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.330+0000 7f5668ff9700 1 -- 172.21.15.31:0/1801578436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f565c0882e0 con 0x7f566c0706f0 2024-03-30T01:40:53.337 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.330+0000 7f566b7fe700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f564c06ac90 0x7f564c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.338 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.330+0000 7f566b7fe700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f564c06ac90 0x7f564c06d150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5654009840 tx=0x7f5654008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:53.341 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.335+0000 7f5668ff9700 1 -- 172.21.15.31:0/1801578436 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f565c0538a0 con 0x7f566c0706f0 2024-03-30T01:40:53.552 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.546+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f566c0c1d50 con 0x7f566c0706f0 2024-03-30T01:40:53.553 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.547+0000 7f5668ff9700 1 -- 172.21.15.31:0/1801578436 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12334 (secure 0 0 0) 0x7f565c056ec0 con 0x7f566c0706f0 2024-03-30T01:40:53.553 INFO:teuthology.misc.health.smithi031.stdout: 2024-03-30T01:40:53.553 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":16,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:40:49.909825+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:40:53.556 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f564c06ac90 msgr2=0x7f564c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.556 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f564c06ac90 0x7f564c06d150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5654009840 tx=0x7f5654008030 comp rx=0 tx=0).stop 2024-03-30T01:40:53.556 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0706f0 msgr2=0x7f566c1ee0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.556 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0706f0 0x7f566c1ee0b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f565c00c4e0 tx=0x7f565c00c7f0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.556 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 shutdown_connections 2024-03-30T01:40:53.556 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f566c0706f0 0x7f566c1ee0b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.557 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f564c06ac90 0x7f564c06d150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.557 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f566c0c78c0 0x7f566c0ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.557 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 --2- 172.21.15.31:0/1801578436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566c0c2000 0x7f566c0be020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.557 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 >> 172.21.15.31:0/1801578436 conn(0x7f566c0c48c0 msgr2=0x7f566c0c3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:53.557 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 shutdown_connections 2024-03-30T01:40:53.557 INFO:teuthology.misc.health.smithi031.stderr:2024-03-30T01:40:53.549+0000 7f56722a7700 1 -- 172.21.15.31:0/1801578436 wait complete. 2024-03-30T01:40:53.568 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-30T01:40:53.568 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-30T01:40:53.569 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T01:40:53.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T01:40:53.571 INFO:tasks.ceph.mds.a:Started 2024-03-30T01:40:53.571 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T01:40:53.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T01:40:53.573 INFO:tasks.ceph.mds.c:Started 2024-03-30T01:40:53.573 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T01:40:53.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T01:40:53.576 INFO:tasks.ceph.mds.b:Started 2024-03-30T01:40:53.576 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T01:40:53.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T01:40:53.578 INFO:tasks.ceph.mds.d:Started 2024-03-30T01:40:53.579 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-30T01:40:53.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:40:53.661 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-03-30T01:40:53.662 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2024-03-30T01:40:53.672 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2024-03-30T01:40:53.672 INFO:tasks.ceph.mds.b.smithi059.stdout:starting mds.b at 2024-03-30T01:40:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.716+0000 7f891b631700 1 Processor -- start 2024-03-30T01:40:53.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.716+0000 7f891b631700 1 -- start start 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.717+0000 7f891b631700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89140c5f50 0x7f89140705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.717+0000 7f891b631700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8914157d50 con 0x7f89141547e0 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.717+0000 7f891b631700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8914070c30 con 0x7f8914151f00 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.717+0000 7f891b631700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89140c4010 con 0x7f89140c5f50 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.717+0000 7f8919bce700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f89141547e0 0x7f891415f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51684/0 (socket says 172.21.15.31:51684) 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.717+0000 7f8919bce700 1 -- 172.21.15.31:0/1713355291 learned_addr learned my addr 172.21.15.31:0/1713355291 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f8918bcc700 1 --2- 172.21.15.31:0/1713355291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89140c5f50 0x7f89140705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f890bfff700 1 -- 172.21.15.31:0/1713355291 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993013172 0 0) 0x7f8914157d50 con 0x7f89141547e0 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f890bfff700 1 -- 172.21.15.31:0/1713355291 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89000035b0 con 0x7f89141547e0 2024-03-30T01:40:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f890bfff700 1 -- 172.21.15.31:0/1713355291 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931093690 0 0) 0x7f8914070c30 con 0x7f8914151f00 2024-03-30T01:40:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f890bfff700 1 -- 172.21.15.31:0/1713355291 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8914157d50 con 0x7f8914151f00 2024-03-30T01:40:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f8918bcc700 1 -- 172.21.15.31:0/1713355291 >> v1:172.21.15.59:6789/0 conn(0x7f8914151f00 legacy=0x7f89140c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f890bfff700 1 -- 172.21.15.31:0/1713355291 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1257663982 0 0) 0x7f89000035b0 con 0x7f89141547e0 2024-03-30T01:40:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f8918bcc700 1 -- 172.21.15.31:0/1713355291 >> v1:172.21.15.31:6789/0 conn(0x7f89141547e0 legacy=0x7f891415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f8918bcc700 1 -- 172.21.15.31:0/1713355291 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8914157fa0 con 0x7f89140c5f50 2024-03-30T01:40:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f8918bcc700 1 --2- 172.21.15.31:0/1713355291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89140c5f50 0x7f89140705a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8904009870 tx=0x7f8904009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a765badcf209d3c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f890bfff700 1 -- 172.21.15.31:0/1713355291 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c0019f0 con 0x7f89140c5f50 2024-03-30T01:40:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f890bfff700 1 -- 172.21.15.31:0/1713355291 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904004030 con 0x7f89140c5f50 2024-03-30T01:40:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.718+0000 7f890bfff700 1 -- 172.21.15.31:0/1713355291 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89040043c0 con 0x7f89140c5f50 2024-03-30T01:40:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.719+0000 7f891b631700 1 -- 172.21.15.31:0/1713355291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89140c5f50 msgr2=0x7f89140705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.719+0000 7f891b631700 1 --2- 172.21.15.31:0/1713355291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89140c5f50 0x7f89140705a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8904009870 tx=0x7f8904009b80 comp rx=0 tx=0).stop 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.719+0000 7f891b631700 1 -- 172.21.15.31:0/1713355291 shutdown_connections 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.719+0000 7f891b631700 1 --2- 172.21.15.31:0/1713355291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89140c5f50 0x7f89140705a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.719+0000 7f891b631700 1 -- 172.21.15.31:0/1713355291 >> 172.21.15.31:0/1713355291 conn(0x7f891406ad30 msgr2=0x7f89140a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.719+0000 7f891b631700 1 -- 172.21.15.31:0/1713355291 shutdown_connections 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.719+0000 7f891b631700 1 -- 172.21.15.31:0/1713355291 wait complete. 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.720+0000 7f891b631700 1 Processor -- start 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.720+0000 7f891b631700 1 -- start start 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f891b631700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89140c5f50 0x7f89140acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f891b631700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8914151f00 0x7f89141e9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f891b631700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89141547e0 0x7f891406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f891b631700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89140a99b0 con 0x7f8914151f00 2024-03-30T01:40:53.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f891b631700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89141e9c80 con 0x7f89140c5f50 2024-03-30T01:40:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f891b631700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89140700c0 con 0x7f89141547e0 2024-03-30T01:40:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f8918bcc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8914151f00 0x7f89141e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f8918bcc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8914151f00 0x7f89141e9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54024/0 (socket says 172.21.15.31:54024) 2024-03-30T01:40:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f8918bcc700 1 -- 172.21.15.31:0/2532762121 learned_addr learned my addr 172.21.15.31:0/2532762121 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f8919bce700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89141547e0 0x7f891406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f89193cd700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89140c5f50 0x7f89140acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f8918bcc700 1 -- 172.21.15.31:0/2532762121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89141547e0 msgr2=0x7f891406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f8918bcc700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89141547e0 0x7f891406f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f8918bcc700 1 -- 172.21.15.31:0/2532762121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89140c5f50 msgr2=0x7f89140acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f8918bcc700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89140c5f50 0x7f89140acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.721+0000 7f8918bcc700 1 -- 172.21.15.31:0/2532762121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89040094a0 con 0x7f8914151f00 2024-03-30T01:40:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.722+0000 7f8918bcc700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8914151f00 0x7f89141e9740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8904000c00 tx=0x7f8904004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.722+0000 7f8909ffb700 1 -- 172.21.15.31:0/2532762121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904011630 con 0x7f8914151f00 2024-03-30T01:40:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.722+0000 7f8909ffb700 1 -- 172.21.15.31:0/2532762121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890401f070 con 0x7f8914151f00 2024-03-30T01:40:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.722+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89141e9e90 con 0x7f8914151f00 2024-03-30T01:40:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.722+0000 7f8909ffb700 1 -- 172.21.15.31:0/2532762121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89040119f0 con 0x7f8914151f00 2024-03-30T01:40:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.722+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891406b9b0 con 0x7f8914151f00 2024-03-30T01:40:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.723+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891406b810 con 0x7f8914151f00 2024-03-30T01:40:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.724+0000 7f8909ffb700 1 -- 172.21.15.31:0/2532762121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8904011b50 con 0x7f8914151f00 2024-03-30T01:40:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.724+0000 7f8909ffb700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88f806ac90 0x7f88f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.724+0000 7f8909ffb700 1 -- 172.21.15.31:0/2532762121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8904021070 con 0x7f8914151f00 2024-03-30T01:40:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.724+0000 7f89193cd700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88f806ac90 0x7f88f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.725+0000 7f89193cd700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88f806ac90 0x7f88f806d150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89100011e0 tx=0x7f8910009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:53.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.729+0000 7f8909ffb700 1 -- 172.21.15.31:0/2532762121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f891406b810 con 0x7f8914151f00 2024-03-30T01:40:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.947+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89140ac8e0 con 0x7f8914151f00 2024-03-30T01:40:53.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.948+0000 7f8909ffb700 1 -- 172.21.15.31:0/2532762121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12334 (secure 0 0 0) 0x7f89140ac8e0 con 0x7f8914151f00 2024-03-30T01:40:53.955 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:53.955 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:40:49.909825+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:40:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88f806ac90 msgr2=0x7f88f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88f806ac90 0x7f88f806d150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89100011e0 tx=0x7f8910009040 comp rx=0 tx=0).stop 2024-03-30T01:40:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8914151f00 msgr2=0x7f89141e9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8914151f00 0x7f89141e9740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8904000c00 tx=0x7f8904004750 comp rx=0 tx=0).stop 2024-03-30T01:40:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 shutdown_connections 2024-03-30T01:40:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88f806ac90 0x7f88f806d150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89141547e0 0x7f891406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89140c5f50 0x7f89140acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 --2- 172.21.15.31:0/2532762121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8914151f00 0x7f89141e9740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 >> 172.21.15.31:0/2532762121 conn(0x7f891406ad30 msgr2=0x7f8914151570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 shutdown_connections 2024-03-30T01:40:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:53.951+0000 7f891b631700 1 -- 172.21.15.31:0/2532762121 wait complete. 2024-03-30T01:40:53.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:40:53.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:40:54.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.104+0000 7f52fadd2700 1 Processor -- start 2024-03-30T01:40:54.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.104+0000 7f52fadd2700 1 -- start start 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.104+0000 7f52fadd2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52f40721f0 0x7f52f406a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.104+0000 7f52fadd2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f4155000 con 0x7f52f4155490 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.104+0000 7f52fadd2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f406e650 con 0x7f52f4151ea0 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.104+0000 7f52fadd2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f406e930 con 0x7f52f40721f0 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f936f700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52f4155490 0x7f52f406ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51690/0 (socket says 172.21.15.31:51690) 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f936f700 1 -- 172.21.15.31:0/1010833007 learned_addr learned my addr 172.21.15.31:0/1010833007 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f3fff700 1 --2- 172.21.15.31:0/1010833007 >> v2:172.21.15.59:3300/0 conn(0x7f52f40721f0 0x7f52f406a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026917238 0 0) 0x7f52f4155000 con 0x7f52f4155490 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e40035b0 con 0x7f52f4155490 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3728077577 0 0) 0x7f52e40035b0 con 0x7f52f4155490 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 >> v2:172.21.15.59:3300/0 conn(0x7f52f40721f0 msgr2=0x7f52f406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 --2- 172.21.15.31:0/1010833007 >> v2:172.21.15.59:3300/0 conn(0x7f52f40721f0 0x7f52f406a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 >> v1:172.21.15.59:6789/0 conn(0x7f52f4151ea0 legacy=0x7f52f40c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f41549b0 con 0x7f52f4155490 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f52ec0019f0 con 0x7f52f4155490 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f3fff700 1 --2- 172.21.15.31:0/1010833007 >> v2:172.21.15.59:3300/0 conn(0x7f52f40721f0 0x7f52f406a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 >> v1:172.21.15.31:6789/0 conn(0x7f52f4155490 legacy=0x7f52f406ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003ae0 0x7f52e4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52e4004850 0x7f52e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f406e930 con 0x7f52e4005260 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e400c1b0 con 0x7f52e4003ae0 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e400c370 con 0x7f52e4004850 2024-03-30T01:40:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f936f700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.105+0000 7f52f8b6e700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003ae0 0x7f52e4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f3fff700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52e4004850 0x7f52e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f936f700 1 -- 172.21.15.31:0/1010833007 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52e4004850 msgr2=0x7f52e4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f936f700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52e4004850 0x7f52e4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f936f700 1 -- 172.21.15.31:0/1010833007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003ae0 msgr2=0x7f52e4003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f936f700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003ae0 0x7f52e4003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f936f700 1 -- 172.21.15.31:0/1010833007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e40035b0 con 0x7f52e4005260 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f936f700 1 -- 172.21.15.31:0/1010833007 reap_dead start 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f936f700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52e400ba10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52ec002230 tx=0x7f52e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba3237af8529dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec012600 con 0x7f52e4005260 2024-03-30T01:40:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec012780 con 0x7f52e4005260 2024-03-30T01:40:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.106+0000 7f52f37fe700 1 -- 172.21.15.31:0/1010833007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec012b30 con 0x7f52e4005260 2024-03-30T01:40:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.107+0000 7f52fadd2700 1 -- 172.21.15.31:0/1010833007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 msgr2=0x7f52e400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.107+0000 7f52fadd2700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52e400ba10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52ec002230 tx=0x7f52e4005880 comp rx=0 tx=0).stop 2024-03-30T01:40:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.107+0000 7f52fadd2700 1 -- 172.21.15.31:0/1010833007 shutdown_connections 2024-03-30T01:40:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.107+0000 7f52fadd2700 1 --2- 172.21.15.31:0/1010833007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52e400ba10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.107+0000 7f52fadd2700 1 -- 172.21.15.31:0/1010833007 >> 172.21.15.31:0/1010833007 conn(0x7f52f40c3980 msgr2=0x7f52f40c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.107+0000 7f52fadd2700 1 -- 172.21.15.31:0/1010833007 shutdown_connections 2024-03-30T01:40:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.107+0000 7f52fadd2700 1 -- 172.21.15.31:0/1010833007 wait complete. 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.108+0000 7f52fadd2700 1 Processor -- start 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.109+0000 7f52fadd2700 1 -- start start 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52fadd2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52f41ea080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52fadd2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52f406ac10 0x7f52f4157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52fadd2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52f41581b0 0x7f52f41ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52fadd2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4154820 con 0x7f52e4005260 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52fadd2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f414bd20 con 0x7f52f406ac10 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52fadd2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f41ea5c0 con 0x7f52f41581b0 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f8b6e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52f41ea080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f8b6e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52f41ea080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54038/0 (socket says 172.21.15.31:54038) 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f8b6e700 1 -- 172.21.15.31:0/395145498 learned_addr learned my addr 172.21.15.31:0/395145498 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f936f700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52f41581b0 0x7f52f41ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f3fff700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52f406ac10 0x7f52f4157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f8b6e700 1 -- 172.21.15.31:0/395145498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52f41581b0 msgr2=0x7f52f41ecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f8b6e700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52f41581b0 0x7f52f41ecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f8b6e700 1 -- 172.21.15.31:0/395145498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52f406ac10 msgr2=0x7f52f4157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f8b6e700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52f406ac10 0x7f52f4157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f8b6e700 1 -- 172.21.15.31:0/395145498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4158630 con 0x7f52e4005260 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f8b6e700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52f41ea080 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52e800f7b0 tx=0x7f52e800fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f17fa700 1 -- 172.21.15.31:0/395145498 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e80052e0 con 0x7f52e4005260 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.110+0000 7f52f17fa700 1 -- 172.21.15.31:0/395145498 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e8005460 con 0x7f52e4005260 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.111+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f4154010 con 0x7f52e4005260 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.111+0000 7f52f17fa700 1 -- 172.21.15.31:0/395145498 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e80038c0 con 0x7f52e4005260 2024-03-30T01:40:54.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.111+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f4153940 con 0x7f52e4005260 2024-03-30T01:40:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.111+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f40747d0 con 0x7f52e4005260 2024-03-30T01:40:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.112+0000 7f52f17fa700 1 -- 172.21.15.31:0/395145498 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f52e8003a20 con 0x7f52e4005260 2024-03-30T01:40:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.112+0000 7f52f17fa700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f52d806ac90 0x7f52d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.113+0000 7f52f17fa700 1 -- 172.21.15.31:0/395145498 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52e8088f40 con 0x7f52e4005260 2024-03-30T01:40:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.113+0000 7f52f3fff700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f52d806ac90 0x7f52d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.113+0000 7f52f3fff700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f52d806ac90 0x7f52d806d150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52f4149a90 tx=0x7f52e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.117+0000 7f52f17fa700 1 -- 172.21.15.31:0/395145498 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f52e8054480 con 0x7f52e4005260 2024-03-30T01:40:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.331+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52f40c1520 con 0x7f52e4005260 2024-03-30T01:40:54.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.332+0000 7f52f17fa700 1 -- 172.21.15.31:0/395145498 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52e8057aa0 con 0x7f52e4005260 2024-03-30T01:40:54.339 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:40:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.334+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f52d806ac90 msgr2=0x7f52d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.334+0000 7f52fadd2700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f52d806ac90 0x7f52d806d150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52f4149a90 tx=0x7f52e0008040 comp rx=0 tx=0).stop 2024-03-30T01:40:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.334+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 msgr2=0x7f52f41ea080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.334+0000 7f52fadd2700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52f41ea080 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52e800f7b0 tx=0x7f52e800fb70 comp rx=0 tx=0).stop 2024-03-30T01:40:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.335+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 shutdown_connections 2024-03-30T01:40:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.335+0000 7f52fadd2700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e4005260 0x7f52f41ea080 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.335+0000 7f52fadd2700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f52d806ac90 0x7f52d806d150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.335+0000 7f52fadd2700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52f41581b0 0x7f52f41ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.335+0000 7f52fadd2700 1 --2- 172.21.15.31:0/395145498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52f406ac10 0x7f52f4157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.335+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 >> 172.21.15.31:0/395145498 conn(0x7f52f40c3980 msgr2=0x7f52f414fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.335+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 shutdown_connections 2024-03-30T01:40:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.335+0000 7f52fadd2700 1 -- 172.21.15.31:0/395145498 wait complete. 2024-03-30T01:40:54.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:40:54.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.483+0000 7f0b68e7c700 1 Processor -- start 2024-03-30T01:40:54.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.483+0000 7f0b68e7c700 1 -- start start 2024-03-30T01:40:54.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.484+0000 7f0b68e7c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0b6414fbf0 0x7f0b6406a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.484+0000 7f0b68e7c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0b640a9560 0x7f0b640ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.484+0000 7f0b68e7c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6406e4a0 con 0x7f0b6406d9e0 2024-03-30T01:40:54.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.484+0000 7f0b68e7c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b64156ac0 con 0x7f0b6414fbf0 2024-03-30T01:40:54.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.484+0000 7f0b68e7c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b64155bf0 con 0x7f0b640a9560 2024-03-30T01:40:54.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.484+0000 7f0b6259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0b6414fbf0 0x7f0b6406a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.484+0000 7f0b6259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0b6414fbf0 0x7f0b6406a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54050/0 (socket says 172.21.15.31:54050) 2024-03-30T01:40:54.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.484+0000 7f0b6259c700 1 -- 172.21.15.31:0/2954014505 learned_addr learned my addr 172.21.15.31:0/2954014505 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:54.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.485+0000 7f0b62d9d700 1 --2- 172.21.15.31:0/2954014505 >> v2:172.21.15.59:3300/0 conn(0x7f0b640a9560 0x7f0b640ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.485+0000 7f0b6259c700 1 -- 172.21.15.31:0/2954014505 >> v2:172.21.15.59:3300/0 conn(0x7f0b640a9560 msgr2=0x7f0b640ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.485+0000 7f0b6259c700 1 --2- 172.21.15.31:0/2954014505 >> v2:172.21.15.59:3300/0 conn(0x7f0b640a9560 0x7f0b640ab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.485+0000 7f0b6259c700 1 -- 172.21.15.31:0/2954014505 >> v1:172.21.15.59:6789/0 conn(0x7f0b6406d9e0 legacy=0x7f0b641563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.485+0000 7f0b6259c700 1 -- 172.21.15.31:0/2954014505 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6406dee0 con 0x7f0b6414fbf0 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.485+0000 7f0b6259c700 1 --2- 172.21.15.31:0/2954014505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6414fbf0 0x7f0b6406a700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b640c06d0 tx=0x7f0b54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8606a1f8a2cf505a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.485+0000 7f0b6159a700 1 -- 172.21.15.31:0/2954014505 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54004230 con 0x7f0b6414fbf0 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.485+0000 7f0b6159a700 1 -- 172.21.15.31:0/2954014505 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b54004390 con 0x7f0b6414fbf0 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.485+0000 7f0b6159a700 1 -- 172.21.15.31:0/2954014505 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b540036a0 con 0x7f0b6414fbf0 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.486+0000 7f0b68e7c700 1 -- 172.21.15.31:0/2954014505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6414fbf0 msgr2=0x7f0b6406a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.486+0000 7f0b68e7c700 1 --2- 172.21.15.31:0/2954014505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6414fbf0 0x7f0b6406a700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b640c06d0 tx=0x7f0b54009bd0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.486+0000 7f0b68e7c700 1 -- 172.21.15.31:0/2954014505 shutdown_connections 2024-03-30T01:40:54.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.486+0000 7f0b68e7c700 1 --2- 172.21.15.31:0/2954014505 >> v2:172.21.15.59:3300/0 conn(0x7f0b640a9560 0x7f0b640ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.486+0000 7f0b68e7c700 1 --2- 172.21.15.31:0/2954014505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6414fbf0 0x7f0b6406a700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.486+0000 7f0b68e7c700 1 -- 172.21.15.31:0/2954014505 >> 172.21.15.31:0/2954014505 conn(0x7f0b640c3190 msgr2=0x7f0b640c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:54.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.486+0000 7f0b68e7c700 1 -- 172.21.15.31:0/2954014505 shutdown_connections 2024-03-30T01:40:54.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.486+0000 7f0b68e7c700 1 -- 172.21.15.31:0/2954014505 wait complete. 2024-03-30T01:40:54.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.487+0000 7f0b68e7c700 1 Processor -- start 2024-03-30T01:40:54.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.487+0000 7f0b68e7c700 1 -- start start 2024-03-30T01:40:54.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b68e7c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6406d9e0 0x7f0b641ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b68e7c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b640a9560 0x7f0b641ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b68e7c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b6414fbf0 0x7f0b6414d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b68e7c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b641508f0 con 0x7f0b6406d9e0 2024-03-30T01:40:54.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b68e7c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b64151100 con 0x7f0b6414fbf0 2024-03-30T01:40:54.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b68e7c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b641edc20 con 0x7f0b640a9560 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b6259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6406d9e0 0x7f0b641ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b6259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6406d9e0 0x7f0b641ed1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54052/0 (socket says 172.21.15.31:54052) 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b6259c700 1 -- 172.21.15.31:0/661673970 learned_addr learned my addr 172.21.15.31:0/661673970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b62d9d700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b6414fbf0 0x7f0b6414d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.488+0000 7f0b61d9b700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b640a9560 0x7f0b641ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b6259c700 1 -- 172.21.15.31:0/661673970 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b640a9560 msgr2=0x7f0b641ed6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b6259c700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b640a9560 0x7f0b641ed6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b6259c700 1 -- 172.21.15.31:0/661673970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b6414fbf0 msgr2=0x7f0b6414d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b6259c700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b6414fbf0 0x7f0b6414d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b6259c700 1 -- 172.21.15.31:0/661673970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b54009550 con 0x7f0b6406d9e0 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b62d9d700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b6414fbf0 0x7f0b6414d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:40:54.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b6259c700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6406d9e0 0x7f0b641ed1a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009fd0 tx=0x7f0b54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b52ffd700 1 -- 172.21.15.31:0/661673970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54003e50 con 0x7f0b6406d9e0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b52ffd700 1 -- 172.21.15.31:0/661673970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b54011630 con 0x7f0b6406d9e0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b52ffd700 1 -- 172.21.15.31:0/661673970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b540119e0 con 0x7f0b6406d9e0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b64150b40 con 0x7f0b6406d9e0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.489+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b64151b60 con 0x7f0b6406d9e0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.490+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6404ea80 con 0x7f0b6406d9e0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.490+0000 7f0b52ffd700 1 -- 172.21.15.31:0/661673970 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0b54011b40 con 0x7f0b6406d9e0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.491+0000 7f0b52ffd700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0b4406ac90 0x7f0b4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.491+0000 7f0b52ffd700 1 -- 172.21.15.31:0/661673970 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0b5408a3d0 con 0x7f0b6406d9e0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.491+0000 7f0b61d9b700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0b4406ac90 0x7f0b4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.491+0000 7f0b61d9b700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0b4406ac90 0x7f0b4406d150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0b64068170 tx=0x7f0b58007f10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:54.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.496+0000 7f0b52ffd700 1 -- 172.21.15.31:0/661673970 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0b54055910 con 0x7f0b6406d9e0 2024-03-30T01:40:54.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.761+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b6406b8b0 con 0x7f0b6406d9e0 2024-03-30T01:40:54.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.761+0000 7f0b52ffd700 1 -- 172.21.15.31:0/661673970 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f0b54028de0 con 0x7f0b6406d9e0 2024-03-30T01:40:54.768 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:54.768 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0b4406ac90 msgr2=0x7f0b4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0b4406ac90 0x7f0b4406d150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0b64068170 tx=0x7f0b58007f10 comp rx=0 tx=0).stop 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6406d9e0 msgr2=0x7f0b641ed1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6406d9e0 0x7f0b641ed1a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009fd0 tx=0x7f0b54004750 comp rx=0 tx=0).stop 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 shutdown_connections 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b6406d9e0 0x7f0b641ed1a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0b4406ac90 0x7f0b4406d150 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b6414fbf0 0x7f0b6414d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 --2- 172.21.15.31:0/661673970 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b640a9560 0x7f0b641ed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 >> 172.21.15.31:0/661673970 conn(0x7f0b640c3190 msgr2=0x7f0b640ab030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 shutdown_connections 2024-03-30T01:40:54.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.764+0000 7f0b68e7c700 1 -- 172.21.15.31:0/661673970 wait complete. 2024-03-30T01:40:54.772 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2024-03-30T01:40:54.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:40:54.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.913+0000 7f893267e700 1 Processor -- start 2024-03-30T01:40:54.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.913+0000 7f893267e700 1 -- start start 2024-03-30T01:40:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f893267e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c0c46a0 0x7f892c0c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f893267e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c0a8460 con 0x7f892c154cf0 2024-03-30T01:40:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f893267e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c0c2e40 con 0x7f892c072270 2024-03-30T01:40:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f893267e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c06dcb0 con 0x7f892c0c46a0 2024-03-30T01:40:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f892bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f892c154cf0 0x7f892c06d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51696/0 (socket says 172.21.15.31:51696) 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f892bfff700 1 -- 172.21.15.31:0/1598052031 learned_addr learned my addr 172.21.15.31:0/1598052031 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f892b7fe700 1 --2- 172.21.15.31:0/1598052031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c0c46a0 0x7f892c0c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f892affd700 1 -- 172.21.15.31:0/1598052031 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971446990 0 0) 0x7f892c0a8460 con 0x7f892c154cf0 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f892affd700 1 -- 172.21.15.31:0/1598052031 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89180057c0 con 0x7f892c154cf0 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f892affd700 1 -- 172.21.15.31:0/1598052031 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949667379 0 0) 0x7f892c0c2e40 con 0x7f892c072270 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f892affd700 1 -- 172.21.15.31:0/1598052031 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f892c0a8460 con 0x7f892c072270 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f892b7fe700 1 -- 172.21.15.31:0/1598052031 >> v1:172.21.15.59:6789/0 conn(0x7f892c072270 legacy=0x7f892c151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.914+0000 7f892b7fe700 1 -- 172.21.15.31:0/1598052031 >> v1:172.21.15.31:6789/0 conn(0x7f892c154cf0 legacy=0x7f892c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.915+0000 7f892b7fe700 1 -- 172.21.15.31:0/1598052031 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c155630 con 0x7f892c0c46a0 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.915+0000 7f892b7fe700 1 --2- 172.21.15.31:0/1598052031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c0c46a0 0x7f892c0c32d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8914009870 tx=0x7f8914009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=620d8d864758278e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.915+0000 7f892affd700 1 -- 172.21.15.31:0/1598052031 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914011830 con 0x7f892c0c46a0 2024-03-30T01:40:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.915+0000 7f892affd700 1 -- 172.21.15.31:0/1598052031 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8914011990 con 0x7f892c0c46a0 2024-03-30T01:40:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.915+0000 7f892affd700 1 -- 172.21.15.31:0/1598052031 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914011d00 con 0x7f892c0c46a0 2024-03-30T01:40:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.915+0000 7f893267e700 1 -- 172.21.15.31:0/1598052031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c0c46a0 msgr2=0x7f892c0c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.915+0000 7f893267e700 1 --2- 172.21.15.31:0/1598052031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c0c46a0 0x7f892c0c32d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8914009870 tx=0x7f8914009b80 comp rx=0 tx=0).stop 2024-03-30T01:40:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.915+0000 7f893267e700 1 -- 172.21.15.31:0/1598052031 shutdown_connections 2024-03-30T01:40:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.915+0000 7f893267e700 1 --2- 172.21.15.31:0/1598052031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c0c46a0 0x7f892c0c32d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.916+0000 7f893267e700 1 -- 172.21.15.31:0/1598052031 >> 172.21.15.31:0/1598052031 conn(0x7f892c0c3a60 msgr2=0x7f892c0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.916+0000 7f893267e700 1 -- 172.21.15.31:0/1598052031 shutdown_connections 2024-03-30T01:40:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.916+0000 7f893267e700 1 -- 172.21.15.31:0/1598052031 wait complete. 2024-03-30T01:40:54.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.917+0000 7f893267e700 1 Processor -- start 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.917+0000 7f893267e700 1 -- start start 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f893267e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c072270 0x7f892c1e5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f893267e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f892c0c46a0 0x7f892c0c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f893267e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c154cf0 0x7f892c1416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f893267e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c14cf60 con 0x7f892c0c46a0 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f893267e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0c5d10 con 0x7f892c072270 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f893267e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c148740 con 0x7f892c154cf0 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f892c0c46a0 0x7f892c0c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f892c0c46a0 0x7f892c0c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54064/0 (socket says 172.21.15.31:54064) 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892b7fe700 1 -- 172.21.15.31:0/3176010145 learned_addr learned my addr 172.21.15.31:0/3176010145 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f8930c1b700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c154cf0 0x7f892c1416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892bfff700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c072270 0x7f892c1e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892b7fe700 1 -- 172.21.15.31:0/3176010145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c154cf0 msgr2=0x7f892c1416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892b7fe700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c154cf0 0x7f892c1416b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892b7fe700 1 -- 172.21.15.31:0/3176010145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c072270 msgr2=0x7f892c1e5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892b7fe700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c072270 0x7f892c1e5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892b7fe700 1 -- 172.21.15.31:0/3176010145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89140094a0 con 0x7f892c0c46a0 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892bfff700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c072270 0x7f892c1e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.918+0000 7f892b7fe700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f892c0c46a0 0x7f892c0c4f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89140097c0 tx=0x7f8914004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.919+0000 7f8928ff9700 1 -- 172.21.15.31:0/3176010145 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89140044d0 con 0x7f892c0c46a0 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.919+0000 7f8928ff9700 1 -- 172.21.15.31:0/3176010145 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891401f070 con 0x7f892c0c46a0 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.919+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c1e5810 con 0x7f892c0c46a0 2024-03-30T01:40:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.919+0000 7f8928ff9700 1 -- 172.21.15.31:0/3176010145 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89140038e0 con 0x7f892c0c46a0 2024-03-30T01:40:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.919+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c1e5dd0 con 0x7f892c0c46a0 2024-03-30T01:40:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.920+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892c0c5eb0 con 0x7f892c0c46a0 2024-03-30T01:40:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.920+0000 7f8928ff9700 1 -- 172.21.15.31:0/3176010145 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8914003a40 con 0x7f892c0c46a0 2024-03-30T01:40:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.921+0000 7f8928ff9700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f890c06ac90 0x7f890c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.921+0000 7f892bfff700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f890c06ac90 0x7f890c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.921+0000 7f8928ff9700 1 -- 172.21.15.31:0/3176010145 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8914021070 con 0x7f892c0c46a0 2024-03-30T01:40:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.921+0000 7f892bfff700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f890c06ac90 0x7f890c06d150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f891c000f60 tx=0x7f891c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:54.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:54.926+0000 7f8928ff9700 1 -- 172.21.15.31:0/3176010145 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8914055cb0 con 0x7f892c0c46a0 2024-03-30T01:40:55.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.143+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f892c04ea80 con 0x7f892c0c46a0 2024-03-30T01:40:55.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.144+0000 7f8928ff9700 1 -- 172.21.15.31:0/3176010145 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12334 (secure 0 0 0) 0x7f8914015090 con 0x7f892c0c46a0 2024-03-30T01:40:55.151 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:55.151 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:40:49.909825+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:40:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f890c06ac90 msgr2=0x7f890c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f890c06ac90 0x7f890c06d150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f891c000f60 tx=0x7f891c009040 comp rx=0 tx=0).stop 2024-03-30T01:40:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f892c0c46a0 msgr2=0x7f892c0c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f892c0c46a0 0x7f892c0c4f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89140097c0 tx=0x7f8914004750 comp rx=0 tx=0).stop 2024-03-30T01:40:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 shutdown_connections 2024-03-30T01:40:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f890c06ac90 0x7f890c06d150 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f892c154cf0 0x7f892c1416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f892c072270 0x7f892c1e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 --2- 172.21.15.31:0/3176010145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f892c0c46a0 0x7f892c0c4f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 >> 172.21.15.31:0/3176010145 conn(0x7f892c0c3a60 msgr2=0x7f892c06e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 shutdown_connections 2024-03-30T01:40:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.147+0000 7f893267e700 1 -- 172.21.15.31:0/3176010145 wait complete. 2024-03-30T01:40:55.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:40:55.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:40:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.299+0000 7f4035c65700 1 Processor -- start 2024-03-30T01:40:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.299+0000 7f4035c65700 1 -- start start 2024-03-30T01:40:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f4035c65700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4030151c90 0x7f403006a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f4035c65700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40300c1250 0x7f4030154680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f4035c65700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4030152070 con 0x7f403006d340 2024-03-30T01:40:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f4035c65700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40300acaa0 con 0x7f4030151c90 2024-03-30T01:40:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f4035c65700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40300c3790 con 0x7f40300c1250 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f402ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f403006d340 0x7f4030154bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51702/0 (socket says 172.21.15.31:51702) 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f402ffff700 1 -- 172.21.15.31:0/3537280417 learned_addr learned my addr 172.21.15.31:0/3537280417 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f402f7fe700 1 --2- 172.21.15.31:0/3537280417 >> v2:172.21.15.59:3300/0 conn(0x7f4030151c90 0x7f403006a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f402effd700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40300c1250 0x7f4030154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749610907 0 0) 0x7f4030152070 con 0x7f403006d340 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.300+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f401c0035b0 con 0x7f403006d340 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1836045690 0 0) 0x7f401c0035b0 con 0x7f403006d340 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40300c1250 msgr2=0x7f4030154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40300c1250 0x7f4030154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 >> v2:172.21.15.59:3300/0 conn(0x7f4030151c90 msgr2=0x7f403006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 --2- 172.21.15.31:0/3537280417 >> v2:172.21.15.59:3300/0 conn(0x7f4030151c90 0x7f403006a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40300acef0 con 0x7f403006d340 2024-03-30T01:40:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f40280019f0 con 0x7f403006d340 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402f7fe700 1 --2- 172.21.15.31:0/3537280417 >> v2:172.21.15.59:3300/0 conn(0x7f4030151c90 0x7f403006a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 >> v1:172.21.15.31:6789/0 conn(0x7f403006d340 legacy=0x7f4030154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40300c1250 0x7f401c003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f401c0040b0 0x7f401c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f401c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300acaa0 con 0x7f401c004d90 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300c3790 con 0x7f401c0040b0 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c00bc50 con 0x7f40300c1250 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402ffff700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f401c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402effd700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f401c0040b0 0x7f401c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402f7fe700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40300c1250 0x7f401c003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402ffff700 1 -- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40300c1250 msgr2=0x7f401c003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402ffff700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40300c1250 0x7f401c003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402ffff700 1 -- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f401c0040b0 msgr2=0x7f401c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402ffff700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f401c0040b0 0x7f401c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402ffff700 1 -- 172.21.15.31:0/3537280417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0035b0 con 0x7f401c004d90 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.301+0000 7f402ffff700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f401c005210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f402800a810 tx=0x7f401c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85afad2d8a94449 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402800a250 con 0x7f401c004d90 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4028012600 con 0x7f401c004d90 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f402e7fc700 1 -- 172.21.15.31:0/3537280417 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280129b0 con 0x7f401c004d90 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f4035c65700 1 -- 172.21.15.31:0/3537280417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 msgr2=0x7f401c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f4035c65700 1 --2- 172.21.15.31:0/3537280417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f401c005210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f402800a810 tx=0x7f401c005880 comp rx=0 tx=0).stop 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f402ffff700 1 -- 172.21.15.31:0/3537280417 reap_dead start 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f4035c65700 1 -- 172.21.15.31:0/3537280417 shutdown_connections 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f4035c65700 1 -- 172.21.15.31:0/3537280417 >> 172.21.15.31:0/3537280417 conn(0x7f40300c3910 msgr2=0x7f40300c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f4035c65700 1 -- 172.21.15.31:0/3537280417 shutdown_connections 2024-03-30T01:40:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.302+0000 7f4035c65700 1 -- 172.21.15.31:0/3537280417 wait complete. 2024-03-30T01:40:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.303+0000 7f4035c65700 1 Processor -- start 2024-03-30T01:40:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.304+0000 7f4035c65700 1 -- start start 2024-03-30T01:40:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.304+0000 7f4035c65700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f4030071c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.304+0000 7f4035c65700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40301edc10 0x7f40301e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.304+0000 7f4035c65700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4030157f90 0x7f40301e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.304+0000 7f4035c65700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300bff70 con 0x7f401c004d90 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.304+0000 7f4035c65700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300c04a0 con 0x7f40301edc10 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.304+0000 7f4035c65700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4030158ba0 con 0x7f4030157f90 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.304+0000 7f402f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f4030071c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f4030071c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54088/0 (socket says 172.21.15.31:54088) 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402f7fe700 1 -- 172.21.15.31:0/3967232642 learned_addr learned my addr 172.21.15.31:0/3967232642 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402effd700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40301edc10 0x7f40301e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402ffff700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4030157f90 0x7f40301e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402f7fe700 1 -- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4030157f90 msgr2=0x7f40301e9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402f7fe700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4030157f90 0x7f40301e9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402f7fe700 1 -- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40301edc10 msgr2=0x7f40301e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402f7fe700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40301edc10 0x7f40301e9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402f7fe700 1 -- 172.21.15.31:0/3967232642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40300c01c0 con 0x7f401c004d90 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402effd700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40301edc10 0x7f40301e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402ffff700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4030157f90 0x7f40301e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:40:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f402f7fe700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f4030071c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4020011c20 tx=0x7f402000f240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f4027fff700 1 -- 172.21.15.31:0/3967232642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402000fb80 con 0x7f401c004d90 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f4027fff700 1 -- 172.21.15.31:0/3967232642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402000fce0 con 0x7f401c004d90 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40301ed130 con 0x7f401c004d90 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f4027fff700 1 -- 172.21.15.31:0/3967232642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020004240 con 0x7f401c004d90 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.305+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40301ed6f0 con 0x7f401c004d90 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.306+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4030066f90 con 0x7f401c004d90 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.307+0000 7f4027fff700 1 -- 172.21.15.31:0/3967232642 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f40200043a0 con 0x7f401c004d90 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.307+0000 7f4027fff700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f401006ac90 0x7f401006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.307+0000 7f402effd700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f401006ac90 0x7f401006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.307+0000 7f4027fff700 1 -- 172.21.15.31:0/3967232642 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4020089620 con 0x7f401c004d90 2024-03-30T01:40:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.308+0000 7f402effd700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f401006ac90 0x7f401006d150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40301ee070 tx=0x7f4018008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:55.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.312+0000 7f4027fff700 1 -- 172.21.15.31:0/3967232642 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4020054be0 con 0x7f401c004d90 2024-03-30T01:40:55.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.525+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40300747d0 con 0x7f401c004d90 2024-03-30T01:40:55.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.525+0000 7f4027fff700 1 -- 172.21.15.31:0/3967232642 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4020022020 con 0x7f401c004d90 2024-03-30T01:40:55.532 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f401006ac90 msgr2=0x7f401006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f401006ac90 0x7f401006d150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40301ee070 tx=0x7f4018008040 comp rx=0 tx=0).stop 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 msgr2=0x7f4030071c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f4030071c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4020011c20 tx=0x7f402000f240 comp rx=0 tx=0).stop 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 shutdown_connections 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401c004d90 0x7f4030071c20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f401006ac90 0x7f401006d150 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4030157f90 0x7f40301e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 --2- 172.21.15.31:0/3967232642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40301edc10 0x7f40301e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 >> 172.21.15.31:0/3967232642 conn(0x7f40300c3910 msgr2=0x7f40300c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 shutdown_connections 2024-03-30T01:40:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.528+0000 7f4035c65700 1 -- 172.21.15.31:0/3967232642 wait complete. 2024-03-30T01:40:55.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T01:40:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.677+0000 7f682a9aa700 1 Processor -- start 2024-03-30T01:40:55.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.678+0000 7f682a9aa700 1 -- start start 2024-03-30T01:40:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.678+0000 7f682a9aa700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f682414fd20 0x7f6824153330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.678+0000 7f682a9aa700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68240c4fc0 con 0x7f68240c8230 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.678+0000 7f682a9aa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68240c87b0 con 0x7f68240c6d60 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.678+0000 7f682a9aa700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68240c8a90 con 0x7f682414fd20 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.678+0000 7f6823fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f682414fd20 0x7f6824153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6828f47700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f68240c8230 0x7f68240c72b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54396/0 (socket says 172.21.15.31:54396) 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6823fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f682414fd20 0x7f6824153330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55208/0 (socket says 172.21.15.31:55208) 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6828f47700 1 -- 172.21.15.31:0/1290732770 learned_addr learned my addr 172.21.15.31:0/1290732770 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6823fff700 1 -- 172.21.15.31:0/1290732770 >> v1:172.21.15.59:6789/0 conn(0x7f68240c6d60 legacy=0x7f68240c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6823fff700 1 -- 172.21.15.31:0/1290732770 >> v1:172.21.15.31:6789/0 conn(0x7f68240c8230 legacy=0x7f68240c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6823fff700 1 -- 172.21.15.31:0/1290732770 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6824152b30 con 0x7f682414fd20 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6822ffd700 1 -- 172.21.15.31:0/1290732770 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019912327 0 0) 0x7f68240c4fc0 con 0x7f68240c8230 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6823fff700 1 --2- 172.21.15.31:0/1290732770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 0x7f6824153330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6814009840 tx=0x7f6814009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5405eba75151873 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6822ffd700 1 -- 172.21.15.31:0/1290732770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814011830 con 0x7f682414fd20 2024-03-30T01:40:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6822ffd700 1 -- 172.21.15.31:0/1290732770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814011990 con 0x7f682414fd20 2024-03-30T01:40:55.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.679+0000 7f6822ffd700 1 -- 172.21.15.31:0/1290732770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814011d20 con 0x7f682414fd20 2024-03-30T01:40:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.680+0000 7f682a9aa700 1 -- 172.21.15.31:0/1290732770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 msgr2=0x7f6824153330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.680+0000 7f682a9aa700 1 --2- 172.21.15.31:0/1290732770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 0x7f6824153330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6814009840 tx=0x7f6814009b50 comp rx=0 tx=0).stop 2024-03-30T01:40:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.680+0000 7f682a9aa700 1 -- 172.21.15.31:0/1290732770 shutdown_connections 2024-03-30T01:40:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.680+0000 7f682a9aa700 1 --2- 172.21.15.31:0/1290732770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 0x7f6824153330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.680+0000 7f682a9aa700 1 -- 172.21.15.31:0/1290732770 >> 172.21.15.31:0/1290732770 conn(0x7f682406a8e0 msgr2=0x7f682406acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.680+0000 7f682a9aa700 1 -- 172.21.15.31:0/1290732770 shutdown_connections 2024-03-30T01:40:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.680+0000 7f682a9aa700 1 -- 172.21.15.31:0/1290732770 wait complete. 2024-03-30T01:40:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.681+0000 7f682a9aa700 1 Processor -- start 2024-03-30T01:40:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.682+0000 7f682a9aa700 1 -- start start 2024-03-30T01:40:55.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.682+0000 7f682a9aa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68240c6d60 0x7f682414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.682+0000 7f682a9aa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68240c8230 0x7f682414d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.682+0000 7f682a9aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 0x7f6824153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.682+0000 7f682a9aa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68240c5870 con 0x7f682414fd20 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.682+0000 7f682a9aa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6824156150 con 0x7f68240c6d60 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.682+0000 7f682a9aa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6824156430 con 0x7f68240c8230 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6828f47700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 0x7f6824153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6828f47700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 0x7f6824153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55218/0 (socket says 172.21.15.31:55218) 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6828f47700 1 -- 172.21.15.31:0/4219941117 learned_addr learned my addr 172.21.15.31:0/4219941117 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6823fff700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68240c6d60 0x7f682414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f68237fe700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68240c8230 0x7f682414d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6828f47700 1 -- 172.21.15.31:0/4219941117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68240c8230 msgr2=0x7f682414d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6828f47700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68240c8230 0x7f682414d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6828f47700 1 -- 172.21.15.31:0/4219941117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68240c6d60 msgr2=0x7f682414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6828f47700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68240c6d60 0x7f682414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6828f47700 1 -- 172.21.15.31:0/4219941117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68140094a0 con 0x7f682414fd20 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6828f47700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 0x7f6824153ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f681c00cc70 tx=0x7f681c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6820ff9700 1 -- 172.21.15.31:0/4219941117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00aaa0 con 0x7f682414fd20 2024-03-30T01:40:55.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6820ff9700 1 -- 172.21.15.31:0/4219941117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c00ac20 con 0x7f682414fd20 2024-03-30T01:40:55.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68240a9e30 con 0x7f682414fd20 2024-03-30T01:40:55.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f6820ff9700 1 -- 172.21.15.31:0/4219941117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c0070d0 con 0x7f682414fd20 2024-03-30T01:40:55.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.683+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68240aa3f0 con 0x7f682414fd20 2024-03-30T01:40:55.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.684+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6824073760 con 0x7f682414fd20 2024-03-30T01:40:55.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.685+0000 7f6820ff9700 1 -- 172.21.15.31:0/4219941117 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f681c007230 con 0x7f682414fd20 2024-03-30T01:40:55.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.685+0000 7f6820ff9700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f681006ac90 0x7f681006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:55.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.685+0000 7f6820ff9700 1 -- 172.21.15.31:0/4219941117 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f681c088ce0 con 0x7f682414fd20 2024-03-30T01:40:55.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.685+0000 7f6823fff700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f681006ac90 0x7f681006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:55.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.686+0000 7f6823fff700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f681006ac90 0x7f681006d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68140090b0 tx=0x7f681401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:55.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.690+0000 7f6820ff9700 1 -- 172.21.15.31:0/4219941117 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f681c0542a0 con 0x7f682414fd20 2024-03-30T01:40:55.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.907+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f68240c79a0 con 0x7f682414fd20 2024-03-30T01:40:55.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.907+0000 7f6820ff9700 1 -- 172.21.15.31:0/4219941117 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f68240c79a0 con 0x7f682414fd20 2024-03-30T01:40:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:40:55.914 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-03-30T01:40:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:40:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-03-30T01:40:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-03-30T01:40:55.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:40:55.914 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-03-30T01:40:55.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f681006ac90 msgr2=0x7f681006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f681006ac90 0x7f681006d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68140090b0 tx=0x7f681401c040 comp rx=0 tx=0).stop 2024-03-30T01:40:55.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 msgr2=0x7f6824153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:55.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 0x7f6824153ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f681c00cc70 tx=0x7f681c00cf80 comp rx=0 tx=0).stop 2024-03-30T01:40:55.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 shutdown_connections 2024-03-30T01:40:55.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f682414fd20 0x7f6824153ed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f681006ac90 0x7f681006d150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68240c6d60 0x7f682414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 --2- 172.21.15.31:0/4219941117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68240c8230 0x7f682414d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:55.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 >> 172.21.15.31:0/4219941117 conn(0x7f682406a8e0 msgr2=0x7f682406acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:55.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 shutdown_connections 2024-03-30T01:40:55.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:55.909+0000 7f682a9aa700 1 -- 172.21.15.31:0/4219941117 wait complete. 2024-03-30T01:40:55.929 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T01:40:55.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T01:40:56.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.063+0000 7fea2417d700 1 Processor -- start 2024-03-30T01:40:56.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.064+0000 7fea2417d700 1 -- start start 2024-03-30T01:40:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.064+0000 7fea2417d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fea1c13dba0 0x7fea1c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.064+0000 7fea2417d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fea1c06ecb0 0x7fea1c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea2417d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c0c0c50 con 0x7fea1c158370 2024-03-30T01:40:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea2417d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c1556a0 con 0x7fea1c06ecb0 2024-03-30T01:40:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea2417d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c0c0320 con 0x7fea1c13dba0 2024-03-30T01:40:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea21718700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fea1c06ecb0 0x7fea1c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea21718700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fea1c06ecb0 0x7fea1c157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55226/0 (socket says 172.21.15.31:55226) 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea21718700 1 -- 172.21.15.31:0/1976648570 learned_addr learned my addr 172.21.15.31:0/1976648570 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea21f19700 1 --2- 172.21.15.31:0/1976648570 >> v2:172.21.15.59:3300/0 conn(0x7fea1c13dba0 0x7fea1c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea21718700 1 -- 172.21.15.31:0/1976648570 >> v2:172.21.15.59:3300/0 conn(0x7fea1c13dba0 msgr2=0x7fea1c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea21718700 1 --2- 172.21.15.31:0/1976648570 >> v2:172.21.15.59:3300/0 conn(0x7fea1c13dba0 0x7fea1c06e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea21718700 1 -- 172.21.15.31:0/1976648570 >> v1:172.21.15.59:6789/0 conn(0x7fea1c158370 legacy=0x7fea1c06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea21718700 1 -- 172.21.15.31:0/1976648570 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c0c0600 con 0x7fea1c06ecb0 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea21718700 1 --2- 172.21.15.31:0/1976648570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c06ecb0 0x7fea1c157e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea1c073420 tx=0x7fea18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4107c2a044d76b8d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea20f17700 1 -- 172.21.15.31:0/1976648570 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18011830 con 0x7fea1c06ecb0 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea20f17700 1 -- 172.21.15.31:0/1976648570 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18011990 con 0x7fea1c06ecb0 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.065+0000 7fea20f17700 1 -- 172.21.15.31:0/1976648570 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18011d20 con 0x7fea1c06ecb0 2024-03-30T01:40:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.066+0000 7fea2417d700 1 -- 172.21.15.31:0/1976648570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c06ecb0 msgr2=0x7fea1c157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.066+0000 7fea2417d700 1 --2- 172.21.15.31:0/1976648570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c06ecb0 0x7fea1c157e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea1c073420 tx=0x7fea18009bd0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.066+0000 7fea2417d700 1 -- 172.21.15.31:0/1976648570 shutdown_connections 2024-03-30T01:40:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.066+0000 7fea2417d700 1 --2- 172.21.15.31:0/1976648570 >> v2:172.21.15.59:3300/0 conn(0x7fea1c13dba0 0x7fea1c06e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.066+0000 7fea2417d700 1 --2- 172.21.15.31:0/1976648570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c06ecb0 0x7fea1c157e30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.066+0000 7fea2417d700 1 -- 172.21.15.31:0/1976648570 >> 172.21.15.31:0/1976648570 conn(0x7fea1c0c39f0 msgr2=0x7fea1c0c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.066+0000 7fea2417d700 1 -- 172.21.15.31:0/1976648570 shutdown_connections 2024-03-30T01:40:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.066+0000 7fea2417d700 1 -- 172.21.15.31:0/1976648570 wait complete. 2024-03-30T01:40:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.067+0000 7fea2417d700 1 Processor -- start 2024-03-30T01:40:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.067+0000 7fea2417d700 1 -- start start 2024-03-30T01:40:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea2417d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea1c06ecb0 0x7fea1c1531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea2417d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea1c13dba0 0x7fea1c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea2417d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c158370 0x7fea1c0c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea2417d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c14f540 con 0x7fea1c158370 2024-03-30T01:40:56.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea2417d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c152750 con 0x7fea1c13dba0 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea2417d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0c6940 con 0x7fea1c06ecb0 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea2271a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c158370 0x7fea1c0c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea2271a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c158370 0x7fea1c0c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55230/0 (socket says 172.21.15.31:55230) 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea2271a700 1 -- 172.21.15.31:0/96955610 learned_addr learned my addr 172.21.15.31:0/96955610 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea21718700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea1c13dba0 0x7fea1c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.068+0000 7fea21f19700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea1c06ecb0 0x7fea1c1531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea2271a700 1 -- 172.21.15.31:0/96955610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea1c06ecb0 msgr2=0x7fea1c1531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea2271a700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea1c06ecb0 0x7fea1c1531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea2271a700 1 -- 172.21.15.31:0/96955610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea1c13dba0 msgr2=0x7fea1c156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea2271a700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea1c13dba0 0x7fea1c156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea2271a700 1 -- 172.21.15.31:0/96955610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea18009550 con 0x7fea1c158370 2024-03-30T01:40:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea2271a700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c158370 0x7fea1c0c4ae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea1400cc50 tx=0x7fea1400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea0e7fc700 1 -- 172.21.15.31:0/96955610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1400ad80 con 0x7fea1c158370 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea0e7fc700 1 -- 172.21.15.31:0/96955610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1400ce50 con 0x7fea1c158370 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1c1529f0 con 0x7fea1c158370 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea0e7fc700 1 -- 172.21.15.31:0/96955610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14003c40 con 0x7fea1c158370 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.069+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c0c6c20 con 0x7fea1c158370 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.070+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1c152f10 con 0x7fea1c158370 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.071+0000 7fea0e7fc700 1 -- 172.21.15.31:0/96955610 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fea140050d0 con 0x7fea1c158370 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.071+0000 7fea0e7fc700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fea0006acc0 0x7fea0006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.071+0000 7fea0e7fc700 1 -- 172.21.15.31:0/96955610 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fea14089100 con 0x7fea1c158370 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.071+0000 7fea21f19700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fea0006acc0 0x7fea0006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:56.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.072+0000 7fea21f19700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fea0006acc0 0x7fea0006d180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea10009b70 tx=0x7fea10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:56.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.076+0000 7fea0e7fc700 1 -- 172.21.15.31:0/96955610 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fea1c152f10 con 0x7fea1c158370 2024-03-30T01:40:56.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.292+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fea1c152f10 con 0x7fea1c158370 2024-03-30T01:40:56.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.940+0000 7fea0e7fc700 1 -- 172.21.15.31:0/96955610 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea1c152f10 con 0x7fea1c158370 2024-03-30T01:40:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fea0006acc0 msgr2=0x7fea0006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fea0006acc0 0x7fea0006d180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea10009b70 tx=0x7fea10008040 comp rx=0 tx=0).stop 2024-03-30T01:40:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c158370 msgr2=0x7fea1c0c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c158370 0x7fea1c0c4ae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea1400cc50 tx=0x7fea1400a340 comp rx=0 tx=0).stop 2024-03-30T01:40:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 shutdown_connections 2024-03-30T01:40:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea1c158370 0x7fea1c0c4ae0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fea0006acc0 0x7fea0006d180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea1c06ecb0 0x7fea1c1531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 --2- 172.21.15.31:0/96955610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea1c13dba0 0x7fea1c156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 >> 172.21.15.31:0/96955610 conn(0x7fea1c0c39f0 msgr2=0x7fea1c0c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 shutdown_connections 2024-03-30T01:40:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:56.945+0000 7fea2417d700 1 -- 172.21.15.31:0/96955610 wait complete. 2024-03-30T01:40:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-03-30T01:40:56.970 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-30T01:40:57.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.110+0000 7fb381a8e700 1 Processor -- start 2024-03-30T01:40:57.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.111+0000 7fb381a8e700 1 -- start start 2024-03-30T01:40:57.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.111+0000 7fb381a8e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c151c90 0x7fb37c06a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.111+0000 7fb381a8e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb37c06d340 0x7fb37c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.111+0000 7fb381a8e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb37c152070 con 0x7fb37c0c1250 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.111+0000 7fb381a8e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37c0acaa0 con 0x7fb37c06d340 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.111+0000 7fb381a8e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c0c3790 con 0x7fb37c151c90 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c151c90 0x7fb37c06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb37c06d340 0x7fb37c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37affd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb37c0c1250 0x7fb37c154680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:53516/0 (socket says 172.21.15.31:53516) 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37affd700 1 -- 172.21.15.31:0/676040838 learned_addr learned my addr 172.21.15.31:0/676040838 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb37c06d340 0x7fb37c154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:53702/0 (socket says 172.21.15.31:53702) 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c151c90 0x7fb37c06a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:47182/0 (socket says 172.21.15.31:47182) 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37b7fe700 1 -- 172.21.15.31:0/676040838 >> v2:172.21.15.59:3300/0 conn(0x7fb37c06d340 msgr2=0x7fb37c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37b7fe700 1 --2- 172.21.15.31:0/676040838 >> v2:172.21.15.59:3300/0 conn(0x7fb37c06d340 0x7fb37c154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37b7fe700 1 -- 172.21.15.31:0/676040838 >> v1:172.21.15.59:6789/0 conn(0x7fb37c0c1250 legacy=0x7fb37c154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37b7fe700 1 -- 172.21.15.31:0/676040838 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0acef0 con 0x7fb37c151c90 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37bfff700 1 --2- 172.21.15.31:0/676040838 >> v2:172.21.15.59:3300/0 conn(0x7fb37c06d340 0x7fb37c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37b7fe700 1 --2- 172.21.15.31:0/676040838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c151c90 0x7fb37c06a700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009840 tx=0x7fb36c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=febbf3a050036553 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.112+0000 7fb37a7fc700 1 -- 172.21.15.31:0/676040838 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c011830 con 0x7fb37c151c90 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb37a7fc700 1 -- 172.21.15.31:0/676040838 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36c011990 con 0x7fb37c151c90 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb37a7fc700 1 -- 172.21.15.31:0/676040838 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36c011d20 con 0x7fb37c151c90 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb381a8e700 1 -- 172.21.15.31:0/676040838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c151c90 msgr2=0x7fb37c06a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb381a8e700 1 --2- 172.21.15.31:0/676040838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c151c90 0x7fb37c06a700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009840 tx=0x7fb36c009b50 comp rx=0 tx=0).stop 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb381a8e700 1 -- 172.21.15.31:0/676040838 shutdown_connections 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb381a8e700 1 --2- 172.21.15.31:0/676040838 >> v2:172.21.15.59:3300/0 conn(0x7fb37c06d340 0x7fb37c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb381a8e700 1 --2- 172.21.15.31:0/676040838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c151c90 0x7fb37c06a700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb381a8e700 1 -- 172.21.15.31:0/676040838 >> 172.21.15.31:0/676040838 conn(0x7fb37c0c3910 msgr2=0x7fb37c0c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb381a8e700 1 -- 172.21.15.31:0/676040838 shutdown_connections 2024-03-30T01:40:57.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.113+0000 7fb381a8e700 1 -- 172.21.15.31:0/676040838 wait complete. 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.114+0000 7fb381a8e700 1 Processor -- start 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb381a8e700 1 -- start start 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb381a8e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c06d340 0x7fb37c153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb381a8e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb37c0c1250 0x7fb37c0befc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb381a8e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c151c90 0x7fb37c0bf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb381a8e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c0bde60 con 0x7fb37c151c90 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb381a8e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c1ed700 con 0x7fb37c0c1250 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb381a8e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c157ae0 con 0x7fb37c06d340 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb37bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c151c90 0x7fb37c0bf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb37bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c151c90 0x7fb37c0bf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55242/0 (socket says 172.21.15.31:55242) 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.115+0000 7fb37bfff700 1 -- 172.21.15.31:0/3131857826 learned_addr learned my addr 172.21.15.31:0/3131857826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:57.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb37b7fe700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c06d340 0x7fb37c153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb37affd700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb37c0c1250 0x7fb37c0befc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb37bfff700 1 -- 172.21.15.31:0/3131857826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c06d340 msgr2=0x7fb37c153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb37bfff700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c06d340 0x7fb37c153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb37bfff700 1 -- 172.21.15.31:0/3131857826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb37c0c1250 msgr2=0x7fb37c0befc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb37bfff700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb37c0c1250 0x7fb37c0befc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb37bfff700 1 -- 172.21.15.31:0/3131857826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb374009a70 con 0x7fb37c151c90 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb37affd700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb37c0c1250 0x7fb37c0befc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb37bfff700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c151c90 0x7fb37c0bf500 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb37400efc0 tx=0x7fb37400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb363fff700 1 -- 172.21.15.31:0/3131857826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400cb40 con 0x7fb37c151c90 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb363fff700 1 -- 172.21.15.31:0/3131857826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37400f040 con 0x7fb37c151c90 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c0094a0 con 0x7fb37c151c90 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb363fff700 1 -- 172.21.15.31:0/3131857826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374003a20 con 0x7fb37c151c90 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.116+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37c158b40 con 0x7fb37c151c90 2024-03-30T01:40:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.118+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37c066f90 con 0x7fb37c151c90 2024-03-30T01:40:57.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.118+0000 7fb363fff700 1 -- 172.21.15.31:0/3131857826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb374014480 con 0x7fb37c151c90 2024-03-30T01:40:57.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.119+0000 7fb363fff700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb36806ac90 0x7fb36806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:57.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.119+0000 7fb363fff700 1 -- 172.21.15.31:0/3131857826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb374089b40 con 0x7fb37c151c90 2024-03-30T01:40:57.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.119+0000 7fb37b7fe700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb36806ac90 0x7fb36806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:57.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.119+0000 7fb37b7fe700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb36806ac90 0x7fb36806d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb36c000c00 tx=0x7fb36c01d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:57.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.124+0000 7fb363fff700 1 -- 172.21.15.31:0/3131857826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb374054e60 con 0x7fb37c151c90 2024-03-30T01:40:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.341+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fb37c1edbe0 con 0x7fb37c151c90 2024-03-30T01:40:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.947+0000 7fb363fff700 1 -- 172.21.15.31:0/3131857826 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fb3740180d0 con 0x7fb37c151c90 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb36806ac90 msgr2=0x7fb36806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb36806ac90 0x7fb36806d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb36c000c00 tx=0x7fb36c01d040 comp rx=0 tx=0).stop 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c151c90 msgr2=0x7fb37c0bf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c151c90 0x7fb37c0bf500 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb37400efc0 tx=0x7fb37400c3a0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 shutdown_connections 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb37c151c90 0x7fb37c0bf500 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb36806ac90 0x7fb36806d150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb37c06d340 0x7fb37c153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 --2- 172.21.15.31:0/3131857826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb37c0c1250 0x7fb37c0befc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.963+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 >> 172.21.15.31:0/3131857826 conn(0x7fb37c0c3910 msgr2=0x7fb37c06d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:57.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.964+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 shutdown_connections 2024-03-30T01:40:57.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:57.965+0000 7fb381a8e700 1 -- 172.21.15.31:0/3131857826 wait complete. 2024-03-30T01:40:57.973 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-03-30T01:40:57.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T01:40:58.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:40:58.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:40:58.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.125+0000 7f52b3f4b700 1 Processor -- start 2024-03-30T01:40:58.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.126+0000 7f52b3f4b700 1 -- start start 2024-03-30T01:40:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.126+0000 7f52b3f4b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52ac0c46c0 0x7f52ac06ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.126+0000 7f52b3f4b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ac070610 con 0x7f52ac0c5d10 2024-03-30T01:40:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.126+0000 7f52b3f4b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ac0708f0 con 0x7f52ac0c4aa0 2024-03-30T01:40:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.126+0000 7f52b3f4b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac0c3210 con 0x7f52ac0c46c0 2024-03-30T01:40:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b14e6700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52ac0c5d10 0x7f52ac0c26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54412/0 (socket says 172.21.15.31:54412) 2024-03-30T01:40:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b14e6700 1 -- 172.21.15.31:0/2743623842 learned_addr learned my addr 172.21.15.31:0/2743623842 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:40:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b1ce7700 1 --2- 172.21.15.31:0/2743623842 >> v2:172.21.15.59:3300/0 conn(0x7f52ac0c46c0 0x7f52ac06ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381777730 0 0) 0x7f52ac070610 con 0x7f52ac0c5d10 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52980035b0 con 0x7f52ac0c5d10 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1508184510 0 0) 0x7f52980035b0 con 0x7f52ac0c5d10 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 >> v2:172.21.15.59:3300/0 conn(0x7f52ac0c46c0 msgr2=0x7f52ac06ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 --2- 172.21.15.31:0/2743623842 >> v2:172.21.15.59:3300/0 conn(0x7f52ac0c46c0 0x7f52ac06ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 >> v1:172.21.15.59:6789/0 conn(0x7f52ac0c4aa0 legacy=0x7f52ac15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0c34f0 con 0x7f52ac0c5d10 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f529c001670 con 0x7f52ac0c5d10 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 >> v1:172.21.15.31:6789/0 conn(0x7f52ac0c5d10 legacy=0x7f52ac0c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298003ae0 0x7f5298003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5298004850 0x7f5298004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52980052e0 0x7f529800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0c3210 con 0x7f5298003ae0 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529800c230 con 0x7f5298004850 2024-03-30T01:40:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529800c3f0 con 0x7f52980052e0 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b1ce7700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298003ae0 0x7f5298003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.127+0000 7f52b14e6700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5298004850 0x7f5298004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b24e8700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52980052e0 0x7f529800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b1ce7700 1 -- 172.21.15.31:0/2743623842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52980052e0 msgr2=0x7f529800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b1ce7700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52980052e0 0x7f529800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b1ce7700 1 -- 172.21.15.31:0/2743623842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5298004850 msgr2=0x7f5298004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b1ce7700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5298004850 0x7f5298004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b1ce7700 1 -- 172.21.15.31:0/2743623842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52980035b0 con 0x7f5298003ae0 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b24e8700 1 -- 172.21.15.31:0/2743623842 reap_dead start 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b1ce7700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298003ae0 0x7f5298003f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009b60 tx=0x7f5298005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3cb6a3b1047e10 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a800e6c0 con 0x7f5298003ae0 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a800e820 con 0x7f5298003ae0 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.128+0000 7f52b0ce5700 1 -- 172.21.15.31:0/2743623842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a800eb90 con 0x7f5298003ae0 2024-03-30T01:40:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.129+0000 7f52b3f4b700 1 -- 172.21.15.31:0/2743623842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298003ae0 msgr2=0x7f5298003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.129+0000 7f52b3f4b700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298003ae0 0x7f5298003f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009b60 tx=0x7f5298005880 comp rx=0 tx=0).stop 2024-03-30T01:40:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.129+0000 7f52b3f4b700 1 -- 172.21.15.31:0/2743623842 shutdown_connections 2024-03-30T01:40:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.129+0000 7f52b3f4b700 1 --2- 172.21.15.31:0/2743623842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298003ae0 0x7f5298003f00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.129+0000 7f52b3f4b700 1 -- 172.21.15.31:0/2743623842 >> 172.21.15.31:0/2743623842 conn(0x7f52ac0c7cf0 msgr2=0x7f52ac0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.129+0000 7f52b3f4b700 1 -- 172.21.15.31:0/2743623842 shutdown_connections 2024-03-30T01:40:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.129+0000 7f52b3f4b700 1 -- 172.21.15.31:0/2743623842 wait complete. 2024-03-30T01:40:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.130+0000 7f52b3f4b700 1 Processor -- start 2024-03-30T01:40:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.130+0000 7f52b3f4b700 1 -- start start 2024-03-30T01:40:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b3f4b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5298003ae0 0x7f52ac1ea510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b3f4b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0a7f40 0x7f52ac1d6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b3f4b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac1cffa0 0x7f52ac1d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b3f4b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac1eaa50 con 0x7f52ac0a7f40 2024-03-30T01:40:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b3f4b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0a8830 con 0x7f5298003ae0 2024-03-30T01:40:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b3f4b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac1d0990 con 0x7f52ac1cffa0 2024-03-30T01:40:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b14e6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0a7f40 0x7f52ac1d6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b14e6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0a7f40 0x7f52ac1d6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55260/0 (socket says 172.21.15.31:55260) 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b14e6700 1 -- 172.21.15.31:0/1776286121 learned_addr learned my addr 172.21.15.31:0/1776286121 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b1ce7700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5298003ae0 0x7f52ac1ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b24e8700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac1cffa0 0x7f52ac1d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b14e6700 1 -- 172.21.15.31:0/1776286121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac1cffa0 msgr2=0x7f52ac1d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b14e6700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac1cffa0 0x7f52ac1d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b14e6700 1 -- 172.21.15.31:0/1776286121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5298003ae0 msgr2=0x7f52ac1ea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b14e6700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5298003ae0 0x7f52ac1ea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b14e6700 1 -- 172.21.15.31:0/1776286121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac1d7410 con 0x7f52ac0a7f40 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52b14e6700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0a7f40 0x7f52ac1d6750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f529c007310 tx=0x7f529c007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.131+0000 7f52a27fc700 1 -- 172.21.15.31:0/1776286121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c011580 con 0x7f52ac0a7f40 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.132+0000 7f52a27fc700 1 -- 172.21.15.31:0/1776286121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c011720 con 0x7f52ac0a7f40 2024-03-30T01:40:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.132+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac0aa230 con 0x7f52ac0a7f40 2024-03-30T01:40:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.132+0000 7f52a27fc700 1 -- 172.21.15.31:0/1776286121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c011ad0 con 0x7f52ac0a7f40 2024-03-30T01:40:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.132+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac0a84c0 con 0x7f52ac0a7f40 2024-03-30T01:40:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.132+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac0747d0 con 0x7f52ac0a7f40 2024-03-30T01:40:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.134+0000 7f52a27fc700 1 -- 172.21.15.31:0/1776286121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f529c011c30 con 0x7f52ac0a7f40 2024-03-30T01:40:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.134+0000 7f52a27fc700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.134+0000 7f52a27fc700 1 -- 172.21.15.31:0/1776286121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f529c089d10 con 0x7f52ac0a7f40 2024-03-30T01:40:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.134+0000 7f52b1ce7700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.135+0000 7f52b1ce7700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f52a80090b0 tx=0x7f52a801e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.138+0000 7f52a27fc700 1 -- 172.21.15.31:0/1776286121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f529c054fc0 con 0x7f52ac0a7f40 2024-03-30T01:40:58.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.399+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f52ac0a8970 con 0x7f52ac0a7f40 2024-03-30T01:40:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.406+0000 7f52a27fc700 1 -- 172.21.15.31:0/1776286121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f529c0585e0 con 0x7f52ac0a7f40 2024-03-30T01:40:58.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.416+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 msgr2=0x7f529006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.416+0000 7f52b3f4b700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f52a80090b0 tx=0x7f52a801e040 comp rx=0 tx=0).stop 2024-03-30T01:40:58.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.416+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0a7f40 msgr2=0x7f52ac1d6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.416+0000 7f52b3f4b700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0a7f40 0x7f52ac1d6750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f529c007310 tx=0x7f529c007620 comp rx=0 tx=0).stop 2024-03-30T01:40:58.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.419+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 shutdown_connections 2024-03-30T01:40:58.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.419+0000 7f52b3f4b700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.419+0000 7f52b3f4b700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac1cffa0 0x7f52ac1d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.419+0000 7f52b3f4b700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5298003ae0 0x7f52ac1ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.419+0000 7f52b3f4b700 1 --2- 172.21.15.31:0/1776286121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0a7f40 0x7f52ac1d6750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.419+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 >> 172.21.15.31:0/1776286121 conn(0x7f52ac0c7cf0 msgr2=0x7f52ac0c1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:58.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.419+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 shutdown_connections 2024-03-30T01:40:58.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.419+0000 7f52b3f4b700 1 -- 172.21.15.31:0/1776286121 wait complete. 2024-03-30T01:40:58.428 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2024-03-30T01:40:58.448 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-30T01:40:58.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-30T01:40:58.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.625+0000 7fc44407b700 1 Processor -- start 2024-03-30T01:40:58.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.625+0000 7fc44407b700 1 -- start start 2024-03-30T01:40:58.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.626+0000 7fc44407b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc43c156230 0x7fc43c0725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.626+0000 7fc44407b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c072b30 0x7fc43c15c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.626+0000 7fc44407b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43c070c80 con 0x7fc43c0c24d0 2024-03-30T01:40:58.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.626+0000 7fc44407b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c071110 con 0x7fc43c156230 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.626+0000 7fc44407b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c0ac770 con 0x7fc43c072b30 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.626+0000 7fc441e17700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc43c156230 0x7fc43c0725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.626+0000 7fc441e17700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc43c156230 0x7fc43c0725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55264/0 (socket says 172.21.15.31:55264) 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.626+0000 7fc441e17700 1 -- 172.21.15.31:0/649695551 learned_addr learned my addr 172.21.15.31:0/649695551 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.626+0000 7fc441616700 1 --1- 172.21.15.31:0/649695551 >> v1:172.21.15.31:6789/0 conn(0x7fc43c0c24d0 0x7fc43c15c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54420/0 (socket says 172.21.15.31:54420) 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc442618700 1 --2- 172.21.15.31:0/649695551 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c072b30 0x7fc43c15c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc441e17700 1 -- 172.21.15.31:0/649695551 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c072b30 msgr2=0x7fc43c15c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc441e17700 1 --2- 172.21.15.31:0/649695551 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c072b30 0x7fc43c15c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc441e17700 1 -- 172.21.15.31:0/649695551 >> v1:172.21.15.31:6789/0 conn(0x7fc43c0c24d0 legacy=0x7fc43c15c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc440e15700 1 -- 172.21.15.31:0/649695551 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749880615 0 0) 0x7fc43c070c80 con 0x7fc43c0c24d0 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc441e17700 1 -- 172.21.15.31:0/649695551 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c066580 con 0x7fc43c156230 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc441e17700 1 --2- 172.21.15.31:0/649695551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c156230 0x7fc43c0725f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0098f0 tx=0x7fc42c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a477b6e7cd360b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc440e15700 1 -- 172.21.15.31:0/649695551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00b8a0 con 0x7fc43c156230 2024-03-30T01:40:58.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc440e15700 1 -- 172.21.15.31:0/649695551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c00ba00 con 0x7fc43c156230 2024-03-30T01:40:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc440e15700 1 -- 172.21.15.31:0/649695551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00bd90 con 0x7fc43c156230 2024-03-30T01:40:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc44407b700 1 -- 172.21.15.31:0/649695551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c156230 msgr2=0x7fc43c0725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.627+0000 7fc44407b700 1 --2- 172.21.15.31:0/649695551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c156230 0x7fc43c0725f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0098f0 tx=0x7fc42c009c00 comp rx=0 tx=0).stop 2024-03-30T01:40:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.628+0000 7fc44407b700 1 -- 172.21.15.31:0/649695551 shutdown_connections 2024-03-30T01:40:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.628+0000 7fc44407b700 1 --2- 172.21.15.31:0/649695551 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c072b30 0x7fc43c15c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.628+0000 7fc44407b700 1 --2- 172.21.15.31:0/649695551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c156230 0x7fc43c0725f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.628+0000 7fc44407b700 1 -- 172.21.15.31:0/649695551 >> 172.21.15.31:0/649695551 conn(0x7fc43c0c30c0 msgr2=0x7fc43c0c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.628+0000 7fc44407b700 1 -- 172.21.15.31:0/649695551 shutdown_connections 2024-03-30T01:40:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.628+0000 7fc44407b700 1 -- 172.21.15.31:0/649695551 wait complete. 2024-03-30T01:40:58.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.629+0000 7fc44407b700 1 Processor -- start 2024-03-30T01:40:58.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.629+0000 7fc44407b700 1 -- start start 2024-03-30T01:40:58.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc44407b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c072b30 0x7fc43c1f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc44407b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c0c24d0 0x7fc43c0bbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc44407b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c156230 0x7fc43c148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc44407b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c15a920 con 0x7fc43c072b30 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc44407b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c15b1c0 con 0x7fc43c156230 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc44407b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c1f1940 con 0x7fc43c0c24d0 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441e17700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c072b30 0x7fc43c1f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441e17700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c072b30 0x7fc43c1f0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55276/0 (socket says 172.21.15.31:55276) 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441e17700 1 -- 172.21.15.31:0/685402598 learned_addr learned my addr 172.21.15.31:0/685402598 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441616700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c0c24d0 0x7fc43c0bbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441e17700 1 -- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c0c24d0 msgr2=0x7fc43c0bbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc442618700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c156230 0x7fc43c148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441e17700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c0c24d0 0x7fc43c0bbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441e17700 1 -- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c156230 msgr2=0x7fc43c148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441e17700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c156230 0x7fc43c148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441e17700 1 -- 172.21.15.31:0/685402598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c009550 con 0x7fc43c072b30 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441616700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c0c24d0 0x7fc43c0bbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc442618700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c156230 0x7fc43c148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc441e17700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c072b30 0x7fc43c1f0fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc42c014040 tx=0x7fc42c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc4327fc700 1 -- 172.21.15.31:0/685402598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0036a0 con 0x7fc43c072b30 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc4327fc700 1 -- 172.21.15.31:0/685402598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c01e070 con 0x7fc43c072b30 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c15ab70 con 0x7fc43c072b30 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.630+0000 7fc4327fc700 1 -- 172.21.15.31:0/685402598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c003a40 con 0x7fc43c072b30 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.631+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c0c5ac0 con 0x7fc43c072b30 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.632+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c04ea80 con 0x7fc43c072b30 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.632+0000 7fc4327fc700 1 -- 172.21.15.31:0/685402598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc42c003ba0 con 0x7fc43c072b30 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.632+0000 7fc4327fc700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc42006ac90 0x7fc42006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.633+0000 7fc4327fc700 1 -- 172.21.15.31:0/685402598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc42c020070 con 0x7fc43c072b30 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.633+0000 7fc441616700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc42006ac90 0x7fc42006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.633+0000 7fc441616700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc42006ac90 0x7fc42006d150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc43c15c130 tx=0x7fc438009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:58.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.637+0000 7fc4327fc700 1 -- 172.21.15.31:0/685402598 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc42c055960 con 0x7fc43c072b30 2024-03-30T01:40:58.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:58.867+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fc43c073570 con 0x7fc43c072b30 2024-03-30T01:40:59.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.410+0000 7fc4327fc700 1 -- 172.21.15.31:0/685402598 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fc42c0150e0 con 0x7fc43c072b30 2024-03-30T01:40:59.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.413+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc42006ac90 msgr2=0x7fc42006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:59.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.413+0000 7fc44407b700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc42006ac90 0x7fc42006d150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc43c15c130 tx=0x7fc438009040 comp rx=0 tx=0).stop 2024-03-30T01:40:59.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.413+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c072b30 msgr2=0x7fc43c1f0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:59.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.413+0000 7fc44407b700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c072b30 0x7fc43c1f0fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc42c014040 tx=0x7fc42c004750 comp rx=0 tx=0).stop 2024-03-30T01:40:59.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.414+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 shutdown_connections 2024-03-30T01:40:59.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.414+0000 7fc44407b700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc43c072b30 0x7fc43c1f0fb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.414+0000 7fc44407b700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc42006ac90 0x7fc42006d150 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.414+0000 7fc44407b700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc43c156230 0x7fc43c148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.414+0000 7fc44407b700 1 --2- 172.21.15.31:0/685402598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc43c0c24d0 0x7fc43c0bbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.414+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 >> 172.21.15.31:0/685402598 conn(0x7fc43c0c30c0 msgr2=0x7fc43c154b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:59.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.414+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 shutdown_connections 2024-03-30T01:40:59.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.414+0000 7fc44407b700 1 -- 172.21.15.31:0/685402598 wait complete. 2024-03-30T01:40:59.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T01:40:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.578+0000 7f7bb5813700 1 Processor -- start 2024-03-30T01:40:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.579+0000 7f7bb5813700 1 -- start start 2024-03-30T01:40:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.579+0000 7f7bb5813700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bb013d810 0x7f7bb00c4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:59.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.579+0000 7f7bb5813700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013dd70 0x7f7bb006bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.579+0000 7f7bb5813700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7bb00c4eb0 0x7f7bb006c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.579+0000 7f7bb5813700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb013dbf0 con 0x7f7bb013d810 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.579+0000 7f7bb5813700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb0154970 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.579+0000 7f7bb5813700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0154c50 con 0x7f7bb013dd70 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.579+0000 7f7baeffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bb013d810 0x7f7bb00c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7baeffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bb013d810 0x7f7bb00c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55282/0 (socket says 172.21.15.31:55282) 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7baeffd700 1 -- 172.21.15.31:0/4047803075 learned_addr learned my addr 172.21.15.31:0/4047803075 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7bae7fc700 1 --2- 172.21.15.31:0/4047803075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013dd70 0x7f7bb006bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7baf7fe700 1 --2- 172.21.15.31:0/4047803075 >> v2:172.21.15.59:3300/0 conn(0x7f7bb00c4eb0 0x7f7bb006c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7baeffd700 1 -- 172.21.15.31:0/4047803075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013dd70 msgr2=0x7f7bb006bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7baeffd700 1 --2- 172.21.15.31:0/4047803075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013dd70 0x7f7bb006bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7baeffd700 1 -- 172.21.15.31:0/4047803075 >> v2:172.21.15.59:3300/0 conn(0x7f7bb00c4eb0 msgr2=0x7f7bb006c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:40:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7baeffd700 1 --2- 172.21.15.31:0/4047803075 >> v2:172.21.15.59:3300/0 conn(0x7f7bb00c4eb0 0x7f7bb006c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7baeffd700 1 -- 172.21.15.31:0/4047803075 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0154f30 con 0x7f7bb013d810 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7baeffd700 1 --2- 172.21.15.31:0/4047803075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb013d810 0x7f7bb00c4970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0158e70 tx=0x7f7ba4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9190b5c1a0d8ef4a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7badffb700 1 -- 172.21.15.31:0/4047803075 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400b870 con 0x7f7bb013d810 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7badffb700 1 -- 172.21.15.31:0/4047803075 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba400b9d0 con 0x7f7bb013d810 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.580+0000 7f7badffb700 1 -- 172.21.15.31:0/4047803075 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400bd60 con 0x7f7bb013d810 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.581+0000 7f7bb5813700 1 -- 172.21.15.31:0/4047803075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb013d810 msgr2=0x7f7bb00c4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.581+0000 7f7bb5813700 1 --2- 172.21.15.31:0/4047803075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb013d810 0x7f7bb00c4970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0158e70 tx=0x7f7ba4009bd0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.581+0000 7f7bb5813700 1 -- 172.21.15.31:0/4047803075 shutdown_connections 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.581+0000 7f7bb5813700 1 --2- 172.21.15.31:0/4047803075 >> v2:172.21.15.59:3300/0 conn(0x7f7bb00c4eb0 0x7f7bb006c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.581+0000 7f7bb5813700 1 --2- 172.21.15.31:0/4047803075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb013d810 0x7f7bb00c4970 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.581+0000 7f7bb5813700 1 --2- 172.21.15.31:0/4047803075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013dd70 0x7f7bb006bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.581+0000 7f7bb5813700 1 -- 172.21.15.31:0/4047803075 >> 172.21.15.31:0/4047803075 conn(0x7f7bb00c9ff0 msgr2=0x7f7bb00ca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.581+0000 7f7bb5813700 1 -- 172.21.15.31:0/4047803075 shutdown_connections 2024-03-30T01:40:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.581+0000 7f7bb5813700 1 -- 172.21.15.31:0/4047803075 wait complete. 2024-03-30T01:40:59.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.582+0000 7f7bb5813700 1 Processor -- start 2024-03-30T01:40:59.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.582+0000 7f7bb5813700 1 -- start start 2024-03-30T01:40:59.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7bb5813700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb00c4eb0 0x7f7bb00c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:59.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7bb5813700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013d810 0x7f7bb0137c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:59.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7bb5813700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb013dd70 0x7f7bb0151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:59.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7bb5813700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0138570 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7bb5813700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01f27a0 con 0x7f7bb013dd70 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7bb5813700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01f19e0 con 0x7f7bb013d810 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb00c4eb0 0x7f7bb00c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb00c4eb0 0x7f7bb00c28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55294/0 (socket says 172.21.15.31:55294) 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7baeffd700 1 -- 172.21.15.31:0/2911478235 learned_addr learned my addr 172.21.15.31:0/2911478235 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7bae7fc700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013d810 0x7f7bb0137c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7baf7fe700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb013dd70 0x7f7bb0151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7baeffd700 1 -- 172.21.15.31:0/2911478235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013d810 msgr2=0x7f7bb0137c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7baeffd700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013d810 0x7f7bb0137c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7baeffd700 1 -- 172.21.15.31:0/2911478235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb013dd70 msgr2=0x7f7bb0151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7baeffd700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb013dd70 0x7f7bb0151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.583+0000 7f7baeffd700 1 -- 172.21.15.31:0/2911478235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba4009550 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.584+0000 7f7baeffd700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb00c4eb0 0x7f7bb00c28b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4000c00 tx=0x7f7ba4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:59.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.584+0000 7f7b97fff700 1 -- 172.21.15.31:0/2911478235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4004310 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.584+0000 7f7b97fff700 1 -- 172.21.15.31:0/2911478235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba401e070 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.584+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb01f1cc0 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.584+0000 7f7b97fff700 1 -- 172.21.15.31:0/2911478235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40036a0 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.584+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb01f2280 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.585+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb01554d0 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.585+0000 7f7b97fff700 1 -- 172.21.15.31:0/2911478235 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7ba4020030 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.586+0000 7f7b97fff700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7b9c06ac90 0x7f7b9c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:40:59.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.586+0000 7f7b97fff700 1 -- 172.21.15.31:0/2911478235 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f7ba408a220 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.586+0000 7f7bae7fc700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7b9c06ac90 0x7f7b9c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:40:59.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.586+0000 7f7bae7fc700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7b9c06ac90 0x7f7b9c06d150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0158da0 tx=0x7f7ba0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:40:59.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.590+0000 7f7b97fff700 1 -- 172.21.15.31:0/2911478235 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7bb01554d0 con 0x7f7bb00c4eb0 2024-03-30T01:40:59.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:40:59.806+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7bb01554d0 con 0x7f7bb00c4eb0 2024-03-30T01:41:01.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.763+0000 7f7b97fff700 1 -- 172.21.15.31:0/2911478235 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f7bb01554d0 con 0x7f7bb00c4eb0 2024-03-30T01:41:01.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.768+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7b9c06ac90 msgr2=0x7f7b9c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.768+0000 7f7bb5813700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7b9c06ac90 0x7f7b9c06d150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0158da0 tx=0x7f7ba0008040 comp rx=0 tx=0).stop 2024-03-30T01:41:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.768+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb00c4eb0 msgr2=0x7f7bb00c28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.768+0000 7f7bb5813700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb00c4eb0 0x7f7bb00c28b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4000c00 tx=0x7f7ba4004750 comp rx=0 tx=0).stop 2024-03-30T01:41:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.768+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 shutdown_connections 2024-03-30T01:41:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.768+0000 7f7bb5813700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb00c4eb0 0x7f7bb00c28b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.768+0000 7f7bb5813700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7b9c06ac90 0x7f7b9c06d150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.768+0000 7f7bb5813700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb013dd70 0x7f7bb0151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.768+0000 7f7bb5813700 1 --2- 172.21.15.31:0/2911478235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb013d810 0x7f7bb0137c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.769+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 >> 172.21.15.31:0/2911478235 conn(0x7f7bb00c9ff0 msgr2=0x7f7bb0153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.769+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 shutdown_connections 2024-03-30T01:41:01.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.769+0000 7f7bb5813700 1 -- 172.21.15.31:0/2911478235 wait complete. 2024-03-30T01:41:01.783 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data_ec' created 2024-03-30T01:41:01.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-30T01:41:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.934+0000 7f192d9d8700 1 Processor -- start 2024-03-30T01:41:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.934+0000 7f192d9d8700 1 -- start start 2024-03-30T01:41:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f192d9d8700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19280af690 0x7f19281551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f192d9d8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19280c7f20 0x7f19280c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f192d9d8700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280c6170 con 0x7f19280c6c70 2024-03-30T01:41:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f192d9d8700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19280c49a0 con 0x7f19280af690 2024-03-30T01:41:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f192d9d8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19280c4bb0 con 0x7f19280c7f20 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f1926ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19280af690 0x7f19281551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f19277fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f19280c6c70 0x7f192806b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54422/0 (socket says 172.21.15.31:54422) 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f1926ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19280af690 0x7f19281551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55300/0 (socket says 172.21.15.31:55300) 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f19277fe700 1 -- 172.21.15.31:0/509082352 learned_addr learned my addr 172.21.15.31:0/509082352 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f19267fc700 1 --2- 172.21.15.31:0/509082352 >> v2:172.21.15.59:3300/0 conn(0x7f19280c7f20 0x7f19280c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f1926ffd700 1 -- 172.21.15.31:0/509082352 >> v2:172.21.15.59:3300/0 conn(0x7f19280c7f20 msgr2=0x7f19280c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f1926ffd700 1 --2- 172.21.15.31:0/509082352 >> v2:172.21.15.59:3300/0 conn(0x7f19280c7f20 0x7f19280c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f1926ffd700 1 -- 172.21.15.31:0/509082352 >> v1:172.21.15.31:6789/0 conn(0x7f19280c6c70 legacy=0x7f192806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f1926ffd700 1 -- 172.21.15.31:0/509082352 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280c4e90 con 0x7f19280af690 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.935+0000 7f1925ffb700 1 -- 172.21.15.31:0/509082352 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361346206 0 0) 0x7f1920001490 con 0x7f19280c6c70 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f1926ffd700 1 --2- 172.21.15.31:0/509082352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 0x7f19281551c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19280c7610 tx=0x7f1918009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c5a4775a4c0d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f1925ffb700 1 -- 172.21.15.31:0/509082352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918011830 con 0x7f19280af690 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f1925ffb700 1 -- 172.21.15.31:0/509082352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1918011990 con 0x7f19280af690 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f1925ffb700 1 -- 172.21.15.31:0/509082352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918011d20 con 0x7f19280af690 2024-03-30T01:41:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f192d9d8700 1 -- 172.21.15.31:0/509082352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 msgr2=0x7f19281551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f192d9d8700 1 --2- 172.21.15.31:0/509082352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 0x7f19281551c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19280c7610 tx=0x7f1918009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f192d9d8700 1 -- 172.21.15.31:0/509082352 shutdown_connections 2024-03-30T01:41:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f192d9d8700 1 --2- 172.21.15.31:0/509082352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 0x7f19281551c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f192d9d8700 1 --2- 172.21.15.31:0/509082352 >> v2:172.21.15.59:3300/0 conn(0x7f19280c7f20 0x7f19280c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f192d9d8700 1 -- 172.21.15.31:0/509082352 >> 172.21.15.31:0/509082352 conn(0x7f19280abd90 msgr2=0x7f19280ac1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f192d9d8700 1 -- 172.21.15.31:0/509082352 shutdown_connections 2024-03-30T01:41:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.936+0000 7f192d9d8700 1 -- 172.21.15.31:0/509082352 wait complete. 2024-03-30T01:41:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.938+0000 7f192d9d8700 1 Processor -- start 2024-03-30T01:41:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.938+0000 7f192d9d8700 1 -- start start 2024-03-30T01:41:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f192d9d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 0x7f19280c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f192d9d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19280c6c70 0x7f19281ea740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f192d9d8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19280c7f20 0x7f19280c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f192d9d8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19280c32c0 con 0x7f19280af690 2024-03-30T01:41:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f192d9d8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19281eac80 con 0x7f19280c6c70 2024-03-30T01:41:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f192d9d8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1928158bc0 con 0x7f19280c7f20 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f1926ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 0x7f19280c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f1926ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 0x7f19280c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55314/0 (socket says 172.21.15.31:55314) 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f1926ffd700 1 -- 172.21.15.31:0/681886337 learned_addr learned my addr 172.21.15.31:0/681886337 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f19267fc700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19280c6c70 0x7f19281ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f1926ffd700 1 -- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19280c7f20 msgr2=0x7f19280c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f19277fe700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19280c7f20 0x7f19280c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f1926ffd700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19280c7f20 0x7f19280c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f1926ffd700 1 -- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19280c6c70 msgr2=0x7f19281ea740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f1926ffd700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19280c6c70 0x7f19281ea740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f1926ffd700 1 -- 172.21.15.31:0/681886337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19180094a0 con 0x7f19280af690 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f19267fc700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19280c6c70 0x7f19281ea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f19277fe700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19280c7f20 0x7f19280c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f1926ffd700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 0x7f19280c40c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f19180090b0 tx=0x7f1918004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f190ffff700 1 -- 172.21.15.31:0/681886337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180038a0 con 0x7f19280af690 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f190ffff700 1 -- 172.21.15.31:0/681886337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191801e070 con 0x7f19280af690 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19281579b0 con 0x7f19280af690 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.939+0000 7f190ffff700 1 -- 172.21.15.31:0/681886337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918003c20 con 0x7f19280af690 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.940+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19281588e0 con 0x7f19280af690 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.941+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192804ea80 con 0x7f19280af690 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.941+0000 7f190ffff700 1 -- 172.21.15.31:0/681886337 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f191801a420 con 0x7f19280af690 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.942+0000 7f190ffff700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f191406ac90 0x7f191406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.942+0000 7f190ffff700 1 -- 172.21.15.31:0/681886337 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f191808a6a0 con 0x7f19280af690 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.942+0000 7f19267fc700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f191406ac90 0x7f191406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.942+0000 7f19267fc700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f191406ac90 0x7f191406d150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1928157040 tx=0x7f191c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:01.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:01.947+0000 7f190ffff700 1 -- 172.21.15.31:0/681886337 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1918055640 con 0x7f19280af690 2024-03-30T01:41:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.158+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f19280ad9c0 con 0x7f19280af690 2024-03-30T01:41:02.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.768+0000 7f190ffff700 1 -- 172.21.15.31:0/681886337 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1918058c60 con 0x7f19280af690 2024-03-30T01:41:02.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.776+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f191406ac90 msgr2=0x7f191406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:02.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.776+0000 7f192d9d8700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f191406ac90 0x7f191406d150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1928157040 tx=0x7f191c008040 comp rx=0 tx=0).stop 2024-03-30T01:41:02.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.776+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 msgr2=0x7f19280c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:02.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.776+0000 7f192d9d8700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 0x7f19280c40c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f19180090b0 tx=0x7f1918004750 comp rx=0 tx=0).stop 2024-03-30T01:41:02.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.777+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 shutdown_connections 2024-03-30T01:41:02.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.777+0000 7f192d9d8700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19280af690 0x7f19280c40c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:02.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.777+0000 7f192d9d8700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f191406ac90 0x7f191406d150 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:02.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.777+0000 7f192d9d8700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19280c7f20 0x7f19280c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:02.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.777+0000 7f192d9d8700 1 --2- 172.21.15.31:0/681886337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19280c6c70 0x7f19281ea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:02.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.777+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 >> 172.21.15.31:0/681886337 conn(0x7f19280abd90 msgr2=0x7f19280c0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:02.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.777+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 shutdown_connections 2024-03-30T01:41:02.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.777+0000 7f192d9d8700 1 -- 172.21.15.31:0/681886337 wait complete. 2024-03-30T01:41:02.799 INFO:teuthology.orchestra.run.smithi031.stderr:set pool 4 allow_ec_overwrites to true 2024-03-30T01:41:02.812 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-30T01:41:02.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.945+0000 7fe0994be700 1 Processor -- start 2024-03-30T01:41:02.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.946+0000 7fe0994be700 1 -- start start 2024-03-30T01:41:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.946+0000 7fe0994be700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 0x7fe0941546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.946+0000 7fe0994be700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0940c4c90 0x7fe0940c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.946+0000 7fe0994be700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe09406bc80 con 0x7fe09406c3f0 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.946+0000 7fe0994be700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0940c5110 con 0x7fe0940c4c90 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.946+0000 7fe0994be700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0940c6e80 con 0x7fe0940c48b0 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.946+0000 7fe0927fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0940c4c90 0x7fe0940c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe0927fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0940c4c90 0x7fe0940c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55316/0 (socket says 172.21.15.31:55316) 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe0927fc700 1 -- 172.21.15.31:0/1231750378 learned_addr learned my addr 172.21.15.31:0/1231750378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe092ffd700 1 --2- 172.21.15.31:0/1231750378 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 0x7fe0941546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe0927fc700 1 -- 172.21.15.31:0/1231750378 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 msgr2=0x7fe0941546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe0927fc700 1 --2- 172.21.15.31:0/1231750378 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 0x7fe0941546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe0927fc700 1 -- 172.21.15.31:0/1231750378 >> v1:172.21.15.59:6789/0 conn(0x7fe09406c3f0 legacy=0x7fe094157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe0927fc700 1 -- 172.21.15.31:0/1231750378 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0940c7120 con 0x7fe0940c4c90 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe0927fc700 1 --2- 172.21.15.31:0/1231750378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0940c4c90 0x7fe0940c6940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0880098f0 tx=0x7fe088009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbacb9e44320af9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe091ffb700 1 -- 172.21.15.31:0/1231750378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08800b8a0 con 0x7fe0940c4c90 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe091ffb700 1 -- 172.21.15.31:0/1231750378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08800ba00 con 0x7fe0940c4c90 2024-03-30T01:41:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.947+0000 7fe091ffb700 1 -- 172.21.15.31:0/1231750378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08800bd90 con 0x7fe0940c4c90 2024-03-30T01:41:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.948+0000 7fe0994be700 1 -- 172.21.15.31:0/1231750378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0940c4c90 msgr2=0x7fe0940c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.948+0000 7fe0994be700 1 --2- 172.21.15.31:0/1231750378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0940c4c90 0x7fe0940c6940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0880098f0 tx=0x7fe088009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.948+0000 7fe0994be700 1 -- 172.21.15.31:0/1231750378 shutdown_connections 2024-03-30T01:41:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.948+0000 7fe0994be700 1 --2- 172.21.15.31:0/1231750378 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 0x7fe0941546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.948+0000 7fe0994be700 1 --2- 172.21.15.31:0/1231750378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0940c4c90 0x7fe0940c6940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.948+0000 7fe0994be700 1 -- 172.21.15.31:0/1231750378 >> 172.21.15.31:0/1231750378 conn(0x7fe0940abac0 msgr2=0x7fe0940abed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.948+0000 7fe0994be700 1 -- 172.21.15.31:0/1231750378 shutdown_connections 2024-03-30T01:41:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.948+0000 7fe0994be700 1 -- 172.21.15.31:0/1231750378 wait complete. 2024-03-30T01:41:02.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.949+0000 7fe0994be700 1 Processor -- start 2024-03-30T01:41:02.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.949+0000 7fe0994be700 1 -- start start 2024-03-30T01:41:02.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe0994be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09406c3f0 0x7fe0940aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:02.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe0994be700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 0x7fe094152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:02.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe0994be700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0940c4c90 0x7fe0941ed220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:02.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe0994be700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0940c1000 con 0x7fe09406c3f0 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe0994be700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe094152760 con 0x7fe0940c4c90 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe0994be700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0940c1250 con 0x7fe0940c48b0 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe092ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09406c3f0 0x7fe0940aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe092ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09406c3f0 0x7fe0940aabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55330/0 (socket says 172.21.15.31:55330) 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe092ffd700 1 -- 172.21.15.31:0/1549051517 learned_addr learned my addr 172.21.15.31:0/1549051517 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe0937fe700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0940c4c90 0x7fe0941ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe0927fc700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 0x7fe094152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe092ffd700 1 -- 172.21.15.31:0/1549051517 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 msgr2=0x7fe094152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe092ffd700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 0x7fe094152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe092ffd700 1 -- 172.21.15.31:0/1549051517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0940c4c90 msgr2=0x7fe0941ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe092ffd700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0940c4c90 0x7fe0941ed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.950+0000 7fe092ffd700 1 -- 172.21.15.31:0/1549051517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe088009550 con 0x7fe09406c3f0 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.951+0000 7fe092ffd700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09406c3f0 0x7fe0940aabf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe08400d830 tx=0x7fe08400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.951+0000 7fe07b7fe700 1 -- 172.21.15.31:0/1549051517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0840125f0 con 0x7fe09406c3f0 2024-03-30T01:41:02.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.951+0000 7fe07b7fe700 1 -- 172.21.15.31:0/1549051517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe084012750 con 0x7fe09406c3f0 2024-03-30T01:41:02.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.951+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0940c1530 con 0x7fe09406c3f0 2024-03-30T01:41:02.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.951+0000 7fe07b7fe700 1 -- 172.21.15.31:0/1549051517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084012b00 con 0x7fe09406c3f0 2024-03-30T01:41:02.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.951+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0940ab130 con 0x7fe09406c3f0 2024-03-30T01:41:02.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.952+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe094152a40 con 0x7fe09406c3f0 2024-03-30T01:41:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.952+0000 7fe07b7fe700 1 -- 172.21.15.31:0/1549051517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe084012c60 con 0x7fe09406c3f0 2024-03-30T01:41:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.953+0000 7fe07b7fe700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.953+0000 7fe07b7fe700 1 -- 172.21.15.31:0/1549051517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fe084089920 con 0x7fe09406c3f0 2024-03-30T01:41:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.953+0000 7fe0927fc700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.953+0000 7fe0927fc700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe088009c40 tx=0x7fe088008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:02.958+0000 7fe07b7fe700 1 -- 172.21.15.31:0/1549051517 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe0840548c0 con 0x7fe09406c3f0 2024-03-30T01:41:03.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.227+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fe094152550 con 0x7fe09406c3f0 2024-03-30T01:41:03.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.232+0000 7fe07b7fe700 1 -- 172.21.15.31:0/1549051517 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fe084057ee0 con 0x7fe09406c3f0 2024-03-30T01:41:03.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 msgr2=0x7fe08006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe088009c40 tx=0x7fe088008030 comp rx=0 tx=0).stop 2024-03-30T01:41:03.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09406c3f0 msgr2=0x7fe0940aabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09406c3f0 0x7fe0940aabf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe08400d830 tx=0x7fe08400dbf0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 shutdown_connections 2024-03-30T01:41:03.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09406c3f0 0x7fe0940aabf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0940c4c90 0x7fe0941ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 --2- 172.21.15.31:0/1549051517 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0940c48b0 0x7fe094152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 >> 172.21.15.31:0/1549051517 conn(0x7fe0940abac0 msgr2=0x7fe09414dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:03.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.234+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 shutdown_connections 2024-03-30T01:41:03.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.235+0000 7fe0994be700 1 -- 172.21.15.31:0/1549051517 wait complete. 2024-03-30T01:41:03.242 INFO:teuthology.orchestra.run.smithi031.stderr:added data pool 4 to fsmap 2024-03-30T01:41:03.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.383+0000 7f98e5274700 1 Processor -- start 2024-03-30T01:41:03.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.383+0000 7f98e5274700 1 -- start start 2024-03-30T01:41:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98e5274700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f98e014fa40 0x7f98e0156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98e5274700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f98e00ab100 0x7f98e00c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98e5274700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e0071ee0 con 0x7f98e00c4100 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98e5274700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e0152570 con 0x7f98e014fa40 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98e5274700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e006d5c0 con 0x7f98e00ab100 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98deffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f98e014fa40 0x7f98e0156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98deffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f98e014fa40 0x7f98e0156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55342/0 (socket says 172.21.15.31:55342) 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98deffd700 1 -- 172.21.15.31:0/1650554570 learned_addr learned my addr 172.21.15.31:0/1650554570 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98df7fe700 1 --1- 172.21.15.31:0/1650554570 >> v1:172.21.15.31:6789/0 conn(0x7f98e00c4100 0x7f98e0152ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54426/0 (socket says 172.21.15.31:54426) 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.384+0000 7f98de7fc700 1 --2- 172.21.15.31:0/1650554570 >> v2:172.21.15.59:3300/0 conn(0x7f98e00ab100 0x7f98e00c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.385+0000 7f98deffd700 1 -- 172.21.15.31:0/1650554570 >> v2:172.21.15.59:3300/0 conn(0x7f98e00ab100 msgr2=0x7f98e00c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.385+0000 7f98deffd700 1 --2- 172.21.15.31:0/1650554570 >> v2:172.21.15.59:3300/0 conn(0x7f98e00ab100 0x7f98e00c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.385+0000 7f98deffd700 1 -- 172.21.15.31:0/1650554570 >> v1:172.21.15.31:6789/0 conn(0x7f98e00c4100 legacy=0x7f98e0152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.385+0000 7f98ddffb700 1 -- 172.21.15.31:0/1650554570 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650468267 0 0) 0x7f98e0071ee0 con 0x7f98e00c4100 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.385+0000 7f98deffd700 1 -- 172.21.15.31:0/1650554570 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e006d860 con 0x7f98e014fa40 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.385+0000 7f98deffd700 1 --2- 172.21.15.31:0/1650554570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 0x7f98e0156570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f98e00715e0 tx=0x7f98d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7919a35d2003d5fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.385+0000 7f98ddffb700 1 -- 172.21.15.31:0/1650554570 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4011830 con 0x7f98e014fa40 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.385+0000 7f98ddffb700 1 -- 172.21.15.31:0/1650554570 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d4011990 con 0x7f98e014fa40 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.385+0000 7f98ddffb700 1 -- 172.21.15.31:0/1650554570 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4011d20 con 0x7f98e014fa40 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.386+0000 7f98e5274700 1 -- 172.21.15.31:0/1650554570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 msgr2=0x7f98e0156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.386+0000 7f98e5274700 1 --2- 172.21.15.31:0/1650554570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 0x7f98e0156570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f98e00715e0 tx=0x7f98d4009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.386+0000 7f98e5274700 1 -- 172.21.15.31:0/1650554570 shutdown_connections 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.386+0000 7f98e5274700 1 --2- 172.21.15.31:0/1650554570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 0x7f98e0156570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.386+0000 7f98e5274700 1 --2- 172.21.15.31:0/1650554570 >> v2:172.21.15.59:3300/0 conn(0x7f98e00ab100 0x7f98e00c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.386+0000 7f98e5274700 1 -- 172.21.15.31:0/1650554570 >> 172.21.15.31:0/1650554570 conn(0x7f98e006ba20 msgr2=0x7f98e006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.386+0000 7f98e5274700 1 -- 172.21.15.31:0/1650554570 shutdown_connections 2024-03-30T01:41:03.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.386+0000 7f98e5274700 1 -- 172.21.15.31:0/1650554570 wait complete. 2024-03-30T01:41:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.387+0000 7f98e5274700 1 Processor -- start 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.387+0000 7f98e5274700 1 -- start start 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98e5274700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98e00ab100 0x7f98e0153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98e5274700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00c4100 0x7f98e00c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98e5274700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 0x7f98e01eaab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98e5274700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e01e7540 con 0x7f98e014fa40 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98e5274700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e01e84d0 con 0x7f98e00c4100 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98e5274700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0153f20 con 0x7f98e00ab100 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 0x7f98e01eaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 0x7f98e01eaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55348/0 (socket says 172.21.15.31:55348) 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98df7fe700 1 -- 172.21.15.31:0/2955146385 learned_addr learned my addr 172.21.15.31:0/2955146385 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98de7fc700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00c4100 0x7f98e00c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98deffd700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98e00ab100 0x7f98e0153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98df7fe700 1 -- 172.21.15.31:0/2955146385 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98e00ab100 msgr2=0x7f98e0153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98df7fe700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98e00ab100 0x7f98e0153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98df7fe700 1 -- 172.21.15.31:0/2955146385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00c4100 msgr2=0x7f98e00c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98df7fe700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00c4100 0x7f98e00c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.388+0000 7f98df7fe700 1 -- 172.21.15.31:0/2955146385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4009550 con 0x7f98e014fa40 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.389+0000 7f98df7fe700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 0x7f98e01eaab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98d800cca0 tx=0x7f98d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.389+0000 7f98c77fe700 1 -- 172.21.15.31:0/2955146385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800aaf0 con 0x7f98e014fa40 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.389+0000 7f98c77fe700 1 -- 172.21.15.31:0/2955146385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d800ac70 con 0x7f98e014fa40 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.389+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e00c6af0 con 0x7f98e014fa40 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.389+0000 7f98c77fe700 1 -- 172.21.15.31:0/2955146385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80070d0 con 0x7f98e014fa40 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.389+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e01e7cd0 con 0x7f98e014fa40 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.390+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e004ea80 con 0x7f98e014fa40 2024-03-30T01:41:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.390+0000 7f98c77fe700 1 -- 172.21.15.31:0/2955146385 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f98d8007230 con 0x7f98e014fa40 2024-03-30T01:41:03.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.391+0000 7f98c77fe700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98cc06ac90 0x7f98cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.391+0000 7f98c77fe700 1 -- 172.21.15.31:0/2955146385 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f98d8089150 con 0x7f98e014fa40 2024-03-30T01:41:03.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.391+0000 7f98deffd700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98cc06ac90 0x7f98cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.391+0000 7f98deffd700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98cc06ac90 0x7f98cc06d150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98d4014040 tx=0x7f98d4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:03.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.395+0000 7f98c77fe700 1 -- 172.21.15.31:0/2955146385 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f98d8054030 con 0x7f98e014fa40 2024-03-30T01:41:03.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.665+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98e00720d0 con 0x7f98e014fa40 2024-03-30T01:41:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.665+0000 7f98c77fe700 1 -- 172.21.15.31:0/2955146385 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f98d8018090 con 0x7f98e014fa40 2024-03-30T01:41:03.672 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:03.672 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:03.229632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-30T01:41:03.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98cc06ac90 msgr2=0x7f98cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98cc06ac90 0x7f98cc06d150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98d4014040 tx=0x7f98d4008030 comp rx=0 tx=0).stop 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 msgr2=0x7f98e01eaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 0x7f98e01eaab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f98d800cca0 tx=0x7f98d800a060 comp rx=0 tx=0).stop 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 shutdown_connections 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98e014fa40 0x7f98e01eaab0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98cc06ac90 0x7f98cc06d150 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98e00ab100 0x7f98e0153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 --2- 172.21.15.31:0/2955146385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98e00c4100 0x7f98e00c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 >> 172.21.15.31:0/2955146385 conn(0x7f98e006ba20 msgr2=0x7f98e00c1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 shutdown_connections 2024-03-30T01:41:03.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.668+0000 7f98e5274700 1 -- 172.21.15.31:0/2955146385 wait complete. 2024-03-30T01:41:03.676 INFO:tasks.ceph.mds.a.smithi031.stderr:2024-03-30T01:41:03.669+0000 7f0491f37700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T01:41:03.676 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-03-30T01:41:03.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:03.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.818+0000 7fd63bc60700 1 Processor -- start 2024-03-30T01:41:03.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.818+0000 7fd63bc60700 1 -- start start 2024-03-30T01:41:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63bc60700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6340c4f60 0x7fd6340c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63bc60700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd6340c62b0 0x7fd63415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63bc60700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6340c7fd0 con 0x7fd63406b400 2024-03-30T01:41:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63bc60700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd634154830 con 0x7fd6340c62b0 2024-03-30T01:41:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63bc60700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340c5380 con 0x7fd6340c4f60 2024-03-30T01:41:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63a1fd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd6340c62b0 0x7fd63415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63a1fd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd6340c62b0 0x7fd63415f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55354/0 (socket says 172.21.15.31:55354) 2024-03-30T01:41:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63a1fd700 1 -- 172.21.15.31:0/4201453377 learned_addr learned my addr 172.21.15.31:0/4201453377 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:03.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd6391fb700 1 --2- 172.21.15.31:0/4201453377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6340c4f60 0x7fd6340c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63a1fd700 1 -- 172.21.15.31:0/4201453377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6340c4f60 msgr2=0x7fd6340c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63a1fd700 1 --2- 172.21.15.31:0/4201453377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6340c4f60 0x7fd6340c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63a1fd700 1 -- 172.21.15.31:0/4201453377 >> v1:172.21.15.59:6789/0 conn(0x7fd63406b400 legacy=0x7fd634151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd63a1fd700 1 -- 172.21.15.31:0/4201453377 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd634154a20 con 0x7fd6340c62b0 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.819+0000 7fd6391fb700 1 --2- 172.21.15.31:0/4201453377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6340c4f60 0x7fd6340c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd63a1fd700 1 --2- 172.21.15.31:0/4201453377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c62b0 0x7fd63415f1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009e50 tx=0x7fd62c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b925c87dda8b53f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd6389fa700 1 -- 172.21.15.31:0/4201453377 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00e070 con 0x7fd6340c62b0 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd6389fa700 1 -- 172.21.15.31:0/4201453377 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c00b9c0 con 0x7fd6340c62b0 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd6389fa700 1 -- 172.21.15.31:0/4201453377 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00bd70 con 0x7fd6340c62b0 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd63bc60700 1 -- 172.21.15.31:0/4201453377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c62b0 msgr2=0x7fd63415f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd63bc60700 1 --2- 172.21.15.31:0/4201453377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c62b0 0x7fd63415f1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009e50 tx=0x7fd62c00b330 comp rx=0 tx=0).stop 2024-03-30T01:41:03.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd63bc60700 1 -- 172.21.15.31:0/4201453377 shutdown_connections 2024-03-30T01:41:03.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd63bc60700 1 --2- 172.21.15.31:0/4201453377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c62b0 0x7fd63415f1b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd63bc60700 1 --2- 172.21.15.31:0/4201453377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6340c4f60 0x7fd6340c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd63bc60700 1 -- 172.21.15.31:0/4201453377 >> 172.21.15.31:0/4201453377 conn(0x7fd6340ca0d0 msgr2=0x7fd6340ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:03.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.820+0000 7fd63bc60700 1 -- 172.21.15.31:0/4201453377 shutdown_connections 2024-03-30T01:41:03.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.821+0000 7fd63bc60700 1 -- 172.21.15.31:0/4201453377 wait complete. 2024-03-30T01:41:03.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.822+0000 7fd63bc60700 1 Processor -- start 2024-03-30T01:41:03.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.822+0000 7fd63bc60700 1 -- start start 2024-03-30T01:41:03.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.822+0000 7fd63bc60700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd63406b400 0x7fd6340c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.822+0000 7fd63bc60700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c4f60 0x7fd6340c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd63bc60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6340c62b0 0x7fd6341ea020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd63bc60700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6341586e0 con 0x7fd6340c4f60 2024-03-30T01:41:03.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd63bc60700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd634158120 con 0x7fd6340c62b0 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd63bc60700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd634158400 con 0x7fd63406b400 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6391fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c4f60 0x7fd6340c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6391fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c4f60 0x7fd6340c05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55370/0 (socket says 172.21.15.31:55370) 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6391fb700 1 -- 172.21.15.31:0/3421100627 learned_addr learned my addr 172.21.15.31:0/3421100627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd63a1fd700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6340c62b0 0x7fd6341ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6399fc700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd63406b400 0x7fd6340c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6391fb700 1 -- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd63406b400 msgr2=0x7fd6340c00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6391fb700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd63406b400 0x7fd6340c00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6391fb700 1 -- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6340c62b0 msgr2=0x7fd6341ea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6391fb700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6340c62b0 0x7fd6341ea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6391fb700 1 -- 172.21.15.31:0/3421100627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c009b30 con 0x7fd6340c4f60 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6399fc700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd63406b400 0x7fd6340c00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd63a1fd700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6340c62b0 0x7fd6341ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd6391fb700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c4f60 0x7fd6340c05f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd62400e7e0 tx=0x7fd62400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd62a7fc700 1 -- 172.21.15.31:0/3421100627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624004230 con 0x7fd6340c4f60 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd62a7fc700 1 -- 172.21.15.31:0/3421100627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd624004390 con 0x7fd6340c4f60 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6341566d0 con 0x7fd6340c4f60 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd62a7fc700 1 -- 172.21.15.31:0/3421100627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6240036a0 con 0x7fd6340c4f60 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.823+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6340c16d0 con 0x7fd6340c4f60 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.824+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6340bd980 con 0x7fd6340c4f60 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.825+0000 7fd62a7fc700 1 -- 172.21.15.31:0/3421100627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd624003800 con 0x7fd6340c4f60 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.825+0000 7fd62a7fc700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd61806ac90 0x7fd61806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.825+0000 7fd62a7fc700 1 -- 172.21.15.31:0/3421100627 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd624089160 con 0x7fd6340c4f60 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.826+0000 7fd6399fc700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd61806ac90 0x7fd61806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.826+0000 7fd6399fc700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd61806ac90 0x7fd61806d150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f60 tx=0x7fd630009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:03.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:03.830+0000 7fd62a7fc700 1 -- 172.21.15.31:0/3421100627 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd624053fb0 con 0x7fd6340c4f60 2024-03-30T01:41:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.046+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd63404ea80 con 0x7fd6340c4f60 2024-03-30T01:41:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.048+0000 7fd62a7fc700 1 -- 172.21.15.31:0/3421100627 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7fd6240575d0 con 0x7fd6340c4f60 2024-03-30T01:41:04.055 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:04.055 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd61806ac90 msgr2=0x7fd61806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd61806ac90 0x7fd61806d150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f60 tx=0x7fd630009040 comp rx=0 tx=0).stop 2024-03-30T01:41:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c4f60 msgr2=0x7fd6340c05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c4f60 0x7fd6340c05f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd62400e7e0 tx=0x7fd62400eba0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 shutdown_connections 2024-03-30T01:41:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd61806ac90 0x7fd61806d150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6340c62b0 0x7fd6341ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd63406b400 0x7fd6340c00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 --2- 172.21.15.31:0/3421100627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6340c4f60 0x7fd6340c05f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 >> 172.21.15.31:0/3421100627 conn(0x7fd6340ca0d0 msgr2=0x7fd6341526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 shutdown_connections 2024-03-30T01:41:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.050+0000 7fd63bc60700 1 -- 172.21.15.31:0/3421100627 wait complete. 2024-03-30T01:41:04.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:04.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.200+0000 7fd0b6a5e700 1 Processor -- start 2024-03-30T01:41:04.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.200+0000 7fd0b6a5e700 1 -- start start 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.200+0000 7fd0b6a5e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c4f60 0x7fd0b00c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.200+0000 7fd0b6a5e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b00c7fd0 con 0x7fd0b006b400 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0b6a5e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b0154a20 con 0x7fd0b00c62b0 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0b6a5e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b0154830 con 0x7fd0b00c4f60 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0affff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd0b006b400 0x7fd0b0151a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54434/0 (socket says 172.21.15.31:54434) 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0affff700 1 -- 172.21.15.31:0/2493573653 learned_addr learned my addr 172.21.15.31:0/2493573653 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/2493573653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c4f60 0x7fd0b00c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0aeffd700 1 -- 172.21.15.31:0/2493573653 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705478034 0 0) 0x7fd0b00c7fd0 con 0x7fd0b006b400 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0aeffd700 1 -- 172.21.15.31:0/2493573653 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0980035b0 con 0x7fd0b006b400 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0aeffd700 1 -- 172.21.15.31:0/2493573653 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914364448 0 0) 0x7fd0b0154a20 con 0x7fd0b00c62b0 2024-03-30T01:41:04.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0aeffd700 1 -- 172.21.15.31:0/2493573653 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b00c7fd0 con 0x7fd0b00c62b0 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0af7fe700 1 -- 172.21.15.31:0/2493573653 >> v1:172.21.15.59:6789/0 conn(0x7fd0b00c62b0 legacy=0x7fd0b015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0af7fe700 1 -- 172.21.15.31:0/2493573653 >> v1:172.21.15.31:6789/0 conn(0x7fd0b006b400 legacy=0x7fd0b0151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0aeffd700 1 -- 172.21.15.31:0/2493573653 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1516817190 0 0) 0x7fd0980035b0 con 0x7fd0b006b400 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0af7fe700 1 -- 172.21.15.31:0/2493573653 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b00c5590 con 0x7fd0b00c4f60 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.201+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/2493573653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c4f60 0x7fd0b00c5d70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009870 tx=0x7fd0a4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b3084420fb554cd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0aeffd700 1 -- 172.21.15.31:0/2493573653 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0001670 con 0x7fd0b00c4f60 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0aeffd700 1 -- 172.21.15.31:0/2493573653 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a4004030 con 0x7fd0b00c4f60 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0aeffd700 1 -- 172.21.15.31:0/2493573653 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40043c0 con 0x7fd0b00c4f60 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2493573653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c4f60 msgr2=0x7fd0b00c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0b6a5e700 1 --2- 172.21.15.31:0/2493573653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c4f60 0x7fd0b00c5d70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4009870 tx=0x7fd0a4009b80 comp rx=0 tx=0).stop 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2493573653 shutdown_connections 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0b6a5e700 1 --2- 172.21.15.31:0/2493573653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c4f60 0x7fd0b00c5d70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2493573653 >> 172.21.15.31:0/2493573653 conn(0x7fd0b00ca0d0 msgr2=0x7fd0b00ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:04.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2493573653 shutdown_connections 2024-03-30T01:41:04.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.202+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2493573653 wait complete. 2024-03-30T01:41:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.204+0000 7fd0b6a5e700 1 Processor -- start 2024-03-30T01:41:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.204+0000 7fd0b6a5e700 1 -- start start 2024-03-30T01:41:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.204+0000 7fd0b6a5e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006b400 0x7fd0b014fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.204+0000 7fd0b6a5e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0b00c4f60 0x7fd0b0156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.204+0000 7fd0b6a5e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c62b0 0x7fd0b0157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.204+0000 7fd0b6a5e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b0157d70 con 0x7fd0b006b400 2024-03-30T01:41:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.204+0000 7fd0b6a5e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00af9d0 con 0x7fd0b00c4f60 2024-03-30T01:41:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.204+0000 7fd0b6a5e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00afc40 con 0x7fd0b00c62b0 2024-03-30T01:41:04.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006b400 0x7fd0b014fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0affff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006b400 0x7fd0b014fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55386/0 (socket says 172.21.15.31:55386) 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0affff700 1 -- 172.21.15.31:0/2583151661 learned_addr learned my addr 172.21.15.31:0/2583151661 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0b00c4f60 0x7fd0b0156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0b4ffb700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c62b0 0x7fd0b0157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0affff700 1 -- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c62b0 msgr2=0x7fd0b0157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0affff700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c62b0 0x7fd0b0157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0affff700 1 -- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0b00c4f60 msgr2=0x7fd0b0156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0affff700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0b00c4f60 0x7fd0b0156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0affff700 1 -- 172.21.15.31:0/2583151661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a40094a0 con 0x7fd0b006b400 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0b00c4f60 0x7fd0b0156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0b4ffb700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c62b0 0x7fd0b0157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0affff700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006b400 0x7fd0b014fa50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000c7a0 tx=0x7fd0a000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0acff9700 1 -- 172.21.15.31:0/2583151661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0007560 con 0x7fd0b006b400 2024-03-30T01:41:04.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0acff9700 1 -- 172.21.15.31:0/2583151661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a00076c0 con 0x7fd0b006b400 2024-03-30T01:41:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b01ea6a0 con 0x7fd0b006b400 2024-03-30T01:41:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0acff9700 1 -- 172.21.15.31:0/2583151661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0004a70 con 0x7fd0b006b400 2024-03-30T01:41:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.205+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b01554f0 con 0x7fd0b006b400 2024-03-30T01:41:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.206+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b004ea80 con 0x7fd0b006b400 2024-03-30T01:41:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.207+0000 7fd0acff9700 1 -- 172.21.15.31:0/2583151661 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd0a0004bd0 con 0x7fd0b006b400 2024-03-30T01:41:04.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.207+0000 7fd0acff9700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd09c06ac90 0x7fd09c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.208+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd09c06ac90 0x7fd09c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.208+0000 7fd0acff9700 1 -- 172.21.15.31:0/2583151661 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd0a0088bc0 con 0x7fd0b006b400 2024-03-30T01:41:04.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.208+0000 7fd0af7fe700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd09c06ac90 0x7fd09c06d150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0b008c2c0 tx=0x7fd0a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:04.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.212+0000 7fd0acff9700 1 -- 172.21.15.31:0/2583151661 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd0a0053b40 con 0x7fd0b006b400 2024-03-30T01:41:04.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.426+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0b014c9d0 con 0x7fd0b006b400 2024-03-30T01:41:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.427+0000 7fd0acff9700 1 -- 172.21.15.31:0/2583151661 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7fd0a0018070 con 0x7fd0b006b400 2024-03-30T01:41:04.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:04.434 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd09c06ac90 msgr2=0x7fd09c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd09c06ac90 0x7fd09c06d150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0b008c2c0 tx=0x7fd0a4008030 comp rx=0 tx=0).stop 2024-03-30T01:41:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006b400 msgr2=0x7fd0b014fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006b400 0x7fd0b014fa50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000c7a0 tx=0x7fd0a000cb60 comp rx=0 tx=0).stop 2024-03-30T01:41:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 shutdown_connections 2024-03-30T01:41:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0b006b400 0x7fd0b014fa50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd09c06ac90 0x7fd09c06d150 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0b00c62b0 0x7fd0b0157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 --2- 172.21.15.31:0/2583151661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0b00c4f60 0x7fd0b0156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.429+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 >> 172.21.15.31:0/2583151661 conn(0x7fd0b00ca0d0 msgr2=0x7fd0b014d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.430+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 shutdown_connections 2024-03-30T01:41:04.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.430+0000 7fd0b6a5e700 1 -- 172.21.15.31:0/2583151661 wait complete. 2024-03-30T01:41:04.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:04.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.598+0000 7f77e5332700 1 Processor -- start 2024-03-30T01:41:04.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.599+0000 7f77e5332700 1 -- start start 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.599+0000 7f77e5332700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77e0157da0 0x7f77e0158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.599+0000 7f77e5332700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e006bf10 con 0x7f77e00c7270 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.599+0000 7f77e5332700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e00c5990 con 0x7f77e0151ac0 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.599+0000 7f77e5332700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e006b300 con 0x7f77e0157da0 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.599+0000 7f77df7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77e0157da0 0x7f77e0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77deffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f77e00c7270 0x7f77e00c5df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54450/0 (socket says 172.21.15.31:54450) 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77df7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77e0157da0 0x7f77e0158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55392/0 (socket says 172.21.15.31:55392) 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77deffd700 1 -- 172.21.15.31:0/1729911189 learned_addr learned my addr 172.21.15.31:0/1729911189 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77df7fe700 1 -- 172.21.15.31:0/1729911189 >> v1:172.21.15.59:6789/0 conn(0x7f77e0151ac0 legacy=0x7f77e0151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77df7fe700 1 -- 172.21.15.31:0/1729911189 >> v1:172.21.15.31:6789/0 conn(0x7f77e00c7270 legacy=0x7f77e00c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77ddffb700 1 -- 172.21.15.31:0/1729911189 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613266684 0 0) 0x7f77e006bf10 con 0x7f77e00c7270 2024-03-30T01:41:04.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77df7fe700 1 -- 172.21.15.31:0/1729911189 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e006b5e0 con 0x7f77e0157da0 2024-03-30T01:41:04.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77df7fe700 1 --2- 172.21.15.31:0/1729911189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0157da0 0x7f77e0158210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009d90 tx=0x7f77d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f37a3c79ef06a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:04.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77ddffb700 1 -- 172.21.15.31:0/1729911189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d800e070 con 0x7f77e0157da0 2024-03-30T01:41:04.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77ddffb700 1 -- 172.21.15.31:0/1729911189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d800bbd0 con 0x7f77e0157da0 2024-03-30T01:41:04.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.600+0000 7f77ddffb700 1 -- 172.21.15.31:0/1729911189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8012430 con 0x7f77e0157da0 2024-03-30T01:41:04.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.601+0000 7f77e5332700 1 -- 172.21.15.31:0/1729911189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0157da0 msgr2=0x7f77e0158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.601+0000 7f77e5332700 1 --2- 172.21.15.31:0/1729911189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0157da0 0x7f77e0158210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009d90 tx=0x7f77d800b330 comp rx=0 tx=0).stop 2024-03-30T01:41:04.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.601+0000 7f77e5332700 1 -- 172.21.15.31:0/1729911189 shutdown_connections 2024-03-30T01:41:04.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.601+0000 7f77e5332700 1 --2- 172.21.15.31:0/1729911189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0157da0 0x7f77e0158210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.601+0000 7f77e5332700 1 -- 172.21.15.31:0/1729911189 >> 172.21.15.31:0/1729911189 conn(0x7f77e00abf60 msgr2=0x7f77e00ac370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:04.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.601+0000 7f77e5332700 1 -- 172.21.15.31:0/1729911189 shutdown_connections 2024-03-30T01:41:04.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.601+0000 7f77e5332700 1 -- 172.21.15.31:0/1729911189 wait complete. 2024-03-30T01:41:04.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.602+0000 7f77e5332700 1 Processor -- start 2024-03-30T01:41:04.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.602+0000 7f77e5332700 1 -- start start 2024-03-30T01:41:04.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77e5332700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77e00c7270 0x7f77e00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77e5332700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0151ac0 0x7f77e00ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77e5332700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77e0157da0 0x7f77e01506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77e5332700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e01e01a0 con 0x7f77e0151ac0 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77e5332700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00ad490 con 0x7f77e0157da0 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77e5332700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00ace60 con 0x7f77e00c7270 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77de7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0151ac0 0x7f77e00ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77de7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0151ac0 0x7f77e00ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55402/0 (socket says 172.21.15.31:55402) 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77de7fc700 1 -- 172.21.15.31:0/3816260559 learned_addr learned my addr 172.21.15.31:0/3816260559 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77df7fe700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77e0157da0 0x7f77e01506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.603+0000 7f77deffd700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77e00c7270 0x7f77e00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77de7fc700 1 -- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77e00c7270 msgr2=0x7f77e00ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77de7fc700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77e00c7270 0x7f77e00ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77de7fc700 1 -- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77e0157da0 msgr2=0x7f77e01506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77de7fc700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77e0157da0 0x7f77e01506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77de7fc700 1 -- 172.21.15.31:0/3816260559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8009a70 con 0x7f77e0151ac0 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77deffd700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77e00c7270 0x7f77e00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77df7fe700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77e0157da0 0x7f77e01506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:04.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77de7fc700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0151ac0 0x7f77e00ad8b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c510 tx=0x7f77d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:04.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77c77fe700 1 -- 172.21.15.31:0/3816260559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d40052d0 con 0x7f77e0151ac0 2024-03-30T01:41:04.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77c77fe700 1 -- 172.21.15.31:0/3816260559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d4005430 con 0x7f77e0151ac0 2024-03-30T01:41:04.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77c77fe700 1 -- 172.21.15.31:0/3816260559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d40038c0 con 0x7f77e0151ac0 2024-03-30T01:41:04.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e00aea80 con 0x7f77e0151ac0 2024-03-30T01:41:04.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.604+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e00ad0b0 con 0x7f77e0151ac0 2024-03-30T01:41:04.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.605+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e00ad5d0 con 0x7f77e0151ac0 2024-03-30T01:41:04.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.606+0000 7f77c77fe700 1 -- 172.21.15.31:0/3816260559 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f77d4003a20 con 0x7f77e0151ac0 2024-03-30T01:41:04.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.606+0000 7f77c77fe700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f77cc06ac90 0x7f77cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:04.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.606+0000 7f77c77fe700 1 -- 172.21.15.31:0/3816260559 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f77d4088a50 con 0x7f77e0151ac0 2024-03-30T01:41:04.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.606+0000 7f77deffd700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f77cc06ac90 0x7f77cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:04.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.607+0000 7f77deffd700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f77cc06ac90 0x7f77cc06d150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f77d0000f60 tx=0x7f77d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:04.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.611+0000 7f77c77fe700 1 -- 172.21.15.31:0/3816260559 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f77e00ad5d0 con 0x7f77e0151ac0 2024-03-30T01:41:04.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.878+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77e00ad5d0 con 0x7f77e0151ac0 2024-03-30T01:41:04.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.878+0000 7f77c77fe700 1 -- 172.21.15.31:0/3816260559 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f77e00ad5d0 con 0x7f77e0151ac0 2024-03-30T01:41:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:04.885 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:03.229632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-30T01:41:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f77cc06ac90 msgr2=0x7f77cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f77cc06ac90 0x7f77cc06d150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f77d0000f60 tx=0x7f77d0009040 comp rx=0 tx=0).stop 2024-03-30T01:41:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0151ac0 msgr2=0x7f77e00ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0151ac0 0x7f77e00ad8b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c510 tx=0x7f77d400c8d0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 shutdown_connections 2024-03-30T01:41:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f77cc06ac90 0x7f77cc06d150 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77e0157da0 0x7f77e01506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77e00c7270 0x7f77e00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 --2- 172.21.15.31:0/3816260559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e0151ac0 0x7f77e00ad8b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.880+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 >> 172.21.15.31:0/3816260559 conn(0x7f77e00abf60 msgr2=0x7f77e014c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.881+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 shutdown_connections 2024-03-30T01:41:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:04.881+0000 7f77e5332700 1 -- 172.21.15.31:0/3816260559 wait complete. 2024-03-30T01:41:04.888 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-03-30T01:41:04.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:05.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.030+0000 7f2467b7a700 1 Processor -- start 2024-03-30T01:41:05.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.030+0000 7f2467b7a700 1 -- start start 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.031+0000 7f2467b7a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f24600c8310 0x7f24600c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.031+0000 7f2467b7a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600c5df0 0x7f2460151d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.031+0000 7f2467b7a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24601556d0 con 0x7f24600c2870 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.031+0000 7f2467b7a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24601559b0 con 0x7f24600c8310 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.031+0000 7f2467b7a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24600c1f70 con 0x7f24600c5df0 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.031+0000 7f2466117700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f24600c2870 0x7f24600a6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54458/0 (socket says 172.21.15.31:54458) 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.031+0000 7f2466117700 1 -- 172.21.15.31:0/1534032711 learned_addr learned my addr 172.21.15.31:0/1534032711 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2465916700 1 --2- 172.21.15.31:0/1534032711 >> v2:172.21.15.59:3300/0 conn(0x7f24600c8310 0x7f24600c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2465115700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600c5df0 0x7f2460151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016558887 0 0) 0x7f24601556d0 con 0x7f24600c2870 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c0035b0 con 0x7f24600c2870 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2380424263 0 0) 0x7f244c0035b0 con 0x7f24600c2870 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600c5df0 msgr2=0x7f2460151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600c5df0 0x7f2460151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 >> v2:172.21.15.59:3300/0 conn(0x7f24600c8310 msgr2=0x7f24600c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 --2- 172.21.15.31:0/1534032711 >> v2:172.21.15.59:3300/0 conn(0x7f24600c8310 0x7f24600c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600c2250 con 0x7f24600c2870 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f24580019f0 con 0x7f24600c2870 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 >> v1:172.21.15.31:6789/0 conn(0x7f24600c2870 legacy=0x7f24600a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f244c003a00 0x7f244c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600c5df0 0x7f244c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c004d90 0x7f244c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24601559b0 con 0x7f244c003a00 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24600c1f70 con 0x7f244c004d90 2024-03-30T01:41:05.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c00ba10 con 0x7f24600c5df0 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2465916700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f244c003a00 0x7f244c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2465115700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600c5df0 0x7f244c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.032+0000 7f2466117700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c004d90 0x7f244c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2465916700 1 -- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600c5df0 msgr2=0x7f244c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2465916700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600c5df0 0x7f244c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2465916700 1 -- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c004d90 msgr2=0x7f244c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2465916700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c004d90 0x7f244c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2465916700 1 -- 172.21.15.31:0/1534032711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c0035b0 con 0x7f244c003a00 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2466117700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c004d90 0x7f244c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2465916700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f244c003a00 0x7f244c003e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f245c009b90 tx=0x7f244c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb25da34f7c1ab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c00e6c0 con 0x7f244c003a00 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c00e820 con 0x7f244c003a00 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2464914700 1 -- 172.21.15.31:0/1534032711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c00eb90 con 0x7f244c003a00 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2467b7a700 1 -- 172.21.15.31:0/1534032711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f244c003a00 msgr2=0x7f244c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2467b7a700 1 --2- 172.21.15.31:0/1534032711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f244c003a00 0x7f244c003e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f245c009b90 tx=0x7f244c005880 comp rx=0 tx=0).stop 2024-03-30T01:41:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.033+0000 7f2466117700 1 -- 172.21.15.31:0/1534032711 reap_dead start 2024-03-30T01:41:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.034+0000 7f2467b7a700 1 -- 172.21.15.31:0/1534032711 shutdown_connections 2024-03-30T01:41:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.034+0000 7f2467b7a700 1 -- 172.21.15.31:0/1534032711 >> 172.21.15.31:0/1534032711 conn(0x7f246006ab20 msgr2=0x7f246006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.034+0000 7f2467b7a700 1 -- 172.21.15.31:0/1534032711 shutdown_connections 2024-03-30T01:41:05.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.034+0000 7f2467b7a700 1 -- 172.21.15.31:0/1534032711 wait complete. 2024-03-30T01:41:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.035+0000 7f2467b7a700 1 Processor -- start 2024-03-30T01:41:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.035+0000 7f2467b7a700 1 -- start start 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2467b7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c003a00 0x7f24601ea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2467b7a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2460157b70 0x7f24600a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2467b7a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600a9b80 0x7f246006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2467b7a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24601ea710 con 0x7f2460157b70 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2467b7a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246006fab0 con 0x7f244c003a00 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2467b7a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24600aca20 con 0x7f24600a9b80 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465115700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2460157b70 0x7f24600a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465115700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2460157b70 0x7f24600a9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55418/0 (socket says 172.21.15.31:55418) 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465115700 1 -- 172.21.15.31:0/685569899 learned_addr learned my addr 172.21.15.31:0/685569899 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465916700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c003a00 0x7f24601ea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2466117700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600a9b80 0x7f246006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465115700 1 -- 172.21.15.31:0/685569899 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600a9b80 msgr2=0x7f246006ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465115700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600a9b80 0x7f246006ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465115700 1 -- 172.21.15.31:0/685569899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c003a00 msgr2=0x7f24601ea1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465115700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c003a00 0x7f24601ea1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465115700 1 -- 172.21.15.31:0/685569899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600acd00 con 0x7f2460157b70 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2465115700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2460157b70 0x7f24600a9640 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2450011aa0 tx=0x7f2450011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f24567fc700 1 -- 172.21.15.31:0/685569899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245000f920 con 0x7f2460157b70 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f24567fc700 1 -- 172.21.15.31:0/685569899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245000fa80 con 0x7f2460157b70 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.036+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246006f740 con 0x7f2460157b70 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.037+0000 7f24567fc700 1 -- 172.21.15.31:0/685569899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245000fe30 con 0x7f2460157b70 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.037+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24600aa000 con 0x7f2460157b70 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.038+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2460073760 con 0x7f2460157b70 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.038+0000 7f24567fc700 1 -- 172.21.15.31:0/685569899 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2450014440 con 0x7f2460157b70 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.038+0000 7f24567fc700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.039+0000 7f24567fc700 1 -- 172.21.15.31:0/685569899 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2450089cb0 con 0x7f2460157b70 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.039+0000 7f2465916700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.039+0000 7f2465916700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f245c009ab0 tx=0x7f245c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.043+0000 7f24567fc700 1 -- 172.21.15.31:0/685569899 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2450054cf0 con 0x7f2460157b70 2024-03-30T01:41:05.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.260+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f246006f5a0 con 0x7f2460157b70 2024-03-30T01:41:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.261+0000 7f24567fc700 1 -- 172.21.15.31:0/685569899 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f246006f5a0 con 0x7f2460157b70 2024-03-30T01:41:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:05.268 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.263+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 msgr2=0x7f244406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.263+0000 7f2467b7a700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f245c009ab0 tx=0x7f245c008030 comp rx=0 tx=0).stop 2024-03-30T01:41:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.263+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2460157b70 msgr2=0x7f24600a9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.263+0000 7f2467b7a700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2460157b70 0x7f24600a9640 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2450011aa0 tx=0x7f2450011e60 comp rx=0 tx=0).stop 2024-03-30T01:41:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.264+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 shutdown_connections 2024-03-30T01:41:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.264+0000 7f2467b7a700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.264+0000 7f2467b7a700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24600a9b80 0x7f246006ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.264+0000 7f2467b7a700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244c003a00 0x7f24601ea1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.264+0000 7f2467b7a700 1 --2- 172.21.15.31:0/685569899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2460157b70 0x7f24600a9640 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.264+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 >> 172.21.15.31:0/685569899 conn(0x7f246006ab20 msgr2=0x7f24601509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.264+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 shutdown_connections 2024-03-30T01:41:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.264+0000 7f2467b7a700 1 -- 172.21.15.31:0/685569899 wait complete. 2024-03-30T01:41:05.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T01:40:58.401733+0000', 'modified': '2024-03-30T01:41:03.229632+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4269}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4269': {'gid': 4269, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6837/3724657244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6835', 'nonce': 3724657244}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 3724657244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-30T01:41:05.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T01:41:05.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T01:41:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.826+0000 7f194a981700 1 Processor -- start 2024-03-30T01:41:05.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.826+0000 7f194a981700 1 -- start start 2024-03-30T01:41:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.827+0000 7f194a981700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19440c3180 0x7f194406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.827+0000 7f194a981700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f194406a700 0x7f194406f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.827+0000 7f194a981700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 0x7f19440c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.827+0000 7f194a981700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19441547d0 con 0x7f19440c3180 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.827+0000 7f194a981700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1944154a90 con 0x7f194406a700 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.827+0000 7f194a981700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1944154d70 con 0x7f19440c4710 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.827+0000 7f1943fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19440c3180 0x7f194406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.827+0000 7f1943fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19440c3180 0x7f194406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51912/0 (socket says 172.21.15.31:51912) 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.827+0000 7f1943fff700 1 -- 172.21.15.31:0/1537238496 learned_addr learned my addr 172.21.15.31:0/1537238496 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f19437fe700 1 --2- 172.21.15.31:0/1537238496 >> v2:172.21.15.59:3300/0 conn(0x7f194406a700 0x7f194406f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1948f1e700 1 --2- 172.21.15.31:0/1537238496 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 0x7f19440c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1943fff700 1 -- 172.21.15.31:0/1537238496 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 msgr2=0x7f19440c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1943fff700 1 --2- 172.21.15.31:0/1537238496 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 0x7f19440c0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1943fff700 1 -- 172.21.15.31:0/1537238496 >> v2:172.21.15.59:3300/0 conn(0x7f194406a700 msgr2=0x7f194406f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1943fff700 1 --2- 172.21.15.31:0/1537238496 >> v2:172.21.15.59:3300/0 conn(0x7f194406a700 0x7f194406f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1943fff700 1 -- 172.21.15.31:0/1537238496 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944155050 con 0x7f19440c3180 2024-03-30T01:41:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1948f1e700 1 --2- 172.21.15.31:0/1537238496 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 0x7f19440c0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1943fff700 1 --2- 172.21.15.31:0/1537238496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19440c3180 0x7f194406eb00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f19440673d0 tx=0x7f1934009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e19bb045e4b7ce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1942ffd700 1 -- 172.21.15.31:0/1537238496 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400b870 con 0x7f19440c3180 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1942ffd700 1 -- 172.21.15.31:0/1537238496 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193400b9d0 con 0x7f19440c3180 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.828+0000 7f1942ffd700 1 -- 172.21.15.31:0/1537238496 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400bd60 con 0x7f19440c3180 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.829+0000 7f194a981700 1 -- 172.21.15.31:0/1537238496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19440c3180 msgr2=0x7f194406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.829+0000 7f194a981700 1 --2- 172.21.15.31:0/1537238496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19440c3180 0x7f194406eb00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f19440673d0 tx=0x7f1934009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.829+0000 7f194a981700 1 -- 172.21.15.31:0/1537238496 shutdown_connections 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.829+0000 7f194a981700 1 --2- 172.21.15.31:0/1537238496 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 0x7f19440c0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.829+0000 7f194a981700 1 --2- 172.21.15.31:0/1537238496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19440c3180 0x7f194406eb00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.829+0000 7f194a981700 1 --2- 172.21.15.31:0/1537238496 >> v2:172.21.15.59:3300/0 conn(0x7f194406a700 0x7f194406f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.829+0000 7f194a981700 1 -- 172.21.15.31:0/1537238496 >> 172.21.15.31:0/1537238496 conn(0x7f19440c3a60 msgr2=0x7f19440c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.829+0000 7f194a981700 1 -- 172.21.15.31:0/1537238496 shutdown_connections 2024-03-30T01:41:05.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.829+0000 7f194a981700 1 -- 172.21.15.31:0/1537238496 wait complete. 2024-03-30T01:41:05.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.830+0000 7f194a981700 1 Processor -- start 2024-03-30T01:41:05.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.830+0000 7f194a981700 1 -- start start 2024-03-30T01:41:05.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f194a981700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194406a700 0x7f194406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f194a981700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c3180 0x7f19441e9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f194a981700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 0x7f19441ea450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f194a981700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19441563d0 con 0x7f194406a700 2024-03-30T01:41:05.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f194a981700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19441eaf40 con 0x7f19440c3180 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f194a981700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19441e9a50 con 0x7f19440c4710 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f1943fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194406a700 0x7f194406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f1943fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194406a700 0x7f194406db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51922/0 (socket says 172.21.15.31:51922) 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f1943fff700 1 -- 172.21.15.31:0/3279328576 learned_addr learned my addr 172.21.15.31:0/3279328576 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f19437fe700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c3180 0x7f19441e9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f1948f1e700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 0x7f19441ea450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f1943fff700 1 -- 172.21.15.31:0/3279328576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 msgr2=0x7f19441ea450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f1943fff700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 0x7f19441ea450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f1943fff700 1 -- 172.21.15.31:0/3279328576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c3180 msgr2=0x7f19441e9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f1943fff700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c3180 0x7f19441e9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.831+0000 7f1943fff700 1 -- 172.21.15.31:0/3279328576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934009550 con 0x7f194406a700 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.832+0000 7f1943fff700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194406a700 0x7f194406db30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1934000c00 tx=0x7f1934004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.832+0000 7f1940ff9700 1 -- 172.21.15.31:0/3279328576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934004480 con 0x7f194406a700 2024-03-30T01:41:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.832+0000 7f1940ff9700 1 -- 172.21.15.31:0/3279328576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19340045e0 con 0x7f194406a700 2024-03-30T01:41:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.832+0000 7f194a981700 1 -- 172.21.15.31:0/3279328576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1944147bc0 con 0x7f194406a700 2024-03-30T01:41:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.832+0000 7f1940ff9700 1 -- 172.21.15.31:0/3279328576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19340038e0 con 0x7f194406a700 2024-03-30T01:41:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.832+0000 7f194a981700 1 -- 172.21.15.31:0/3279328576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1944148140 con 0x7f194406a700 2024-03-30T01:41:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.833+0000 7f1940ff9700 1 -- 172.21.15.31:0/3279328576 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1934021070 con 0x7f194406a700 2024-03-30T01:41:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.834+0000 7f1940ff9700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f193006ac90 0x7f193006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.834+0000 7f1940ff9700 1 -- 172.21.15.31:0/3279328576 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f193408ad40 con 0x7f194406a700 2024-03-30T01:41:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.834+0000 7f19437fe700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f193006ac90 0x7f193006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.834+0000 7f19437fe700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f193006ac90 0x7f193006d150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1944153070 tx=0x7f1938008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.836+0000 7f1956b17b80 1 Processor -- start 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.836+0000 7f1956b17b80 1 -- start start 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09bdc0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc6090 0x55c09be02f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc53d0 0x55c09bde2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1956b17b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09bd1c940 con 0x55c09bdc4ff0 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1956b17b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09bd1a300 con 0x55c09bdc53d0 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1956b17b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c09bd15bf0 con 0x55c09bdc6090 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1943fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09bdc0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1943fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09bdc0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51926/0 (socket says 172.21.15.31:51926) 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1943fff700 1 -- 172.21.15.31:0/1661702241 learned_addr learned my addr 172.21.15.31:0/1661702241 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f19437fe700 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc6090 0x55c09be02f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1948f1e700 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc53d0 0x55c09bde2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1943fff700 1 -- 172.21.15.31:0/1661702241 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc6090 msgr2=0x55c09be02f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1943fff700 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc6090 0x55c09be02f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1943fff700 1 -- 172.21.15.31:0/1661702241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc53d0 msgr2=0x55c09bde2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1943fff700 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc53d0 0x55c09bde2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1943fff700 1 -- 172.21.15.31:0/1661702241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c09bb43b10 con 0x55c09bdc4ff0 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1948f1e700 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc53d0 0x55c09bde2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1943fff700 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09bdc0690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f19340618c0 tx=0x7f1934006df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e225de82c5a86 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1929ffb700 1 -- 172.21.15.31:0/1661702241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934059a00 con 0x55c09bdc4ff0 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1929ffb700 1 -- 172.21.15.31:0/1661702241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193405d830 con 0x55c09bdc4ff0 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.837+0000 7f1929ffb700 1 -- 172.21.15.31:0/1661702241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934061df0 con 0x55c09bdc4ff0 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.838+0000 7f1956b17b80 1 -- 172.21.15.31:0/1661702241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 msgr2=0x55c09bdc0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.838+0000 7f1956b17b80 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09bdc0690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f19340618c0 tx=0x7f1934006df0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.838+0000 7f1956b17b80 1 -- 172.21.15.31:0/1661702241 shutdown_connections 2024-03-30T01:41:05.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.838+0000 7f1956b17b80 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09bdc0690 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.838+0000 7f1956b17b80 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc53d0 0x55c09bde2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.838+0000 7f1956b17b80 1 --2- 172.21.15.31:0/1661702241 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc6090 0x55c09be02f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.838+0000 7f1956b17b80 1 -- 172.21.15.31:0/1661702241 >> 172.21.15.31:0/1661702241 conn(0x55c09bd23650 msgr2=0x55c09bdc2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.838+0000 7f1956b17b80 1 -- 172.21.15.31:0/1661702241 shutdown_connections 2024-03-30T01:41:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.838+0000 7f1956b17b80 1 -- 172.21.15.31:0/1661702241 wait complete. 2024-03-30T01:41:05.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.839+0000 7f1956b17b80 1 -- start start 2024-03-30T01:41:05.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.839+0000 7f1956b17b80 1 Processor -- start 2024-03-30T01:41:05.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09be03cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 0x55c09be10b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09be11040 0x55c09bde2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1956b17b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09bcb1d30 con 0x55c09bdc4ff0 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1956b17b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09bcb19d0 con 0x55c09be11040 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1956b17b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c09bb99650 con 0x55c09bdc53d0 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1956b17b80 10 client.0 _ll_register_callbacks cb 0x55c09bcbb420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1943fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09be03cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1943fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09be03cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51938/0 (socket says 172.21.15.31:51938) 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1943fff700 1 -- 172.21.15.31:0/24713811 learned_addr learned my addr 172.21.15.31:0/24713811 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1948f1e700 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09be11040 0x55c09bde2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f19437fe700 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 0x55c09be10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1943fff700 1 -- 172.21.15.31:0/24713811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 msgr2=0x55c09be10b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1943fff700 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 0x55c09be10b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1943fff700 1 -- 172.21.15.31:0/24713811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09be11040 msgr2=0x55c09bde2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1943fff700 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09be11040 0x55c09bde2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1943fff700 1 -- 172.21.15.31:0/24713811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1944148140 con 0x55c09bdc4ff0 2024-03-30T01:41:05.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1943fff700 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09be03cf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1934088f70 tx=0x7f193401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e03c28319b8c4e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.840+0000 7f1922ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-03-30T01:41:05.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1922ffd700 1 -- 172.21.15.31:0/24713811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934059e60 con 0x55c09bdc4ff0 2024-03-30T01:41:05.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1956b17b80 1 -- 172.21.15.31:0/24713811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c09bcd6330 con 0x55c09bdc4ff0 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1922ffd700 1 -- 172.21.15.31:0/24713811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1934061df0 con 0x55c09bdc4ff0 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1922ffd700 1 -- 172.21.15.31:0/24713811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1934058ef0 con 0x55c09bdc4ff0 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1956b17b80 10 client.4410 fetch_fsmap learned FSMap version 7 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1956b17b80 1 -- 172.21.15.31:0/24713811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55c09bcdbe50 con 0x55c09bdc4ff0 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1922ffd700 1 -- 172.21.15.31:0/24713811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c09bcd6330 con 0x55c09bdc4ff0 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1922ffd700 1 -- 172.21.15.31:0/24713811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f1934061960 con 0x55c09bdc4ff0 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1956b17b80 10 client.4410 fetch_fsmap finished waiting for FSMap version 7 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1956b17b80 10 client.4410 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1956b17b80 20 client.4410 populate_metadata read hostname 'smithi031' 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1956b17b80 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x55c09be13c20 0x55c09be14000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1956b17b80 4 client.4410 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f1956b17b80 1 -- 172.21.15.31:0/24713811 --> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c09bd0e6b0 con 0x55c09be13c20 2024-03-30T01:41:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.841+0000 7f19437fe700 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x55c09be13c20 0x55c09be14000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1922ffd700 10 client.4410 ms_handle_connect on v2:172.21.15.31:6835/3724657244 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f19437fe700 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x55c09be13c20 0x55c09be14000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d95f37b08f21b9e server_cookie=988145b8bc0e3a74 in_seq=0 out_seq=0 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1922ffd700 1 -- 172.21.15.31:0/24713811 <== mds.0 v2:172.21.15.31:6835/3724657244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c09bd0e6b0 con 0x55c09be13c20 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1922ffd700 10 client.4410 handle_command_reply: tid=0 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1956b17b80 1 client.4410 shutdown 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1956b17b80 1 -- 172.21.15.31:0/24713811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 msgr2=0x55c09be03cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1956b17b80 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09be03cf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1934088f70 tx=0x7f193401e000 comp rx=0 tx=0).stop 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1956b17b80 1 -- 172.21.15.31:0/24713811 shutdown_connections 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1956b17b80 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x55c09be13c20 0x55c09be14000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1956b17b80 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc4ff0 0x55c09be03cf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1956b17b80 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09be11040 0x55c09bde2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1956b17b80 1 --2- 172.21.15.31:0/24713811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 0x55c09be10b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1922ffd700 0 client.4410 ms_handle_reset on v2:172.21.15.31:6835/3724657244 2024-03-30T01:41:05.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.842+0000 7f1956b17b80 1 -- 172.21.15.31:0/24713811 >> 172.21.15.31:0/24713811 conn(0x55c09bd23650 msgr2=0x55c09bde6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:05.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.843+0000 7f1956b17b80 1 -- 172.21.15.31:0/24713811 shutdown_connections 2024-03-30T01:41:05.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.843+0000 7f1956b17b80 1 -- 172.21.15.31:0/24713811 wait complete. 2024-03-30T01:41:05.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.843+0000 7f1956b17b80 20 client.4410 trim_cache size 0 max 16384 2024-03-30T01:41:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.861+0000 7f1956b17b80 1 Processor -- start 2024-03-30T01:41:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.861+0000 7f1956b17b80 1 -- start start 2024-03-30T01:41:05.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 0x55c09bdf7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc53d0 0x55c09be1a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09be11040 0x55c09be1a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f1956b17b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09be05520 con 0x55c09bdc53d0 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f1956b17b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09bdc1b80 con 0x55c09bdc4ff0 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f1956b17b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c09bdbfeb0 con 0x55c09be11040 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f19437fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc53d0 0x55c09be1a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f19437fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc53d0 0x55c09be1a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51940/0 (socket says 172.21.15.31:51940) 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f19437fe700 1 -- 172.21.15.31:0/608266553 learned_addr learned my addr 172.21.15.31:0/608266553 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f1943fff700 1 --2- 172.21.15.31:0/608266553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 0x55c09bdf7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f1948f1e700 1 --2- 172.21.15.31:0/608266553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09be11040 0x55c09be1a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f19437fe700 1 -- 172.21.15.31:0/608266553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09be11040 msgr2=0x55c09be1a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f19437fe700 1 --2- 172.21.15.31:0/608266553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09be11040 0x55c09be1a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f19437fe700 1 -- 172.21.15.31:0/608266553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 msgr2=0x55c09bdf7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f19437fe700 1 --2- 172.21.15.31:0/608266553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 0x55c09bdf7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.862+0000 7f19437fe700 1 -- 172.21.15.31:0/608266553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c09bb9bc80 con 0x55c09bdc53d0 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f19437fe700 1 --2- 172.21.15.31:0/608266553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc53d0 0x55c09be1a030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1938009730 tx=0x7f1938017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ef9ff7d42394b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f192a7fc700 1 -- 172.21.15.31:0/608266553 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938028040 con 0x55c09bdc53d0 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f192a7fc700 1 -- 172.21.15.31:0/608266553 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193802be60 con 0x55c09bdc53d0 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f192a7fc700 1 -- 172.21.15.31:0/608266553 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193802b530 con 0x55c09bdc53d0 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f1956b17b80 1 -- 172.21.15.31:0/608266553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc53d0 msgr2=0x55c09be1a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f1956b17b80 1 --2- 172.21.15.31:0/608266553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc53d0 0x55c09be1a030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1938009730 tx=0x7f1938017000 comp rx=0 tx=0).stop 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f1956b17b80 1 -- 172.21.15.31:0/608266553 shutdown_connections 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f1956b17b80 1 --2- 172.21.15.31:0/608266553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09be11040 0x55c09be1a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f1956b17b80 1 --2- 172.21.15.31:0/608266553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 0x55c09bdf7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f1956b17b80 1 --2- 172.21.15.31:0/608266553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09bdc53d0 0x55c09be1a030 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f1956b17b80 1 -- 172.21.15.31:0/608266553 >> 172.21.15.31:0/608266553 conn(0x55c09bd23650 msgr2=0x55c09bdbb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f1956b17b80 1 -- 172.21.15.31:0/608266553 shutdown_connections 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.863+0000 7f1956b17b80 1 -- 172.21.15.31:0/608266553 wait complete. 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.864+0000 7f1956b17b80 1 -- start start 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.864+0000 7f1956b17b80 1 Processor -- start 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 0x55c09be1a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 0x55c09be6be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1956b17b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09be11040 0x55c09be65bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1956b17b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09be1f3b0 con 0x55c09be11040 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1956b17b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09be6cb60 con 0x55c09bdc4ff0 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1956b17b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c09be12e00 con 0x55c09bdc53d0 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1956b17b80 10 client.0 _ll_register_callbacks cb 0x55c09bcbb420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1948f1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09be11040 0x55c09be65bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1948f1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09be11040 0x55c09be65bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51946/0 (socket says 172.21.15.31:51946) 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1948f1e700 1 -- 172.21.15.31:0/2633663905 learned_addr learned my addr 172.21.15.31:0/2633663905 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f19437fe700 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 0x55c09be6be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1943fff700 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 0x55c09be1a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1948f1e700 1 -- 172.21.15.31:0/2633663905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 msgr2=0x55c09be6be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1948f1e700 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 0x55c09be6be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1948f1e700 1 -- 172.21.15.31:0/2633663905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 msgr2=0x55c09be1a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1948f1e700 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 0x55c09be1a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.865+0000 7f1948f1e700 1 -- 172.21.15.31:0/2633663905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193800e040 con 0x55c09be11040 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1948f1e700 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09be11040 0x55c09be65bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f193c00fea0 tx=0x7f193c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf81e8e9f5c076fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1922ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1922ffd700 1 -- 172.21.15.31:0/2633663905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00eed0 con 0x55c09be11040 2024-03-30T01:41:05.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1922ffd700 1 -- 172.21.15.31:0/2633663905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f193c01a3f0 con 0x55c09be11040 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1956b17b80 1 -- 172.21.15.31:0/2633663905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f193405d830 con 0x55c09be11040 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1922ffd700 1 -- 172.21.15.31:0/2633663905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f193c01a5b0 con 0x55c09be11040 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1956b17b80 10 client.4416 fetch_fsmap learned FSMap version 7 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1956b17b80 1 -- 172.21.15.31:0/2633663905 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55c09bb41fe0 con 0x55c09be11040 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1922ffd700 1 -- 172.21.15.31:0/2633663905 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193405d830 con 0x55c09be11040 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1922ffd700 1 -- 172.21.15.31:0/2633663905 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f193c01a710 con 0x55c09be11040 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1956b17b80 10 client.4416 fetch_fsmap finished waiting for FSMap version 7 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1956b17b80 10 client.4416 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1956b17b80 20 client.4416 populate_metadata read hostname 'smithi031' 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1956b17b80 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x55c09be66110 0x55c09bdfeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1956b17b80 4 client.4416 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1956b17b80 1 -- 172.21.15.31:0/2633663905 --> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c09bb64ca0 con 0x55c09be66110 2024-03-30T01:41:05.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.866+0000 7f1943fff700 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x55c09be66110 0x55c09bdfeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1943fff700 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x55c09be66110 0x55c09bdfeca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c6a4372773f3dc server_cookie=1d6a0dd7c5bde843 in_seq=0 out_seq=0 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1922ffd700 10 client.4416 ms_handle_connect on v2:172.21.15.31:6835/3724657244 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1922ffd700 1 -- 172.21.15.31:0/2633663905 <== mds.0 v2:172.21.15.31:6835/3724657244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x55c09bb64ca0 con 0x55c09be66110 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1922ffd700 10 client.4416 handle_command_reply: tid=0 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 client.4416 shutdown 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 -- 172.21.15.31:0/2633663905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09be11040 msgr2=0x55c09be65bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09be11040 0x55c09be65bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f193c00fea0 tx=0x7f193c00e6c0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 -- 172.21.15.31:0/2633663905 shutdown_connections 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x55c09be66110 0x55c09bdfeca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c09be11040 0x55c09be65bd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c09bdc4ff0 0x55c09be1a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 --2- 172.21.15.31:0/2633663905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c09bdc53d0 0x55c09be6be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1922ffd700 0 client.4416 ms_handle_reset on v2:172.21.15.31:6835/3724657244 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 -- 172.21.15.31:0/2633663905 >> 172.21.15.31:0/2633663905 conn(0x55c09bd23650 msgr2=0x55c09bde67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 -- 172.21.15.31:0/2633663905 shutdown_connections 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 1 -- 172.21.15.31:0/2633663905 wait complete. 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.867+0000 7f1956b17b80 20 client.4416 trim_cache size 0 max 16384 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9739890c-57ec-4503-9fd1-d4b0145023f4", 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4269, 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-03-30T01:41:05.881 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-03-30T01:41:05.882 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-03-30T01:41:05.882 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 7.465120958, 2024-03-30T01:41:05.882 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 7, 2024-03-30T01:41:05.882 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 24, 2024-03-30T01:41:05.882 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 23, 2024-03-30T01:41:05.882 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.209431121 2024-03-30T01:41:05.882 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 -- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f193006ac90 msgr2=0x7f193006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f193006ac90 0x7f193006d150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1944153070 tx=0x7f1938008040 comp rx=0 tx=0).stop 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 -- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194406a700 msgr2=0x7f194406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194406a700 0x7f194406db30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1934000c00 tx=0x7f1934004750 comp rx=0 tx=0).stop 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 -- 172.21.15.31:0/3279328576 shutdown_connections 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194406a700 0x7f194406db30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f193006ac90 0x7f193006d150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19440c4710 0x7f19441ea450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 --2- 172.21.15.31:0/3279328576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c3180 0x7f19441e9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:05.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 -- 172.21.15.31:0/3279328576 >> 172.21.15.31:0/3279328576 conn(0x7f19440c3a60 msgr2=0x7f19440c0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:05.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 -- 172.21.15.31:0/3279328576 shutdown_connections 2024-03-30T01:41:05.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:05.868+0000 7f194a981700 1 -- 172.21.15.31:0/3279328576 wait complete. 2024-03-30T01:41:05.888 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T01:41:05.888 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T01:41:05.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:41:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.019+0000 7f1172e9c700 1 Processor -- start 2024-03-30T01:41:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.019+0000 7f1172e9c700 1 -- start start 2024-03-30T01:41:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1172e9c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f116c151c00 0x7f116c0c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1172e9c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 0x7f116c06a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1172e9c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c155480 con 0x7f116c0acd20 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1172e9c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c155740 con 0x7f116c151c00 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1172e9c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c157e80 con 0x7f116c158960 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f116bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f116c0acd20 0x7f116c0bbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48604/0 (socket says 172.21.15.31:48604) 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f116bfff700 1 -- 172.21.15.31:0/3666819546 learned_addr learned my addr 172.21.15.31:0/3666819546 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1171439700 1 --2- 172.21.15.31:0/3666819546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 0x7f116c06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1170c38700 1 --2- 172.21.15.31:0/3666819546 >> v2:172.21.15.59:3300/0 conn(0x7f116c151c00 0x7f116c0c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f116b7fe700 1 -- 172.21.15.31:0/3666819546 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150805812 0 0) 0x7f116c155480 con 0x7f116c0acd20 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f116b7fe700 1 -- 172.21.15.31:0/3666819546 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f115c0035b0 con 0x7f116c0acd20 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f116b7fe700 1 -- 172.21.15.31:0/3666819546 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1819309873 0 0) 0x7f115c0035b0 con 0x7f116c0acd20 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1170c38700 1 -- 172.21.15.31:0/3666819546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 msgr2=0x7f116c06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1170c38700 1 --2- 172.21.15.31:0/3666819546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 0x7f116c06a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1170c38700 1 -- 172.21.15.31:0/3666819546 >> v1:172.21.15.31:6789/0 conn(0x7f116c0acd20 legacy=0x7f116c0bbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1170c38700 1 -- 172.21.15.31:0/3666819546 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c158160 con 0x7f116c151c00 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.020+0000 7f1171439700 1 --2- 172.21.15.31:0/3666819546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 0x7f116c06a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.021+0000 7f1170c38700 1 --2- 172.21.15.31:0/3666819546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 0x7f116c0c2f60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1160009840 tx=0x7f1160009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a0755edc77035 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.021+0000 7f116b7fe700 1 -- 172.21.15.31:0/3666819546 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158001670 con 0x7f116c151c00 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.021+0000 7f116b7fe700 1 -- 172.21.15.31:0/3666819546 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1160004030 con 0x7f116c151c00 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.021+0000 7f116b7fe700 1 -- 172.21.15.31:0/3666819546 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11600043c0 con 0x7f116c151c00 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.021+0000 7f1172e9c700 1 -- 172.21.15.31:0/3666819546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 msgr2=0x7f116c0c2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.021+0000 7f1172e9c700 1 --2- 172.21.15.31:0/3666819546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 0x7f116c0c2f60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1160009840 tx=0x7f1160009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.021+0000 7f1172e9c700 1 -- 172.21.15.31:0/3666819546 shutdown_connections 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.021+0000 7f1172e9c700 1 --2- 172.21.15.31:0/3666819546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 0x7f116c06a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.022+0000 7f1172e9c700 1 --2- 172.21.15.31:0/3666819546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 0x7f116c0c2f60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.022+0000 7f1172e9c700 1 -- 172.21.15.31:0/3666819546 >> 172.21.15.31:0/3666819546 conn(0x7f116c0c3980 msgr2=0x7f116c0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.022+0000 7f1172e9c700 1 -- 172.21.15.31:0/3666819546 shutdown_connections 2024-03-30T01:41:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.022+0000 7f1172e9c700 1 -- 172.21.15.31:0/3666819546 wait complete. 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.023+0000 7f1172e9c700 1 Processor -- start 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.023+0000 7f1172e9c700 1 -- start start 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1172e9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f116c0acd20 0x7f116c0c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1172e9c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 0x7f116c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1172e9c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 0x7f116c0bf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1172e9c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c1e9b90 con 0x7f116c0acd20 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1172e9c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c1e9da0 con 0x7f116c151c00 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1172e9c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c0c12c0 con 0x7f116c158960 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1170c38700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f116c0acd20 0x7f116c0c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1170c38700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f116c0acd20 0x7f116c0c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51952/0 (socket says 172.21.15.31:51952) 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1170c38700 1 -- 172.21.15.31:0/4193484806 learned_addr learned my addr 172.21.15.31:0/4193484806 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1171439700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 0x7f116c0bf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f116bfff700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 0x7f116c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1170c38700 1 -- 172.21.15.31:0/4193484806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 msgr2=0x7f116c0bf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1170c38700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 0x7f116c0bf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1170c38700 1 -- 172.21.15.31:0/4193484806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 msgr2=0x7f116c06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1170c38700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 0x7f116c06ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1170c38700 1 -- 172.21.15.31:0/4193484806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1164009a70 con 0x7f116c0acd20 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f116bfff700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 0x7f116c06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f1170c38700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f116c0acd20 0x7f116c0c1da0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f11600090b0 tx=0x7f1160004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.024+0000 7f11697fa700 1 -- 172.21.15.31:0/4193484806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160003e20 con 0x7f116c0acd20 2024-03-30T01:41:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.025+0000 7f11697fa700 1 -- 172.21.15.31:0/4193484806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116001f070 con 0x7f116c0acd20 2024-03-30T01:41:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.025+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11600094a0 con 0x7f116c0acd20 2024-03-30T01:41:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.025+0000 7f11697fa700 1 -- 172.21.15.31:0/4193484806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160011870 con 0x7f116c0acd20 2024-03-30T01:41:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.025+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c0c1880 con 0x7f116c0acd20 2024-03-30T01:41:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.026+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116c0720d0 con 0x7f116c0acd20 2024-03-30T01:41:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.027+0000 7f11697fa700 1 -- 172.21.15.31:0/4193484806 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f11600119d0 con 0x7f116c0acd20 2024-03-30T01:41:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.027+0000 7f11697fa700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f115006ac90 0x7f115006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.027+0000 7f11697fa700 1 -- 172.21.15.31:0/4193484806 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1160021070 con 0x7f116c0acd20 2024-03-30T01:41:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.027+0000 7f116bfff700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f115006ac90 0x7f115006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.027+0000 7f116bfff700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f115006ac90 0x7f115006d150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f116c0c6d00 tx=0x7f1158001d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.031+0000 7f11697fa700 1 -- 172.21.15.31:0/4193484806 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1160056040 con 0x7f116c0acd20 2024-03-30T01:41:06.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.293+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f116c073570 con 0x7f116c0acd20 2024-03-30T01:41:06.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.294+0000 7f11697fa700 1 -- 172.21.15.31:0/4193484806 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1160059660 con 0x7f116c0acd20 2024-03-30T01:41:06.300 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f115006ac90 msgr2=0x7f115006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f115006ac90 0x7f115006d150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f116c0c6d00 tx=0x7f1158001d80 comp rx=0 tx=0).stop 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f116c0acd20 msgr2=0x7f116c0c1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f116c0acd20 0x7f116c0c1da0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f11600090b0 tx=0x7f1160004750 comp rx=0 tx=0).stop 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 shutdown_connections 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f116c0acd20 0x7f116c0c1da0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f115006ac90 0x7f115006d150 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f116c158960 0x7f116c0bf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 --2- 172.21.15.31:0/4193484806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f116c151c00 0x7f116c06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:06.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.296+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 >> 172.21.15.31:0/4193484806 conn(0x7f116c0c3980 msgr2=0x7f116c0c4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:06.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.297+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 shutdown_connections 2024-03-30T01:41:06.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:06.297+0000 7f1172e9c700 1 -- 172.21.15.31:0/4193484806 wait complete. 2024-03-30T01:41:06.316 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T01:41:06.316 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:41:06.317 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-03-30T01:41:06.317 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-30T01:41:06.317 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T01:41:06.317 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T01:41:06.317 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T01:41:06.317 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T01:41:06.317 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T01:41:06.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:06.317 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:41:06.337 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42258sec preferred_lft 42258sec 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:06.338 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-03-30T01:41:06.339 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T01:41:06.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:06.339 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:06.339 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:06.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-03-30T01:41:06.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-03-30T01:41:06.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-03-30T01:41:06.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T01:41:06.340 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:06.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:06.517 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:41:06.550 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-03-30T01:41:06.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:06.552 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-03-30T01:41:06.630 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-03-30T01:41:06.630 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:41:06.630 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:41:06.630 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:41:06.631 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:41:06.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:06.631 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:06.631 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:06.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:41:06.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:41:06.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:41:06.631 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:06.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:06.923 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-03-30T01:41:06.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:06.944 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-03-30T01:41:07.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:07.007 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:07.007 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:07.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T01:41:07.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-30T01:41:07.007 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:07.120 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-30T01:41:07.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:07.121 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:07.121 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:07.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-30T01:41:07.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T01:41:07.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-30T01:41:07.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-30T01:41:07.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-30T01:41:07.121 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:07.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:07.349 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:07.349 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:07.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-03-30T01:41:07.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T01:41:07.349 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:07.462 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-30T01:41:07.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:41:07.462 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-30T01:41:07.481 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T01:41:07.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:41:07.482 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-30T01:41:07.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-03-30T01:41:07.618 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T01:41:07.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T01:41:07.693 INFO:teuthology.orchestra.run.smithi031.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T01:41:07.694 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T01:41:07.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:07.695 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-03-30T01:41:07.757 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T01:41:07.757 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-30T01:41:07.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-03-30T01:41:07.834 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T01:41:07.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T01:41:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T01:41:07.907 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T01:41:07.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:07.908 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-03-30T01:41:07.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:2024-03-30T01:41:07.903+0000 7fcc450f95c0 -1 init, newargv = 0x556ff1ada1a0 newargc=15 2024-03-30T01:41:07.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:ceph-fuse[54666]: starting ceph client 2024-03-30T01:41:07.916 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:ceph-fuse[54666]: starting fuse 2024-03-30T01:41:07.928 INFO:teuthology.orchestra.run.smithi031.stdout:47 2024-03-30T01:41:07.929 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2024-03-30T01:41:07.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T01:41:07.929 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-03-30T01:41:07.929 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-03-30T01:41:07.929 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-03-30T01:41:07.929 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-03-30T01:41:07.929 DEBUG:teuthology.orchestra.run.smithi031:> 2024-03-30T01:41:07.929 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-03-30T01:41:07.929 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-03-30T01:41:07.930 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T01:41:07.931 DEBUG:teuthology.orchestra.run.smithi031:> 2024-03-30T01:41:07.931 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.admin")) 2024-03-30T01:41:07.931 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-03-30T01:41:08.066 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.admin.54666.asok 2024-03-30T01:41:08.248 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.54666.asok 2024-03-30T01:41:08.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:08.248 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.54666.asok status 2024-03-30T01:41:08.427 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-03-30T01:41:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-03-30T01:41:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T01:41:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T01:41:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "admin", 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "54666", 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4428, 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 4428 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:41:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2208092416 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2208092416 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.4428 192.168.0.1:0/2208092416", 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/2208092416", 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-03-30T01:41:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 7, 2024-03-30T01:41:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 24, 2024-03-30T01:41:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-03-30T01:41:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-03-30T01:41:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-03-30T01:41:08.430 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-03-30T01:41:08.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:08.439 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-03-30T01:41:08.439 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-30T01:41:08.458 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-03-30T01:41:08.458 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-30T01:41:08.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:08.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-30T01:41:08.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:08.533 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-30T01:41:08.647 INFO:teuthology.orchestra.run.smithi031.stdout:# file: . 2024-03-30T01:41:08.648 INFO:teuthology.orchestra.run.smithi031.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-30T01:41:08.648 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:08.648 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:41:08.648 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:08.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.848+0000 7f65e6561700 1 Processor -- start 2024-03-30T01:41:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.848+0000 7f65e6561700 1 -- start start 2024-03-30T01:41:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e6561700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e014e1a0 0x7f65e00c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e6561700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65e006d550 0x7f65e0154220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e6561700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65e00c15d0 0x7f65e0154760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e6561700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e00c2020 con 0x7f65e00c15d0 2024-03-30T01:41:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e6561700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e0071950 con 0x7f65e006d550 2024-03-30T01:41:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e6561700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0071670 con 0x7f65e014e1a0 2024-03-30T01:41:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e4afe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65e00c15d0 0x7f65e0154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e4afe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65e00c15d0 0x7f65e0154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51966/0 (socket says 172.21.15.31:51966) 2024-03-30T01:41:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e4afe700 1 -- 172.21.15.31:0/784611559 learned_addr learned my addr 172.21.15.31:0/784611559 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65dffff700 1 --2- 172.21.15.31:0/784611559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e014e1a0 0x7f65e00c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65df7fe700 1 --2- 172.21.15.31:0/784611559 >> v2:172.21.15.59:3300/0 conn(0x7f65e006d550 0x7f65e0154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e4afe700 1 -- 172.21.15.31:0/784611559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e014e1a0 msgr2=0x7f65e00c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e4afe700 1 --2- 172.21.15.31:0/784611559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e014e1a0 0x7f65e00c1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e4afe700 1 -- 172.21.15.31:0/784611559 >> v2:172.21.15.59:3300/0 conn(0x7f65e006d550 msgr2=0x7f65e0154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.849+0000 7f65e4afe700 1 --2- 172.21.15.31:0/784611559 >> v2:172.21.15.59:3300/0 conn(0x7f65e006d550 0x7f65e0154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.850+0000 7f65e4afe700 1 -- 172.21.15.31:0/784611559 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e006dcb0 con 0x7f65e00c15d0 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.850+0000 7f65df7fe700 1 --2- 172.21.15.31:0/784611559 >> v2:172.21.15.59:3300/0 conn(0x7f65e006d550 0x7f65e0154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.850+0000 7f65e4afe700 1 --2- 172.21.15.31:0/784611559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e00c15d0 0x7f65e0154760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f65e006e6f0 tx=0x7f65d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35ee40ee8245e84 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.850+0000 7f65deffd700 1 -- 172.21.15.31:0/784611559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d800e070 con 0x7f65e00c15d0 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.850+0000 7f65deffd700 1 -- 172.21.15.31:0/784611559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d800b9c0 con 0x7f65e00c15d0 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.850+0000 7f65deffd700 1 -- 172.21.15.31:0/784611559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d800bd70 con 0x7f65e00c15d0 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.850+0000 7f65e6561700 1 -- 172.21.15.31:0/784611559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e00c15d0 msgr2=0x7f65e0154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.850+0000 7f65e6561700 1 --2- 172.21.15.31:0/784611559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e00c15d0 0x7f65e0154760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f65e006e6f0 tx=0x7f65d800b330 comp rx=0 tx=0).stop 2024-03-30T01:41:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.850+0000 7f65e6561700 1 -- 172.21.15.31:0/784611559 shutdown_connections 2024-03-30T01:41:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.851+0000 7f65e6561700 1 --2- 172.21.15.31:0/784611559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e00c15d0 0x7f65e0154760 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.851+0000 7f65e6561700 1 --2- 172.21.15.31:0/784611559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e014e1a0 0x7f65e00c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.851+0000 7f65e6561700 1 --2- 172.21.15.31:0/784611559 >> v2:172.21.15.59:3300/0 conn(0x7f65e006d550 0x7f65e0154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.851+0000 7f65e6561700 1 -- 172.21.15.31:0/784611559 >> 172.21.15.31:0/784611559 conn(0x7f65e006af80 msgr2=0x7f65e00c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.851+0000 7f65e6561700 1 -- 172.21.15.31:0/784611559 shutdown_connections 2024-03-30T01:41:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.851+0000 7f65e6561700 1 -- 172.21.15.31:0/784611559 wait complete. 2024-03-30T01:41:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.852+0000 7f65e6561700 1 Processor -- start 2024-03-30T01:41:08.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.852+0000 7f65e6561700 1 -- start start 2024-03-30T01:41:08.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e6561700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e006d550 0x7f65e0142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:08.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e6561700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e00c15d0 0x7f65e014b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:08.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e6561700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e014e1a0 0x7f65e00bf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:08.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e6561700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e014d6c0 con 0x7f65e014e1a0 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e6561700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e01ec090 con 0x7f65e00c15d0 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e6561700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e01ec370 con 0x7f65e006d550 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e4afe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e014e1a0 0x7f65e00bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65dffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e006d550 0x7f65e0142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65df7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e00c15d0 0x7f65e014b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e4afe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e014e1a0 0x7f65e00bf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51974/0 (socket says 172.21.15.31:51974) 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65dffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e006d550 0x7f65e0142320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:46646/0 (socket says 172.21.15.31:46646) 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e4afe700 1 -- 172.21.15.31:0/697785475 learned_addr learned my addr 172.21.15.31:0/697785475 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e4afe700 1 -- 172.21.15.31:0/697785475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e006d550 msgr2=0x7f65e0142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e4afe700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e006d550 0x7f65e0142320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e4afe700 1 -- 172.21.15.31:0/697785475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e00c15d0 msgr2=0x7f65e014b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e4afe700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e00c15d0 0x7f65e014b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.853+0000 7f65e4afe700 1 -- 172.21.15.31:0/697785475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d8009b30 con 0x7f65e014e1a0 2024-03-30T01:41:08.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.854+0000 7f65e4afe700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e014e1a0 0x7f65e00bf430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65d800b370 tx=0x7f65d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:08.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.855+0000 7f65dcff9700 1 -- 172.21.15.31:0/697785475 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8004250 con 0x7f65e014e1a0 2024-03-30T01:41:08.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.855+0000 7f65dcff9700 1 -- 172.21.15.31:0/697785475 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d80043d0 con 0x7f65e014e1a0 2024-03-30T01:41:08.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.855+0000 7f65dcff9700 1 -- 172.21.15.31:0/697785475 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8004780 con 0x7f65e014e1a0 2024-03-30T01:41:08.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.855+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e014e700 con 0x7f65e014e1a0 2024-03-30T01:41:08.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.855+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e00bb280 con 0x7f65e014e1a0 2024-03-30T01:41:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.856+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e00c0140 con 0x7f65e014e1a0 2024-03-30T01:41:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.857+0000 7f65dcff9700 1 -- 172.21.15.31:0/697785475 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f65d801b430 con 0x7f65e014e1a0 2024-03-30T01:41:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.857+0000 7f65dcff9700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f65cc06ac90 0x7f65cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.857+0000 7f65dcff9700 1 -- 172.21.15.31:0/697785475 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f65d808a6a0 con 0x7f65e014e1a0 2024-03-30T01:41:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.857+0000 7f65dffff700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f65cc06ac90 0x7f65cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.858+0000 7f65dffff700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f65cc06ac90 0x7f65cc06d150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65e014d560 tx=0x7f65d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:08.862+0000 7f65dcff9700 1 -- 172.21.15.31:0/697785475 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f65e00c0140 con 0x7f65e014e1a0 2024-03-30T01:41:09.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.082+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65e00c0140 con 0x7f65e014e1a0 2024-03-30T01:41:09.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.083+0000 7f65dcff9700 1 -- 172.21.15.31:0/697785475 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f65e00c0140 con 0x7f65e014e1a0 2024-03-30T01:41:09.090 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:09.090 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:09.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f65cc06ac90 msgr2=0x7f65cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f65cc06ac90 0x7f65cc06d150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65e014d560 tx=0x7f65d0008040 comp rx=0 tx=0).stop 2024-03-30T01:41:09.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e014e1a0 msgr2=0x7f65e00bf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e014e1a0 0x7f65e00bf430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65d800b370 tx=0x7f65d8004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 shutdown_connections 2024-03-30T01:41:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e014e1a0 0x7f65e00bf430 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f65cc06ac90 0x7f65cc06d150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65e006d550 0x7f65e0142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 --2- 172.21.15.31:0/697785475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65e00c15d0 0x7f65e014b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.085+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 >> 172.21.15.31:0/697785475 conn(0x7f65e006af80 msgr2=0x7f65e01488b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.086+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 shutdown_connections 2024-03-30T01:41:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.086+0000 7f65e6561700 1 -- 172.21.15.31:0/697785475 wait complete. 2024-03-30T01:41:09.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:09.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:09.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.238+0000 7f67cb56f700 1 Processor -- start 2024-03-30T01:41:09.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.238+0000 7f67cb56f700 1 -- start start 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67cb56f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67cb56f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67c4154930 0x7f67c4154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67cb56f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c40c31f0 con 0x7f67c406d340 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67cb56f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c40c2220 con 0x7f67c4154930 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67cb56f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c414ed80 con 0x7f67c414e9a0 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67c8b0a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67c406d340 0x7f67c406e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48612/0 (socket says 172.21.15.31:48612) 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67c8b0a700 1 -- 172.21.15.31:0/1506814878 learned_addr learned my addr 172.21.15.31:0/1506814878 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67c930b700 1 --2- 172.21.15.31:0/1506814878 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67c9b0c700 1 --2- 172.21.15.31:0/1506814878 >> v2:172.21.15.59:3300/0 conn(0x7f67c4154930 0x7f67c4154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67bbfff700 1 -- 172.21.15.31:0/1506814878 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477271405 0 0) 0x7f67c40c31f0 con 0x7f67c406d340 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67bbfff700 1 -- 172.21.15.31:0/1506814878 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67b00035b0 con 0x7f67c406d340 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67bbfff700 1 -- 172.21.15.31:0/1506814878 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2506931266 0 0) 0x7f67b00035b0 con 0x7f67c406d340 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67c930b700 1 -- 172.21.15.31:0/1506814878 >> v2:172.21.15.59:3300/0 conn(0x7f67c4154930 msgr2=0x7f67c4154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67c930b700 1 --2- 172.21.15.31:0/1506814878 >> v2:172.21.15.59:3300/0 conn(0x7f67c4154930 0x7f67c4154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67c930b700 1 -- 172.21.15.31:0/1506814878 >> v1:172.21.15.31:6789/0 conn(0x7f67c406d340 legacy=0x7f67c406e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67c930b700 1 -- 172.21.15.31:0/1506814878 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c40a8790 con 0x7f67c414e9a0 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.239+0000 7f67c9b0c700 1 --2- 172.21.15.31:0/1506814878 >> v2:172.21.15.59:3300/0 conn(0x7f67c4154930 0x7f67c4154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67c930b700 1 --2- 172.21.15.31:0/1506814878 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c406e050 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009840 tx=0x7f67c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d0a02cc44553d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67bbfff700 1 -- 172.21.15.31:0/1506814878 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4001670 con 0x7f67c414e9a0 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67bbfff700 1 -- 172.21.15.31:0/1506814878 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c0004030 con 0x7f67c414e9a0 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67bbfff700 1 -- 172.21.15.31:0/1506814878 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c00043c0 con 0x7f67c414e9a0 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67cb56f700 1 -- 172.21.15.31:0/1506814878 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 msgr2=0x7f67c406e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67cb56f700 1 --2- 172.21.15.31:0/1506814878 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c406e050 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009840 tx=0x7f67c0009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67cb56f700 1 -- 172.21.15.31:0/1506814878 shutdown_connections 2024-03-30T01:41:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67cb56f700 1 --2- 172.21.15.31:0/1506814878 >> v2:172.21.15.59:3300/0 conn(0x7f67c4154930 0x7f67c4154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67cb56f700 1 --2- 172.21.15.31:0/1506814878 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c406e050 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.240+0000 7f67cb56f700 1 -- 172.21.15.31:0/1506814878 >> 172.21.15.31:0/1506814878 conn(0x7f67c40c3910 msgr2=0x7f67c40c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.241+0000 7f67cb56f700 1 -- 172.21.15.31:0/1506814878 shutdown_connections 2024-03-30T01:41:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.241+0000 7f67cb56f700 1 -- 172.21.15.31:0/1506814878 wait complete. 2024-03-30T01:41:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.242+0000 7f67cb56f700 1 Processor -- start 2024-03-30T01:41:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.242+0000 7f67cb56f700 1 -- start start 2024-03-30T01:41:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67cb56f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c406d340 0x7f67c415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67cb56f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c40ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67cb56f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c4154930 0x7f67c415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67cb56f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c4144860 con 0x7f67c406d340 2024-03-30T01:41:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67cb56f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c40c69b0 con 0x7f67c4154930 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67cb56f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c415d900 con 0x7f67c414e9a0 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c930b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c406d340 0x7f67c415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c930b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c406d340 0x7f67c415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51980/0 (socket says 172.21.15.31:51980) 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c930b700 1 -- 172.21.15.31:0/4012034274 learned_addr learned my addr 172.21.15.31:0/4012034274 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c8b0a700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c40ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c9b0c700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c4154930 0x7f67c415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c930b700 1 -- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 msgr2=0x7f67c40ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c930b700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c40ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c930b700 1 -- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c4154930 msgr2=0x7f67c415a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c930b700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c4154930 0x7f67c415a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c930b700 1 -- 172.21.15.31:0/4012034274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc009a70 con 0x7f67c406d340 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c8b0a700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c40ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c9b0c700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c4154930 0x7f67c415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67c930b700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c406d340 0x7f67c415bcc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67c00090b0 tx=0x7f67c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67b9ffb700 1 -- 172.21.15.31:0/4012034274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0003e20 con 0x7f67c406d340 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67b9ffb700 1 -- 172.21.15.31:0/4012034274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c001f070 con 0x7f67c406d340 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c00094a0 con 0x7f67c406d340 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.243+0000 7f67b9ffb700 1 -- 172.21.15.31:0/4012034274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0011870 con 0x7f67c406d340 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.244+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c40c6640 con 0x7f67c406d340 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.245+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c4066f90 con 0x7f67c406d340 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.245+0000 7f67b9ffb700 1 -- 172.21.15.31:0/4012034274 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f67c00119d0 con 0x7f67c406d340 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.246+0000 7f67b9ffb700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67a806ac90 0x7f67a806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.246+0000 7f67b9ffb700 1 -- 172.21.15.31:0/4012034274 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f67c0021070 con 0x7f67c406d340 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.246+0000 7f67c8b0a700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67a806ac90 0x7f67a806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.246+0000 7f67c8b0a700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67a806ac90 0x7f67a806d150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f67b4001eb0 tx=0x7f67b4001d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.250+0000 7f67b9ffb700 1 -- 172.21.15.31:0/4012034274 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f67c0055ed0 con 0x7f67c406d340 2024-03-30T01:41:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.468+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67c40747d0 con 0x7f67c406d340 2024-03-30T01:41:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.468+0000 7f67b9ffb700 1 -- 172.21.15.31:0/4012034274 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67c002b020 con 0x7f67c406d340 2024-03-30T01:41:09.475 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:09.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.470+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67a806ac90 msgr2=0x7f67a806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.470+0000 7f67cb56f700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67a806ac90 0x7f67a806d150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f67b4001eb0 tx=0x7f67b4001d80 comp rx=0 tx=0).stop 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.470+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c406d340 msgr2=0x7f67c415bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.470+0000 7f67cb56f700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c406d340 0x7f67c415bcc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67c00090b0 tx=0x7f67c0004750 comp rx=0 tx=0).stop 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.471+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 shutdown_connections 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.471+0000 7f67cb56f700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c406d340 0x7f67c415bcc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.471+0000 7f67cb56f700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67a806ac90 0x7f67a806d150 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.471+0000 7f67cb56f700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67c4154930 0x7f67c415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.471+0000 7f67cb56f700 1 --2- 172.21.15.31:0/4012034274 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67c414e9a0 0x7f67c40ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.471+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 >> 172.21.15.31:0/4012034274 conn(0x7f67c40c3910 msgr2=0x7f67c40c1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.471+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 shutdown_connections 2024-03-30T01:41:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.471+0000 7f67cb56f700 1 -- 172.21.15.31:0/4012034274 wait complete. 2024-03-30T01:41:09.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.622+0000 7fa76c543700 1 Processor -- start 2024-03-30T01:41:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.622+0000 7fa76c543700 1 -- start start 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.623+0000 7fa76c543700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c7f10 0x7fa7640c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.623+0000 7fa76c543700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa764155240 con 0x7fa7640c8420 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.623+0000 7fa76c543700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7641556d0 con 0x7fa764151d80 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.623+0000 7fa76c543700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76406cb20 con 0x7fa7640c7f10 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa76aae0700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa7640c8420 0x7fa7640c6f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48622/0 (socket says 172.21.15.31:48622) 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa76aae0700 1 -- 172.21.15.31:0/1951283134 learned_addr learned my addr 172.21.15.31:0/1951283134 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa769ade700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c7f10 0x7fa7640c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492401880 0 0) 0x7fa764155240 con 0x7fa7640c8420 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7500035b0 con 0x7fa7640c8420 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1593274482 0 0) 0x7fa7500035b0 con 0x7fa7640c8420 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c7f10 msgr2=0x7fa7640c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c7f10 0x7fa7640c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 >> v1:172.21.15.59:6789/0 conn(0x7fa764151d80 legacy=0x7fa7640ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764154910 con 0x7fa7640c8420 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7fa75c0019f0 con 0x7fa7640c8420 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 >> v1:172.21.15.31:6789/0 conn(0x7fa7640c8420 legacy=0x7fa7640c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa750003b10 0x7fa750003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c7f10 0x7fa750004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa750004d90 0x7fa750005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76406cb20 con 0x7fa750004d90 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75000bbd0 con 0x7fa750003b10 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75000bd90 con 0x7fa7640c7f10 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa76aae0700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa750004d90 0x7fa750005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.624+0000 7fa76a2df700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa750003b10 0x7fa750003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa769ade700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c7f10 0x7fa750004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa76aae0700 1 -- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c7f10 msgr2=0x7fa750004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa76aae0700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c7f10 0x7fa750004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa76aae0700 1 -- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa750003b10 msgr2=0x7fa750003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa76aae0700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa750003b10 0x7fa750003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa76aae0700 1 -- 172.21.15.31:0/1951283134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7500035b0 con 0x7fa750004d90 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa76aae0700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa750004d90 0x7fa750005210 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00a800 tx=0x7fa750005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7affa0e75d3cbfd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00a2f0 con 0x7fa750004d90 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c012600 con 0x7fa750004d90 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.625+0000 7fa7692dd700 1 -- 172.21.15.31:0/1951283134 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0129b0 con 0x7fa750004d90 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.626+0000 7fa76c543700 1 -- 172.21.15.31:0/1951283134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa750004d90 msgr2=0x7fa750005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.626+0000 7fa76c543700 1 --2- 172.21.15.31:0/1951283134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa750004d90 0x7fa750005210 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00a800 tx=0x7fa750005880 comp rx=0 tx=0).stop 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.626+0000 7fa76aae0700 1 -- 172.21.15.31:0/1951283134 reap_dead start 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.626+0000 7fa76c543700 1 -- 172.21.15.31:0/1951283134 shutdown_connections 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.626+0000 7fa76c543700 1 -- 172.21.15.31:0/1951283134 >> 172.21.15.31:0/1951283134 conn(0x7fa764072010 msgr2=0x7fa764072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.626+0000 7fa76c543700 1 -- 172.21.15.31:0/1951283134 shutdown_connections 2024-03-30T01:41:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.626+0000 7fa76c543700 1 -- 172.21.15.31:0/1951283134 wait complete. 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.627+0000 7fa76c543700 1 Processor -- start 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.627+0000 7fa76c543700 1 -- start start 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76c543700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa750004d90 0x7fa764153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76c543700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c41d0 0x7fa764148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76c543700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7640c0830 0x7fa764148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76c543700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640c0120 con 0x7fa7640c0830 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76c543700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640c30e0 con 0x7fa750004d90 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76c543700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641ed9d0 con 0x7fa7640c41d0 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76aae0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7640c0830 0x7fa764148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76aae0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7640c0830 0x7fa764148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52000/0 (socket says 172.21.15.31:52000) 2024-03-30T01:41:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76aae0700 1 -- 172.21.15.31:0/3540327700 learned_addr learned my addr 172.21.15.31:0/3540327700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76a2df700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa750004d90 0x7fa764153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa769ade700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c41d0 0x7fa764148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76aae0700 1 -- 172.21.15.31:0/3540327700 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c41d0 msgr2=0x7fa764148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76aae0700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c41d0 0x7fa764148200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76aae0700 1 -- 172.21.15.31:0/3540327700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa750004d90 msgr2=0x7fa764153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76aae0700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa750004d90 0x7fa764153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.628+0000 7fa76aae0700 1 -- 172.21.15.31:0/3540327700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7641edbc0 con 0x7fa7640c0830 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.629+0000 7fa76aae0700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7640c0830 0x7fa764148740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa75000bba0 tx=0x7fa75c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.629+0000 7fa75affd700 1 -- 172.21.15.31:0/3540327700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0017b0 con 0x7fa7640c0830 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.629+0000 7fa75affd700 1 -- 172.21.15.31:0/3540327700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c001970 con 0x7fa7640c0830 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.629+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7640bfa60 con 0x7fa7640c0830 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.629+0000 7fa75affd700 1 -- 172.21.15.31:0/3540327700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c01e670 con 0x7fa7640c0830 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.629+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa764154340 con 0x7fa7640c0830 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.630+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7640747d0 con 0x7fa7640c0830 2024-03-30T01:41:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.630+0000 7fa75affd700 1 -- 172.21.15.31:0/3540327700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa75c01e7d0 con 0x7fa7640c0830 2024-03-30T01:41:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.631+0000 7fa75affd700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa74806ac90 0x7fa74806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.631+0000 7fa75affd700 1 -- 172.21.15.31:0/3540327700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa75c08b4f0 con 0x7fa7640c0830 2024-03-30T01:41:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.631+0000 7fa76a2df700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa74806ac90 0x7fa74806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.631+0000 7fa76a2df700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa74806ac90 0x7fa74806d150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa760007fd0 tx=0x7fa760009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.636+0000 7fa75affd700 1 -- 172.21.15.31:0/3540327700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa75c056470 con 0x7fa7640c0830 2024-03-30T01:41:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.850+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7640bd720 con 0x7fa7640c0830 2024-03-30T01:41:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.851+0000 7fa75affd700 1 -- 172.21.15.31:0/3540327700 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa75c059a90 con 0x7fa7640c0830 2024-03-30T01:41:09.858 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:09.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa74806ac90 msgr2=0x7fa74806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa74806ac90 0x7fa74806d150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa760007fd0 tx=0x7fa760009040 comp rx=0 tx=0).stop 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7640c0830 msgr2=0x7fa764148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7640c0830 0x7fa764148740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa75000bba0 tx=0x7fa75c001d80 comp rx=0 tx=0).stop 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 shutdown_connections 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7640c0830 0x7fa764148740 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa74806ac90 0x7fa74806d150 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa750004d90 0x7fa764153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 --2- 172.21.15.31:0/3540327700 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7640c41d0 0x7fa764148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 >> 172.21.15.31:0/3540327700 conn(0x7fa764072010 msgr2=0x7fa7640c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 shutdown_connections 2024-03-30T01:41:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:09.854+0000 7fa76c543700 1 -- 172.21.15.31:0/3540327700 wait complete. 2024-03-30T01:41:09.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.009+0000 7f09f4c59700 1 Processor -- start 2024-03-30T01:41:10.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.009+0000 7f09f4c59700 1 -- start start 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.010+0000 7f09f4c59700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09f00a4cc0 0x7f09f00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.010+0000 7f09f4c59700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f09f006d4f0 0x7f09f00c1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.010+0000 7f09f4c59700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f0152520 con 0x7f09f0152f70 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.010+0000 7f09f4c59700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f006cca0 con 0x7f09f00a4cc0 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.010+0000 7f09f4c59700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f006d9a0 con 0x7f09f006d4f0 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.010+0000 7f09ee59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09f00a4cc0 0x7f09f00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.010+0000 7f09ee59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f09f00a4cc0 0x7f09f00a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52004/0 (socket says 172.21.15.31:52004) 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.010+0000 7f09ee59c700 1 -- 172.21.15.31:0/2261685178 learned_addr learned my addr 172.21.15.31:0/2261685178 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.011+0000 7f09edd9b700 1 --2- 172.21.15.31:0/2261685178 >> v2:172.21.15.59:3300/0 conn(0x7f09f006d4f0 0x7f09f00c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.011+0000 7f09ee59c700 1 -- 172.21.15.31:0/2261685178 >> v2:172.21.15.59:3300/0 conn(0x7f09f006d4f0 msgr2=0x7f09f00c1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:10.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.011+0000 7f09ee59c700 1 --2- 172.21.15.31:0/2261685178 >> v2:172.21.15.59:3300/0 conn(0x7f09f006d4f0 0x7f09f00c1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.011+0000 7f09ee59c700 1 -- 172.21.15.31:0/2261685178 >> v1:172.21.15.59:6789/0 conn(0x7f09f0152f70 legacy=0x7f09f0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.011+0000 7f09ee59c700 1 -- 172.21.15.31:0/2261685178 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f006dbf0 con 0x7f09f00a4cc0 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.011+0000 7f09ee59c700 1 --2- 172.21.15.31:0/2261685178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 0x7f09f00a50a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f09f006e420 tx=0x7f09e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe0d8b225e41894 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.011+0000 7f09ed59a700 1 -- 172.21.15.31:0/2261685178 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e0004230 con 0x7f09f00a4cc0 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.011+0000 7f09ed59a700 1 -- 172.21.15.31:0/2261685178 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e0004390 con 0x7f09f00a4cc0 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.011+0000 7f09ed59a700 1 -- 172.21.15.31:0/2261685178 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e00036a0 con 0x7f09f00a4cc0 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.012+0000 7f09f4c59700 1 -- 172.21.15.31:0/2261685178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 msgr2=0x7f09f00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.012+0000 7f09f4c59700 1 --2- 172.21.15.31:0/2261685178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 0x7f09f00a50a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f09f006e420 tx=0x7f09e0009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.012+0000 7f09f4c59700 1 -- 172.21.15.31:0/2261685178 shutdown_connections 2024-03-30T01:41:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.012+0000 7f09f4c59700 1 --2- 172.21.15.31:0/2261685178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 0x7f09f00a50a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.012+0000 7f09f4c59700 1 --2- 172.21.15.31:0/2261685178 >> v2:172.21.15.59:3300/0 conn(0x7f09f006d4f0 0x7f09f00c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.012+0000 7f09f4c59700 1 -- 172.21.15.31:0/2261685178 >> 172.21.15.31:0/2261685178 conn(0x7f09f006b110 msgr2=0x7f09f00a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.012+0000 7f09f4c59700 1 -- 172.21.15.31:0/2261685178 shutdown_connections 2024-03-30T01:41:10.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.012+0000 7f09f4c59700 1 -- 172.21.15.31:0/2261685178 wait complete. 2024-03-30T01:41:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.013+0000 7f09f4c59700 1 Processor -- start 2024-03-30T01:41:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.013+0000 7f09f4c59700 1 -- start start 2024-03-30T01:41:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09f4c59700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09f006d4f0 0x7f09f014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09f4c59700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 0x7f09f00bce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09f4c59700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f0152f70 0x7f09f014bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09f4c59700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f0067970 con 0x7f09f00a4cc0 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09f4c59700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00a6640 con 0x7f09f0152f70 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09f4c59700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00a6360 con 0x7f09f006d4f0 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09edd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 0x7f09f00bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09edd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 0x7f09f00bce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52006/0 (socket says 172.21.15.31:52006) 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09edd9b700 1 -- 172.21.15.31:0/2871580388 learned_addr learned my addr 172.21.15.31:0/2871580388 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09eed9d700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f0152f70 0x7f09f014bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09ee59c700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09f006d4f0 0x7f09f014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09edd9b700 1 -- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09f006d4f0 msgr2=0x7f09f014e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09edd9b700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09f006d4f0 0x7f09f014e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09edd9b700 1 -- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f0152f70 msgr2=0x7f09f014bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09edd9b700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f0152f70 0x7f09f014bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09edd9b700 1 -- 172.21.15.31:0/2871580388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0009550 con 0x7f09f00a4cc0 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09eed9d700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f0152f70 0x7f09f014bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.014+0000 7f09ee59c700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09f006d4f0 0x7f09f014e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.015+0000 7f09edd9b700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 0x7f09f00bce50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09d800d800 tx=0x7f09d800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.015+0000 7f09e6ffd700 1 -- 172.21.15.31:0/2871580388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d80125f0 con 0x7f09f00a4cc0 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.015+0000 7f09e6ffd700 1 -- 172.21.15.31:0/2871580388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d8012750 con 0x7f09f00a4cc0 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.015+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f00a5140 con 0x7f09f00a4cc0 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.015+0000 7f09e6ffd700 1 -- 172.21.15.31:0/2871580388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8012b00 con 0x7f09f00a4cc0 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.015+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f00a5700 con 0x7f09f00a4cc0 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.016+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f00747d0 con 0x7f09f00a4cc0 2024-03-30T01:41:10.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.016+0000 7f09e6ffd700 1 -- 172.21.15.31:0/2871580388 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f09d8012c60 con 0x7f09f00a4cc0 2024-03-30T01:41:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.017+0000 7f09e6ffd700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f09d006ac90 0x7f09d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.017+0000 7f09e6ffd700 1 -- 172.21.15.31:0/2871580388 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f09d80899d0 con 0x7f09f00a4cc0 2024-03-30T01:41:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.017+0000 7f09ee59c700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f09d006ac90 0x7f09d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.017+0000 7f09ee59c700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f09d006ac90 0x7f09d006d150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009c10 tx=0x7f09e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.022+0000 7f09e6ffd700 1 -- 172.21.15.31:0/2871580388 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f09d80548b0 con 0x7f09f00a4cc0 2024-03-30T01:41:10.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.234+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09f00c0130 con 0x7f09f00a4cc0 2024-03-30T01:41:10.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.235+0000 7f09e6ffd700 1 -- 172.21.15.31:0/2871580388 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09d8057ed0 con 0x7f09f00a4cc0 2024-03-30T01:41:10.241 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:10.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f09d006ac90 msgr2=0x7f09d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f09d006ac90 0x7f09d006d150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009c10 tx=0x7f09e0008030 comp rx=0 tx=0).stop 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 msgr2=0x7f09f00bce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 0x7f09f00bce50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09d800d800 tx=0x7f09d800dbc0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 shutdown_connections 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f09d006ac90 0x7f09d006d150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f0152f70 0x7f09f014bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09f006d4f0 0x7f09f014e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 --2- 172.21.15.31:0/2871580388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09f00a4cc0 0x7f09f00bce50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 >> 172.21.15.31:0/2871580388 conn(0x7f09f006b110 msgr2=0x7f09f00c24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 shutdown_connections 2024-03-30T01:41:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.237+0000 7f09f4c59700 1 -- 172.21.15.31:0/2871580388 wait complete. 2024-03-30T01:41:10.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:10.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.389+0000 7f992c84d700 1 Processor -- start 2024-03-30T01:41:10.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.389+0000 7f992c84d700 1 -- start start 2024-03-30T01:41:10.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.390+0000 7f992c84d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9928154d90 0x7f9928150b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.390+0000 7f992c84d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99280c2830 0x7f992814dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.390+0000 7f992c84d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9928151160 con 0x7f992806da30 2024-03-30T01:41:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.390+0000 7f992c84d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9928151440 con 0x7f9928154d90 2024-03-30T01:41:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.390+0000 7f992c84d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9928151720 con 0x7f99280c2830 2024-03-30T01:41:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.390+0000 7f9925d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9928154d90 0x7f9928150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.390+0000 7f9926d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99280c2830 0x7f992814dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.390+0000 7f992659c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f992806da30 0x7f9928154680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:44460/0 (socket says 172.21.15.31:44460) 2024-03-30T01:41:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.390+0000 7f992659c700 1 -- 172.21.15.31:0/2094940321 learned_addr learned my addr 172.21.15.31:0/2094940321 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.391+0000 7f9926d9d700 1 -- 172.21.15.31:0/2094940321 >> v2:172.21.15.59:3300/0 conn(0x7f9928154d90 msgr2=0x7f9928150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.391+0000 7f9926d9d700 1 --2- 172.21.15.31:0/2094940321 >> v2:172.21.15.59:3300/0 conn(0x7f9928154d90 0x7f9928150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.391+0000 7f9926d9d700 1 -- 172.21.15.31:0/2094940321 >> v1:172.21.15.59:6789/0 conn(0x7f992806da30 legacy=0x7f9928154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.391+0000 7f9926d9d700 1 -- 172.21.15.31:0/2094940321 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928151a00 con 0x7f99280c2830 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.391+0000 7f9925d9b700 1 --2- 172.21.15.31:0/2094940321 >> v2:172.21.15.59:3300/0 conn(0x7f9928154d90 0x7f9928150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.391+0000 7f9926d9d700 1 --2- 172.21.15.31:0/2094940321 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99280c2830 0x7f992814dc50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9920009d90 tx=0x7f992000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5f311df5a73b6f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.391+0000 7f992559a700 1 -- 172.21.15.31:0/2094940321 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000e070 con 0x7f99280c2830 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.391+0000 7f992559a700 1 -- 172.21.15.31:0/2094940321 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992000bb20 con 0x7f99280c2830 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.391+0000 7f992559a700 1 -- 172.21.15.31:0/2094940321 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920012430 con 0x7f99280c2830 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.392+0000 7f992c84d700 1 -- 172.21.15.31:0/2094940321 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99280c2830 msgr2=0x7f992814dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.392+0000 7f992c84d700 1 --2- 172.21.15.31:0/2094940321 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99280c2830 0x7f992814dc50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9920009d90 tx=0x7f992000b330 comp rx=0 tx=0).stop 2024-03-30T01:41:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.392+0000 7f992c84d700 1 -- 172.21.15.31:0/2094940321 shutdown_connections 2024-03-30T01:41:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.392+0000 7f992c84d700 1 --2- 172.21.15.31:0/2094940321 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99280c2830 0x7f992814dc50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.392+0000 7f992c84d700 1 --2- 172.21.15.31:0/2094940321 >> v2:172.21.15.59:3300/0 conn(0x7f9928154d90 0x7f9928150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.392+0000 7f992c84d700 1 -- 172.21.15.31:0/2094940321 >> 172.21.15.31:0/2094940321 conn(0x7f99280c65c0 msgr2=0x7f99280c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.392+0000 7f992c84d700 1 -- 172.21.15.31:0/2094940321 shutdown_connections 2024-03-30T01:41:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.392+0000 7f992c84d700 1 -- 172.21.15.31:0/2094940321 wait complete. 2024-03-30T01:41:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.393+0000 7f992c84d700 1 Processor -- start 2024-03-30T01:41:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.393+0000 7f992c84d700 1 -- start start 2024-03-30T01:41:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f992c84d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992806da30 0x7f99281e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f992c84d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99280c2830 0x7f99281432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f992c84d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9928154d90 0x7f9928141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f992c84d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99280bbef0 con 0x7f992806da30 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f992c84d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99280bb5e0 con 0x7f99280c2830 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f992c84d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99280bcea0 con 0x7f9928154d90 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f992659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992806da30 0x7f99281e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f992659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992806da30 0x7f99281e9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52016/0 (socket says 172.21.15.31:52016) 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f992659c700 1 -- 172.21.15.31:0/1444221653 learned_addr learned my addr 172.21.15.31:0/1444221653 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f9925d9b700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99280c2830 0x7f99281432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.394+0000 7f9926d9d700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9928154d90 0x7f9928141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f992659c700 1 -- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9928154d90 msgr2=0x7f9928141b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f992659c700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9928154d90 0x7f9928141b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f992659c700 1 -- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99280c2830 msgr2=0x7f99281432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f992659c700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99280c2830 0x7f99281432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f992659c700 1 -- 172.21.15.31:0/1444221653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99100094a0 con 0x7f992806da30 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f9926d9d700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9928154d90 0x7f9928141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f9925d9b700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99280c2830 0x7f99281432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f992659c700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992806da30 0x7f99281e9aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f991800e970 tx=0x7f991800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:10.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f991effd700 1 -- 172.21.15.31:0/1444221653 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918005b40 con 0x7f992806da30 2024-03-30T01:41:10.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f991effd700 1 -- 172.21.15.31:0/1444221653 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918005ca0 con 0x7f992806da30 2024-03-30T01:41:10.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9920009a70 con 0x7f992806da30 2024-03-30T01:41:10.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f991effd700 1 -- 172.21.15.31:0/1444221653 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918001870 con 0x7f992806da30 2024-03-30T01:41:10.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.395+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99281ea3c0 con 0x7f992806da30 2024-03-30T01:41:10.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.396+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99280747d0 con 0x7f992806da30 2024-03-30T01:41:10.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.397+0000 7f991effd700 1 -- 172.21.15.31:0/1444221653 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f99180019d0 con 0x7f992806da30 2024-03-30T01:41:10.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.397+0000 7f991effd700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f990806ac90 0x7f990806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.397+0000 7f991effd700 1 -- 172.21.15.31:0/1444221653 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f991808b820 con 0x7f992806da30 2024-03-30T01:41:10.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.397+0000 7f9925d9b700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f990806ac90 0x7f990806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.398+0000 7f9925d9b700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f990806ac90 0x7f990806d150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f992815c450 tx=0x7f9910009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.402+0000 7f991effd700 1 -- 172.21.15.31:0/1444221653 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f99180567a0 con 0x7f992806da30 2024-03-30T01:41:10.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.618+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9928067fe0 con 0x7f992806da30 2024-03-30T01:41:10.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.618+0000 7f991effd700 1 -- 172.21.15.31:0/1444221653 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9918059dc0 con 0x7f992806da30 2024-03-30T01:41:10.625 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.620+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f990806ac90 msgr2=0x7f990806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.620+0000 7f992c84d700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f990806ac90 0x7f990806d150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f992815c450 tx=0x7f9910009210 comp rx=0 tx=0).stop 2024-03-30T01:41:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.620+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992806da30 msgr2=0x7f99281e9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.620+0000 7f992c84d700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992806da30 0x7f99281e9aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f991800e970 tx=0x7f991800ec80 comp rx=0 tx=0).stop 2024-03-30T01:41:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.620+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 shutdown_connections 2024-03-30T01:41:10.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.620+0000 7f992c84d700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f992806da30 0x7f99281e9aa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.620+0000 7f992c84d700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f990806ac90 0x7f990806d150 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.620+0000 7f992c84d700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9928154d90 0x7f9928141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.621+0000 7f992c84d700 1 --2- 172.21.15.31:0/1444221653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99280c2830 0x7f99281432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.621+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 >> 172.21.15.31:0/1444221653 conn(0x7f99280c65c0 msgr2=0x7f99280707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.621+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 shutdown_connections 2024-03-30T01:41:10.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.621+0000 7f992c84d700 1 -- 172.21.15.31:0/1444221653 wait complete. 2024-03-30T01:41:10.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T01:41:10.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.772+0000 7f245e7f7700 1 Processor -- start 2024-03-30T01:41:10.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.772+0000 7f245e7f7700 1 -- start start 2024-03-30T01:41:10.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.773+0000 7f245e7f7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f245814fbf0 0x7f245806a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.773+0000 7f245e7f7700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245806e4a0 con 0x7f24580a9560 2024-03-30T01:41:10.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.773+0000 7f245e7f7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2458155bf0 con 0x7f245806d9e0 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.773+0000 7f245e7f7700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2458156ac0 con 0x7f245814fbf0 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.773+0000 7f245cd94700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f24580a9560 0x7f24580ab0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48628/0 (socket says 172.21.15.31:48628) 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.773+0000 7f245cd94700 1 -- 172.21.15.31:0/3864882763 learned_addr learned my addr 172.21.15.31:0/3864882763 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.773+0000 7f2457fff700 1 --2- 172.21.15.31:0/3864882763 >> v2:172.21.15.59:3300/0 conn(0x7f245814fbf0 0x7f245806a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.774+0000 7f2456ffd700 1 -- 172.21.15.31:0/3864882763 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224130327 0 0) 0x7f245806e4a0 con 0x7f24580a9560 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.774+0000 7f2456ffd700 1 -- 172.21.15.31:0/3864882763 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24400035b0 con 0x7f24580a9560 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.774+0000 7f2457fff700 1 -- 172.21.15.31:0/3864882763 >> v1:172.21.15.59:6789/0 conn(0x7f245806d9e0 legacy=0x7f24581563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.774+0000 7f2457fff700 1 -- 172.21.15.31:0/3864882763 >> v1:172.21.15.31:6789/0 conn(0x7f24580a9560 legacy=0x7f24580ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.774+0000 7f2457fff700 1 -- 172.21.15.31:0/3864882763 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245806dee0 con 0x7f245814fbf0 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.774+0000 7f2457fff700 1 --2- 172.21.15.31:0/3864882763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245814fbf0 0x7f245806a700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2448009840 tx=0x7f2448009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1eb8b6936706f2a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.774+0000 7f2456ffd700 1 -- 172.21.15.31:0/3864882763 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448011830 con 0x7f245814fbf0 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.774+0000 7f2456ffd700 1 -- 172.21.15.31:0/3864882763 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2448011990 con 0x7f245814fbf0 2024-03-30T01:41:10.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.774+0000 7f2456ffd700 1 -- 172.21.15.31:0/3864882763 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448011d20 con 0x7f245814fbf0 2024-03-30T01:41:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.775+0000 7f245e7f7700 1 -- 172.21.15.31:0/3864882763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245814fbf0 msgr2=0x7f245806a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.775+0000 7f245e7f7700 1 --2- 172.21.15.31:0/3864882763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245814fbf0 0x7f245806a700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2448009840 tx=0x7f2448009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.775+0000 7f245e7f7700 1 -- 172.21.15.31:0/3864882763 shutdown_connections 2024-03-30T01:41:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.775+0000 7f245e7f7700 1 --2- 172.21.15.31:0/3864882763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245814fbf0 0x7f245806a700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.775+0000 7f245e7f7700 1 -- 172.21.15.31:0/3864882763 >> 172.21.15.31:0/3864882763 conn(0x7f24580c3190 msgr2=0x7f24580c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.775+0000 7f245e7f7700 1 -- 172.21.15.31:0/3864882763 shutdown_connections 2024-03-30T01:41:10.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.775+0000 7f245e7f7700 1 -- 172.21.15.31:0/3864882763 wait complete. 2024-03-30T01:41:10.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.776+0000 7f245e7f7700 1 Processor -- start 2024-03-30T01:41:10.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.776+0000 7f245e7f7700 1 -- start start 2024-03-30T01:41:10.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245e7f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245806d9e0 0x7f24580a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245e7f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24580a9560 0x7f24581ea190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245e7f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245814fbf0 0x7f245814cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245e7f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2458152960 con 0x7f245814fbf0 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245e7f7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24581ed5f0 con 0x7f245806d9e0 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245e7f7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24581ea6d0 con 0x7f24580a9560 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245cd94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245814fbf0 0x7f245814cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245cd94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245814fbf0 0x7f245814cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52018/0 (socket says 172.21.15.31:52018) 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245cd94700 1 -- 172.21.15.31:0/104293435 learned_addr learned my addr 172.21.15.31:0/104293435 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f2457fff700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245806d9e0 0x7f24580a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f24577fe700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24580a9560 0x7f24581ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245cd94700 1 -- 172.21.15.31:0/104293435 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24580a9560 msgr2=0x7f24581ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245cd94700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24580a9560 0x7f24581ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245cd94700 1 -- 172.21.15.31:0/104293435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245806d9e0 msgr2=0x7f24580a79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.777+0000 7f245cd94700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245806d9e0 0x7f24580a79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.778+0000 7f245cd94700 1 -- 172.21.15.31:0/104293435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24480094a0 con 0x7f245814fbf0 2024-03-30T01:41:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.778+0000 7f245cd94700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245814fbf0 0x7f245814cbe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f245000cb40 tx=0x7f245000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.778+0000 7f2454ff9700 1 -- 172.21.15.31:0/104293435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245000a980 con 0x7f245814fbf0 2024-03-30T01:41:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.778+0000 7f2454ff9700 1 -- 172.21.15.31:0/104293435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245000aae0 con 0x7f245814fbf0 2024-03-30T01:41:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.778+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24581ecce0 con 0x7f245814fbf0 2024-03-30T01:41:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.778+0000 7f2454ff9700 1 -- 172.21.15.31:0/104293435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245000ae50 con 0x7f245814fbf0 2024-03-30T01:41:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.778+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24581ecf30 con 0x7f245814fbf0 2024-03-30T01:41:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.779+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24581ed730 con 0x7f245814fbf0 2024-03-30T01:41:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.779+0000 7f2454ff9700 1 -- 172.21.15.31:0/104293435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f245001b3b0 con 0x7f245814fbf0 2024-03-30T01:41:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.780+0000 7f2454ff9700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.780+0000 7f2454ff9700 1 -- 172.21.15.31:0/104293435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f24500890d0 con 0x7f245814fbf0 2024-03-30T01:41:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.780+0000 7f2457fff700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.780+0000 7f2457fff700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f24480090b0 tx=0x7f244801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:10.785+0000 7f2454ff9700 1 -- 172.21.15.31:0/104293435 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f24581ed730 con 0x7f245814fbf0 2024-03-30T01:41:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.021+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f24581ed730 con 0x7f245814fbf0 2024-03-30T01:41:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.022+0000 7f2454ff9700 1 -- 172.21.15.31:0/104293435 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f24581ed730 con 0x7f245814fbf0 2024-03-30T01:41:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 msgr2=0x7f244406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f24480090b0 tx=0x7f244801c040 comp rx=0 tx=0).stop 2024-03-30T01:41:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245814fbf0 msgr2=0x7f245814cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245814fbf0 0x7f245814cbe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f245000cb40 tx=0x7f245000ce50 comp rx=0 tx=0).stop 2024-03-30T01:41:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 shutdown_connections 2024-03-30T01:41:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245814fbf0 0x7f245814cbe0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f244406ac90 0x7f244406d150 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f245806d9e0 0x7f24580a79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 --2- 172.21.15.31:0/104293435 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24580a9560 0x7f24581ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 >> 172.21.15.31:0/104293435 conn(0x7f24580c3190 msgr2=0x7f2458154fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:11.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 shutdown_connections 2024-03-30T01:41:11.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.024+0000 7f245e7f7700 1 -- 172.21.15.31:0/104293435 wait complete. 2024-03-30T01:41:11.032 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-03-30T01:41:11.044 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:41:11.044 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:11.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:11.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.237+0000 7f49248ec700 1 Processor -- start 2024-03-30T01:41:11.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.237+0000 7f49248ec700 1 -- start start 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.238+0000 7f49248ec700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f492006a700 0x7f49200c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.238+0000 7f49248ec700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f49200c4710 0x7f49200c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.238+0000 7f49248ec700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49201547d0 con 0x7f49200c3180 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.238+0000 7f49248ec700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4920154a90 con 0x7f492006a700 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.238+0000 7f49248ec700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4920154d70 con 0x7f49200c4710 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.238+0000 7f491dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f492006a700 0x7f49200c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491dd9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f492006a700 0x7f49200c0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52020/0 (socket says 172.21.15.31:52020) 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491dd9b700 1 -- 172.21.15.31:0/3105318034 learned_addr learned my addr 172.21.15.31:0/3105318034 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491ed9d700 1 --2- 172.21.15.31:0/3105318034 >> v2:172.21.15.59:3300/0 conn(0x7f49200c4710 0x7f49200c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491dd9b700 1 -- 172.21.15.31:0/3105318034 >> v2:172.21.15.59:3300/0 conn(0x7f49200c4710 msgr2=0x7f49200c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491dd9b700 1 --2- 172.21.15.31:0/3105318034 >> v2:172.21.15.59:3300/0 conn(0x7f49200c4710 0x7f49200c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491dd9b700 1 -- 172.21.15.31:0/3105318034 >> v1:172.21.15.59:6789/0 conn(0x7f49200c3180 legacy=0x7f492006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491dd9b700 1 -- 172.21.15.31:0/3105318034 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4920155050 con 0x7f492006a700 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491dd9b700 1 --2- 172.21.15.31:0/3105318034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 0x7f49200c0390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f49200c2ea0 tx=0x7f4910009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4edab994bbe899 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491d59a700 1 -- 172.21.15.31:0/3105318034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491000b870 con 0x7f492006a700 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491d59a700 1 -- 172.21.15.31:0/3105318034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491000b9d0 con 0x7f492006a700 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.239+0000 7f491d59a700 1 -- 172.21.15.31:0/3105318034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491000bd60 con 0x7f492006a700 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.240+0000 7f49248ec700 1 -- 172.21.15.31:0/3105318034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 msgr2=0x7f49200c0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.240+0000 7f49248ec700 1 --2- 172.21.15.31:0/3105318034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 0x7f49200c0390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f49200c2ea0 tx=0x7f4910009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.240+0000 7f49248ec700 1 -- 172.21.15.31:0/3105318034 shutdown_connections 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.240+0000 7f49248ec700 1 --2- 172.21.15.31:0/3105318034 >> v2:172.21.15.59:3300/0 conn(0x7f49200c4710 0x7f49200c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.240+0000 7f49248ec700 1 --2- 172.21.15.31:0/3105318034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 0x7f49200c0390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.240+0000 7f49248ec700 1 -- 172.21.15.31:0/3105318034 >> 172.21.15.31:0/3105318034 conn(0x7f49200c3a60 msgr2=0x7f49200c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:11.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.240+0000 7f49248ec700 1 -- 172.21.15.31:0/3105318034 shutdown_connections 2024-03-30T01:41:11.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.240+0000 7f49248ec700 1 -- 172.21.15.31:0/3105318034 wait complete. 2024-03-30T01:41:11.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.241+0000 7f49248ec700 1 Processor -- start 2024-03-30T01:41:11.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f49248ec700 1 -- start start 2024-03-30T01:41:11.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f49248ec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 0x7f492014aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f49248ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49200c3180 0x7f49200c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f49248ec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49200c4710 0x7f49201e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f49248ec700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49201ed550 con 0x7f492006a700 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f49248ec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49200ac400 con 0x7f49200c3180 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f49248ec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49200ac120 con 0x7f49200c4710 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f491e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 0x7f492014aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f491e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 0x7f492014aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52034/0 (socket says 172.21.15.31:52034) 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f491e59c700 1 -- 172.21.15.31:0/568089558 learned_addr learned my addr 172.21.15.31:0/568089558 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f491dd9b700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49200c3180 0x7f49200c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.242+0000 7f491ed9d700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49200c4710 0x7f49201e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f491e59c700 1 -- 172.21.15.31:0/568089558 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49200c4710 msgr2=0x7f49201e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f491e59c700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49200c4710 0x7f49201e9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f491e59c700 1 -- 172.21.15.31:0/568089558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49200c3180 msgr2=0x7f49200c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f491e59c700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49200c3180 0x7f49200c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f491e59c700 1 -- 172.21.15.31:0/568089558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910009550 con 0x7f492006a700 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f491ed9d700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49200c4710 0x7f49201e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f491e59c700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 0x7f492014aa40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f490800c490 tx=0x7f490800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f4916ffd700 1 -- 172.21.15.31:0/568089558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080052d0 con 0x7f492006a700 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f4916ffd700 1 -- 172.21.15.31:0/568089558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908005450 con 0x7f492006a700 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49201ecf00 con 0x7f492006a700 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f4916ffd700 1 -- 172.21.15.31:0/568089558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908006aa0 con 0x7f492006a700 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.243+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492014af80 con 0x7f492006a700 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.244+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492004ea80 con 0x7f492006a700 2024-03-30T01:41:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.245+0000 7f4916ffd700 1 -- 172.21.15.31:0/568089558 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4908003680 con 0x7f492006a700 2024-03-30T01:41:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.245+0000 7f4916ffd700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f490006ac90 0x7f490006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.245+0000 7f4916ffd700 1 -- 172.21.15.31:0/568089558 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f49080887d0 con 0x7f492006a700 2024-03-30T01:41:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.245+0000 7f491dd9b700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f490006ac90 0x7f490006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.246+0000 7f491dd9b700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f490006ac90 0x7f490006d150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4910000c00 tx=0x7f4910008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:11.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.250+0000 7f4916ffd700 1 -- 172.21.15.31:0/568089558 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4908053750 con 0x7f492006a700 2024-03-30T01:41:11.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.465+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49200722d0 con 0x7f492006a700 2024-03-30T01:41:11.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.466+0000 7f4916ffd700 1 -- 172.21.15.31:0/568089558 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f4908056d70 con 0x7f492006a700 2024-03-30T01:41:11.473 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:11.473 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:11.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f490006ac90 msgr2=0x7f490006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f490006ac90 0x7f490006d150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4910000c00 tx=0x7f4910008030 comp rx=0 tx=0).stop 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 msgr2=0x7f492014aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 0x7f492014aa40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f490800c490 tx=0x7f490800c7a0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 shutdown_connections 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492006a700 0x7f492014aa40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f490006ac90 0x7f490006d150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49200c4710 0x7f49201e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 --2- 172.21.15.31:0/568089558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49200c3180 0x7f49200c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 >> 172.21.15.31:0/568089558 conn(0x7f49200c3a60 msgr2=0x7f4920153fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 shutdown_connections 2024-03-30T01:41:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.469+0000 7f49248ec700 1 -- 172.21.15.31:0/568089558 wait complete. 2024-03-30T01:41:11.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:11.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.622+0000 7fa1b2e9a700 1 Processor -- start 2024-03-30T01:41:11.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.622+0000 7fa1b2e9a700 1 -- start start 2024-03-30T01:41:11.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.623+0000 7fa1b2e9a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac1525e0 0x7fa1ac0c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.623+0000 7fa1b2e9a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ac156930 con 0x7fa1ac0a91c0 2024-03-30T01:41:11.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.623+0000 7fa1b2e9a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ac156370 con 0x7fa1ac14fce0 2024-03-30T01:41:11.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.623+0000 7fa1b2e9a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0aa330 con 0x7fa1ac1525e0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.623+0000 7fa1b1437700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa1ac0a91c0 0x7fa1ac06f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48642/0 (socket says 172.21.15.31:48642) 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.623+0000 7fa1b1437700 1 -- 172.21.15.31:0/4247260399 learned_addr learned my addr 172.21.15.31:0/4247260399 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.623+0000 7fa1abfff700 1 --2- 172.21.15.31:0/4247260399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac1525e0 0x7fa1ac0c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.623+0000 7fa1ab7fe700 1 -- 172.21.15.31:0/4247260399 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735150727 0 0) 0x7fa1ac156930 con 0x7fa1ac0a91c0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.623+0000 7fa1ab7fe700 1 -- 172.21.15.31:0/4247260399 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa19c0035b0 con 0x7fa1ac0a91c0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1ab7fe700 1 -- 172.21.15.31:0/4247260399 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474532276 0 0) 0x7fa1ac156370 con 0x7fa1ac14fce0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1ab7fe700 1 -- 172.21.15.31:0/4247260399 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1ac156930 con 0x7fa1ac14fce0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1abfff700 1 -- 172.21.15.31:0/4247260399 >> v1:172.21.15.59:6789/0 conn(0x7fa1ac14fce0 legacy=0x7fa1ac155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1ab7fe700 1 -- 172.21.15.31:0/4247260399 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1517926445 0 0) 0x7fa19c0035b0 con 0x7fa1ac0a91c0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1abfff700 1 -- 172.21.15.31:0/4247260399 >> v1:172.21.15.31:6789/0 conn(0x7fa1ac0a91c0 legacy=0x7fa1ac06f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1abfff700 1 -- 172.21.15.31:0/4247260399 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac156650 con 0x7fa1ac1525e0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1abfff700 1 --2- 172.21.15.31:0/4247260399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac1525e0 0x7fa1ac0c1f80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa198009870 tx=0x7fa198009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e3c465095682715 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1ab7fe700 1 -- 172.21.15.31:0/4247260399 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40019f0 con 0x7fa1ac1525e0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1ab7fe700 1 -- 172.21.15.31:0/4247260399 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa198004030 con 0x7fa1ac1525e0 2024-03-30T01:41:11.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.624+0000 7fa1ab7fe700 1 -- 172.21.15.31:0/4247260399 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1980043c0 con 0x7fa1ac1525e0 2024-03-30T01:41:11.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.625+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/4247260399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac1525e0 msgr2=0x7fa1ac0c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.625+0000 7fa1b2e9a700 1 --2- 172.21.15.31:0/4247260399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac1525e0 0x7fa1ac0c1f80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa198009870 tx=0x7fa198009b80 comp rx=0 tx=0).stop 2024-03-30T01:41:11.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.625+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/4247260399 shutdown_connections 2024-03-30T01:41:11.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.625+0000 7fa1b2e9a700 1 --2- 172.21.15.31:0/4247260399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac1525e0 0x7fa1ac0c1f80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.625+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/4247260399 >> 172.21.15.31:0/4247260399 conn(0x7fa1ac0c3910 msgr2=0x7fa1ac0c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:11.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.625+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/4247260399 shutdown_connections 2024-03-30T01:41:11.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.625+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/4247260399 wait complete. 2024-03-30T01:41:11.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.626+0000 7fa1b2e9a700 1 Processor -- start 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.626+0000 7fa1b2e9a700 1 -- start start 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b2e9a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac0a91c0 0x7fa1ac1ea460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b2e9a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac14fce0 0x7fa1ac0c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b2e9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac1525e0 0x7fa1ac06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b2e9a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac153ce0 con 0x7fa1ac1525e0 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b2e9a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac1ea9a0 con 0x7fa1ac14fce0 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b2e9a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac06d120 con 0x7fa1ac0a91c0 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b1437700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac1525e0 0x7fa1ac06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b1437700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac1525e0 0x7fa1ac06d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52046/0 (socket says 172.21.15.31:52046) 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b1437700 1 -- 172.21.15.31:0/403206816 learned_addr learned my addr 172.21.15.31:0/403206816 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1abfff700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac14fce0 0x7fa1ac0c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b0c36700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac0a91c0 0x7fa1ac1ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b1437700 1 -- 172.21.15.31:0/403206816 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac0a91c0 msgr2=0x7fa1ac1ea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b1437700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac0a91c0 0x7fa1ac1ea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.627+0000 7fa1b1437700 1 -- 172.21.15.31:0/403206816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac14fce0 msgr2=0x7fa1ac0c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.628+0000 7fa1b1437700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac14fce0 0x7fa1ac0c04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.628+0000 7fa1b1437700 1 -- 172.21.15.31:0/403206816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980094a0 con 0x7fa1ac1525e0 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.628+0000 7fa1b1437700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac1525e0 0x7fa1ac06d600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4001700 tx=0x7fa1a400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.628+0000 7fa1a97fa700 1 -- 172.21.15.31:0/403206816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400aaa0 con 0x7fa1ac1525e0 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.628+0000 7fa1a97fa700 1 -- 172.21.15.31:0/403206816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a400ac00 con 0x7fa1ac1525e0 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.628+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac06e770 con 0x7fa1ac1525e0 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.628+0000 7fa1a97fa700 1 -- 172.21.15.31:0/403206816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40070d0 con 0x7fa1ac1525e0 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.628+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac06ed30 con 0x7fa1ac1525e0 2024-03-30T01:41:11.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.629+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ac066f90 con 0x7fa1ac1525e0 2024-03-30T01:41:11.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.630+0000 7fa1a97fa700 1 -- 172.21.15.31:0/403206816 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa1a4007230 con 0x7fa1ac1525e0 2024-03-30T01:41:11.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.630+0000 7fa1a97fa700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa19006ac90 0x7fa19006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:11.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.630+0000 7fa1a97fa700 1 -- 172.21.15.31:0/403206816 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa1a4089090 con 0x7fa1ac1525e0 2024-03-30T01:41:11.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.630+0000 7fa1b0c36700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa19006ac90 0x7fa19006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:11.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.631+0000 7fa1b0c36700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa19006ac90 0x7fa19006d150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0001220 tx=0x7fa1a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:11.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.635+0000 7fa1a97fa700 1 -- 172.21.15.31:0/403206816 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa1a4053f90 con 0x7fa1ac1525e0 2024-03-30T01:41:11.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.848+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1ac0747d0 con 0x7fa1ac1525e0 2024-03-30T01:41:11.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.848+0000 7fa1a97fa700 1 -- 172.21.15.31:0/403206816 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1a40575b0 con 0x7fa1ac1525e0 2024-03-30T01:41:11.855 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:11.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa19006ac90 msgr2=0x7fa19006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa19006ac90 0x7fa19006d150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0001220 tx=0x7fa1a0009040 comp rx=0 tx=0).stop 2024-03-30T01:41:11.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac1525e0 msgr2=0x7fa1ac06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:11.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac1525e0 0x7fa1ac06d600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4001700 tx=0x7fa1a400cf30 comp rx=0 tx=0).stop 2024-03-30T01:41:11.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 shutdown_connections 2024-03-30T01:41:11.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1ac1525e0 0x7fa1ac06d600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa19006ac90 0x7fa19006d150 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1ac0a91c0 0x7fa1ac1ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 --2- 172.21.15.31:0/403206816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac14fce0 0x7fa1ac0c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:11.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 >> 172.21.15.31:0/403206816 conn(0x7fa1ac0c3910 msgr2=0x7fa1ac071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:11.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.850+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 shutdown_connections 2024-03-30T01:41:11.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:11.851+0000 7fa1b2e9a700 1 -- 172.21.15.31:0/403206816 wait complete. 2024-03-30T01:41:11.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:12.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.003+0000 7f04d1b65700 1 Processor -- start 2024-03-30T01:41:12.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.003+0000 7f04d1b65700 1 -- start start 2024-03-30T01:41:12.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.004+0000 7f04d1b65700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c4aa0 0x7f04cc070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.004+0000 7f04d1b65700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04cc0a6e90 0x7f04cc06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.004+0000 7f04d1b65700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04cc0c37c0 con 0x7f04cc0c46c0 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.004+0000 7f04d1b65700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04cc070c60 con 0x7f04cc0a6e90 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.004+0000 7f04d1b65700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc0c3090 con 0x7f04cc0c4aa0 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.004+0000 7f04caffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c4aa0 0x7f04cc070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.004+0000 7f04cbfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04cc0a6e90 0x7f04cc06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.004+0000 7f04caffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c4aa0 0x7f04cc070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:46748/0 (socket says 172.21.15.31:46748) 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.004+0000 7f04caffd700 1 -- 172.21.15.31:0/1634969930 learned_addr learned my addr 172.21.15.31:0/1634969930 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.005+0000 7f04caffd700 1 -- 172.21.15.31:0/1634969930 >> v2:172.21.15.59:3300/0 conn(0x7f04cc0a6e90 msgr2=0x7f04cc06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.005+0000 7f04caffd700 1 --2- 172.21.15.31:0/1634969930 >> v2:172.21.15.59:3300/0 conn(0x7f04cc0a6e90 0x7f04cc06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.005+0000 7f04caffd700 1 -- 172.21.15.31:0/1634969930 >> v1:172.21.15.59:6789/0 conn(0x7f04cc0c46c0 legacy=0x7f04cc0c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.005+0000 7f04caffd700 1 -- 172.21.15.31:0/1634969930 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0c2890 con 0x7f04cc0c4aa0 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.005+0000 7f04cbfff700 1 --2- 172.21.15.31:0/1634969930 >> v2:172.21.15.59:3300/0 conn(0x7f04cc0a6e90 0x7f04cc06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.005+0000 7f04caffd700 1 --2- 172.21.15.31:0/1634969930 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c4aa0 0x7f04cc070680 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04c00098f0 tx=0x7f04c0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f156282a59792579 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:12.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.005+0000 7f04ca7fc700 1 -- 172.21.15.31:0/1634969930 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0011830 con 0x7f04cc0c4aa0 2024-03-30T01:41:12.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.005+0000 7f04ca7fc700 1 -- 172.21.15.31:0/1634969930 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c0011990 con 0x7f04cc0c4aa0 2024-03-30T01:41:12.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.005+0000 7f04ca7fc700 1 -- 172.21.15.31:0/1634969930 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0011d20 con 0x7f04cc0c4aa0 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.006+0000 7f04d1b65700 1 -- 172.21.15.31:0/1634969930 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c4aa0 msgr2=0x7f04cc070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.006+0000 7f04d1b65700 1 --2- 172.21.15.31:0/1634969930 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c4aa0 0x7f04cc070680 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04c00098f0 tx=0x7f04c0009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.006+0000 7f04d1b65700 1 -- 172.21.15.31:0/1634969930 shutdown_connections 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.006+0000 7f04d1b65700 1 --2- 172.21.15.31:0/1634969930 >> v2:172.21.15.59:3300/0 conn(0x7f04cc0a6e90 0x7f04cc06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.006+0000 7f04d1b65700 1 --2- 172.21.15.31:0/1634969930 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c4aa0 0x7f04cc070680 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.006+0000 7f04d1b65700 1 -- 172.21.15.31:0/1634969930 >> 172.21.15.31:0/1634969930 conn(0x7f04cc0c7cf0 msgr2=0x7f04cc0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.006+0000 7f04d1b65700 1 -- 172.21.15.31:0/1634969930 shutdown_connections 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.006+0000 7f04d1b65700 1 -- 172.21.15.31:0/1634969930 wait complete. 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.007+0000 7f04d1b65700 1 Processor -- start 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.007+0000 7f04d1b65700 1 -- start start 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04d1b65700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc0a6e90 0x7f04cc0a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04d1b65700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c46c0 0x7f04cc0a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04d1b65700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04cc0c4aa0 0x7f04cc0ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04d1b65700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc0a9320 con 0x7f04cc0a6e90 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04d1b65700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc06f270 con 0x7f04cc0c4aa0 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04d1b65700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc0a82e0 con 0x7f04cc0c46c0 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc0a6e90 0x7f04cc0a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc0a6e90 0x7f04cc0a8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52048/0 (socket says 172.21.15.31:52048) 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04cb7fe700 1 -- 172.21.15.31:0/2059038068 learned_addr learned my addr 172.21.15.31:0/2059038068 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04cbfff700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04cc0c4aa0 0x7f04cc0ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04caffd700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c46c0 0x7f04cc0a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04cb7fe700 1 -- 172.21.15.31:0/2059038068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c46c0 msgr2=0x7f04cc0a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04cb7fe700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c46c0 0x7f04cc0a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04cb7fe700 1 -- 172.21.15.31:0/2059038068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04cc0c4aa0 msgr2=0x7f04cc0ab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04cb7fe700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04cc0c4aa0 0x7f04cc0ab490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.008+0000 7f04cb7fe700 1 -- 172.21.15.31:0/2059038068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4009a70 con 0x7f04cc0a6e90 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.009+0000 7f04cb7fe700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc0a6e90 0x7f04cc0a8de0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00c4e0 tx=0x7f04bc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:12.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.009+0000 7f04b3fff700 1 -- 172.21.15.31:0/2059038068 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0052d0 con 0x7f04cc0a6e90 2024-03-30T01:41:12.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.009+0000 7f04b3fff700 1 -- 172.21.15.31:0/2059038068 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc005430 con 0x7f04cc0a6e90 2024-03-30T01:41:12.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.009+0000 7f04b3fff700 1 -- 172.21.15.31:0/2059038068 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0038c0 con 0x7f04cc0a6e90 2024-03-30T01:41:12.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.009+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c0009550 con 0x7f04cc0a6e90 2024-03-30T01:41:12.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.009+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04cc06ec20 con 0x7f04cc0a6e90 2024-03-30T01:41:12.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.010+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04cc0747d0 con 0x7f04cc0a6e90 2024-03-30T01:41:12.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.011+0000 7f04b3fff700 1 -- 172.21.15.31:0/2059038068 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f04bc003a20 con 0x7f04cc0a6e90 2024-03-30T01:41:12.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.012+0000 7f04b3fff700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f04b806ac90 0x7f04b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.012+0000 7f04b3fff700 1 -- 172.21.15.31:0/2059038068 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f04bc088dd0 con 0x7f04cc0a6e90 2024-03-30T01:41:12.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.012+0000 7f04caffd700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f04b806ac90 0x7f04b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.012+0000 7f04caffd700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f04b806ac90 0x7f04b806d150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009c40 tx=0x7f04c0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.016+0000 7f04b3fff700 1 -- 172.21.15.31:0/2059038068 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f04bc053d30 con 0x7f04cc0a6e90 2024-03-30T01:41:12.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.230+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04cc070920 con 0x7f04cc0a6e90 2024-03-30T01:41:12.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.230+0000 7f04b3fff700 1 -- 172.21.15.31:0/2059038068 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f04bc017020 con 0x7f04cc0a6e90 2024-03-30T01:41:12.237 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f04b806ac90 msgr2=0x7f04b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f04b806ac90 0x7f04b806d150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009c40 tx=0x7f04c0008030 comp rx=0 tx=0).stop 2024-03-30T01:41:12.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc0a6e90 msgr2=0x7f04cc0a8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc0a6e90 0x7f04cc0a8de0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00c4e0 tx=0x7f04bc00c7f0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 shutdown_connections 2024-03-30T01:41:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04cc0a6e90 0x7f04cc0a8de0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f04b806ac90 0x7f04b806d150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04cc0c4aa0 0x7f04cc0ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 --2- 172.21.15.31:0/2059038068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f04cc0c46c0 0x7f04cc0a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 >> 172.21.15.31:0/2059038068 conn(0x7f04cc0c7cf0 msgr2=0x7f04cc0c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 shutdown_connections 2024-03-30T01:41:12.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.233+0000 7f04d1b65700 1 -- 172.21.15.31:0/2059038068 wait complete. 2024-03-30T01:41:12.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:12.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.386+0000 7f52abf6a700 1 Processor -- start 2024-03-30T01:41:12.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.386+0000 7f52abf6a700 1 -- start start 2024-03-30T01:41:12.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.387+0000 7f52abf6a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52a40c64a0 0x7f52a40c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.387+0000 7f52abf6a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52a406cfc0 0x7f52a40c1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.387+0000 7f52abf6a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a40c1660 con 0x7f52a406c700 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.387+0000 7f52abf6a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a406bc60 con 0x7f52a406cfc0 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.387+0000 7f52abf6a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a40a77e0 con 0x7f52a40c64a0 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.387+0000 7f52a9505700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52a406cfc0 0x7f52a40c1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.387+0000 7f52a9505700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52a406cfc0 0x7f52a40c1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52058/0 (socket says 172.21.15.31:52058) 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.387+0000 7f52a9505700 1 -- 172.21.15.31:0/1612120324 learned_addr learned my addr 172.21.15.31:0/1612120324 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.387+0000 7f52aa507700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52a406c700 0x7f52a40705f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48656/0 (socket says 172.21.15.31:48656) 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.388+0000 7f52a9505700 1 -- 172.21.15.31:0/1612120324 >> v2:172.21.15.59:3300/0 conn(0x7f52a40c64a0 msgr2=0x7f52a40c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.388+0000 7f52a9505700 1 --2- 172.21.15.31:0/1612120324 >> v2:172.21.15.59:3300/0 conn(0x7f52a40c64a0 0x7f52a40c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.388+0000 7f52a9505700 1 -- 172.21.15.31:0/1612120324 >> v1:172.21.15.31:6789/0 conn(0x7f52a406c700 legacy=0x7f52a40705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.388+0000 7f52a9505700 1 -- 172.21.15.31:0/1612120324 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40c5d90 con 0x7f52a406cfc0 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.388+0000 7f52a8d04700 1 -- 172.21.15.31:0/1612120324 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507228773 0 0) 0x7f52a40c1660 con 0x7f52a406c700 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.388+0000 7f52a9505700 1 --2- 172.21.15.31:0/1612120324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a406cfc0 0x7f52a40c1120 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52940098f0 tx=0x7f5294009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7817e49cf122f68 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.388+0000 7f52a8d04700 1 -- 172.21.15.31:0/1612120324 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400b860 con 0x7f52a406cfc0 2024-03-30T01:41:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.388+0000 7f52a8d04700 1 -- 172.21.15.31:0/1612120324 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529400b9c0 con 0x7f52a406cfc0 2024-03-30T01:41:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.388+0000 7f52a8d04700 1 -- 172.21.15.31:0/1612120324 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400bd50 con 0x7f52a406cfc0 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.389+0000 7f52abf6a700 1 -- 172.21.15.31:0/1612120324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a406cfc0 msgr2=0x7f52a40c1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.389+0000 7f52abf6a700 1 --2- 172.21.15.31:0/1612120324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a406cfc0 0x7f52a40c1120 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52940098f0 tx=0x7f5294009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.389+0000 7f52abf6a700 1 -- 172.21.15.31:0/1612120324 shutdown_connections 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.389+0000 7f52abf6a700 1 --2- 172.21.15.31:0/1612120324 >> v2:172.21.15.59:3300/0 conn(0x7f52a40c64a0 0x7f52a40c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.389+0000 7f52abf6a700 1 --2- 172.21.15.31:0/1612120324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a406cfc0 0x7f52a40c1120 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.389+0000 7f52abf6a700 1 -- 172.21.15.31:0/1612120324 >> 172.21.15.31:0/1612120324 conn(0x7f52a40c7cf0 msgr2=0x7f52a40c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.389+0000 7f52abf6a700 1 -- 172.21.15.31:0/1612120324 shutdown_connections 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.389+0000 7f52abf6a700 1 -- 172.21.15.31:0/1612120324 wait complete. 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.390+0000 7f52abf6a700 1 Processor -- start 2024-03-30T01:41:12.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.390+0000 7f52abf6a700 1 -- start start 2024-03-30T01:41:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52abf6a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a406c700 0x7f52a41ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52abf6a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a406cfc0 0x7f52a41dc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52abf6a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40c64a0 0x7f52a406de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52abf6a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a41ea1b0 con 0x7f52a40c64a0 2024-03-30T01:41:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52abf6a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a41d5e80 con 0x7f52a406c700 2024-03-30T01:41:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52abf6a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a41e9b10 con 0x7f52a406cfc0 2024-03-30T01:41:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52aa507700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40c64a0 0x7f52a406de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52aa507700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40c64a0 0x7f52a406de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52074/0 (socket says 172.21.15.31:52074) 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52aa507700 1 -- 172.21.15.31:0/2838910491 learned_addr learned my addr 172.21.15.31:0/2838910491 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52a9d06700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a406c700 0x7f52a41ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52a9505700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a406cfc0 0x7f52a41dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52aa507700 1 -- 172.21.15.31:0/2838910491 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a406cfc0 msgr2=0x7f52a41dc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52aa507700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a406cfc0 0x7f52a41dc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52aa507700 1 -- 172.21.15.31:0/2838910491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a406c700 msgr2=0x7f52a41ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52aa507700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a406c700 0x7f52a41ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.391+0000 7f52aa507700 1 -- 172.21.15.31:0/2838910491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5294009550 con 0x7f52a40c64a0 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.392+0000 7f52aa507700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40c64a0 0x7f52a406de00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f529c002230 tx=0x7f529c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.392+0000 7f529a7fc700 1 -- 172.21.15.31:0/2838910491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c00aa60 con 0x7f52a40c64a0 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.392+0000 7f529a7fc700 1 -- 172.21.15.31:0/2838910491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c00abe0 con 0x7f52a40c64a0 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.392+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a41edb30 con 0x7f52a40c64a0 2024-03-30T01:41:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.392+0000 7f529a7fc700 1 -- 172.21.15.31:0/2838910491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c0070d0 con 0x7f52a40c64a0 2024-03-30T01:41:12.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.392+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a41e9dd0 con 0x7f52a40c64a0 2024-03-30T01:41:12.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.393+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a41ea2f0 con 0x7f52a40c64a0 2024-03-30T01:41:12.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.393+0000 7f529a7fc700 1 -- 172.21.15.31:0/2838910491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f529c007230 con 0x7f52a40c64a0 2024-03-30T01:41:12.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.394+0000 7f529a7fc700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f528806ac90 0x7f528806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.394+0000 7f529a7fc700 1 -- 172.21.15.31:0/2838910491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f529c089000 con 0x7f52a40c64a0 2024-03-30T01:41:12.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.394+0000 7f52a9d06700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f528806ac90 0x7f528806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.394+0000 7f52a9d06700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f528806ac90 0x7f528806d150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009970 tx=0x7f52a0009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:12.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.398+0000 7f529a7fc700 1 -- 172.21.15.31:0/2838910491 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f52a41ea2f0 con 0x7f52a40c64a0 2024-03-30T01:41:12.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.613+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52a41ea2f0 con 0x7f52a40c64a0 2024-03-30T01:41:12.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.614+0000 7f529a7fc700 1 -- 172.21.15.31:0/2838910491 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52a41ea2f0 con 0x7f52a40c64a0 2024-03-30T01:41:12.620 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:12.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f528806ac90 msgr2=0x7f528806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f528806ac90 0x7f528806d150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009970 tx=0x7f52a0009110 comp rx=0 tx=0).stop 2024-03-30T01:41:12.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40c64a0 msgr2=0x7f52a406de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40c64a0 0x7f52a406de00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f529c002230 tx=0x7f529c00cf50 comp rx=0 tx=0).stop 2024-03-30T01:41:12.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 shutdown_connections 2024-03-30T01:41:12.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40c64a0 0x7f52a406de00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f528806ac90 0x7f528806d150 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a406c700 0x7f52a41ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 --2- 172.21.15.31:0/2838910491 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a406cfc0 0x7f52a41dc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 >> 172.21.15.31:0/2838910491 conn(0x7f52a40c7cf0 msgr2=0x7f52a406f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:12.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 shutdown_connections 2024-03-30T01:41:12.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.616+0000 7f52abf6a700 1 -- 172.21.15.31:0/2838910491 wait complete. 2024-03-30T01:41:12.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:12.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.768+0000 7f56a8b69700 1 Processor -- start 2024-03-30T01:41:12.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.768+0000 7f56a8b69700 1 -- start start 2024-03-30T01:41:12.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.769+0000 7f56a8b69700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56a406d340 0x7f56a4154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.769+0000 7f56a8b69700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a4152070 con 0x7f56a40c1250 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.769+0000 7f56a8b69700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a40acef0 con 0x7f56a4151c90 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.769+0000 7f56a8b69700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a40acaa0 con 0x7f56a406d340 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.769+0000 7f56a2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f56a406d340 0x7f56a4154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.769+0000 7f56a1d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f56a40c1250 0x7f56a4154680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48668/0 (socket says 172.21.15.31:48668) 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.769+0000 7f56a1d9b700 1 -- 172.21.15.31:0/3546470501 learned_addr learned my addr 172.21.15.31:0/3546470501 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.770+0000 7f56a2d9d700 1 -- 172.21.15.31:0/3546470501 >> v1:172.21.15.59:6789/0 conn(0x7f56a4151c90 legacy=0x7f56a406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.770+0000 7f56a2d9d700 1 -- 172.21.15.31:0/3546470501 >> v1:172.21.15.31:6789/0 conn(0x7f56a40c1250 legacy=0x7f56a4154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.770+0000 7f56a159a700 1 -- 172.21.15.31:0/3546470501 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432193562 0 0) 0x7f56a4152070 con 0x7f56a40c1250 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.770+0000 7f56a2d9d700 1 -- 172.21.15.31:0/3546470501 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a40c31d0 con 0x7f56a406d340 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.770+0000 7f56a2d9d700 1 --2- 172.21.15.31:0/3546470501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a406d340 0x7f56a4154d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f569c009d90 tx=0x7f569c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e3f306af4232b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.770+0000 7f56a159a700 1 -- 172.21.15.31:0/3546470501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c00e070 con 0x7f56a406d340 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.770+0000 7f56a159a700 1 -- 172.21.15.31:0/3546470501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c00bbd0 con 0x7f56a406d340 2024-03-30T01:41:12.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.770+0000 7f56a159a700 1 -- 172.21.15.31:0/3546470501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c012430 con 0x7f56a406d340 2024-03-30T01:41:12.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.771+0000 7f56a8b69700 1 -- 172.21.15.31:0/3546470501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a406d340 msgr2=0x7f56a4154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.771+0000 7f56a8b69700 1 --2- 172.21.15.31:0/3546470501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a406d340 0x7f56a4154d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f569c009d90 tx=0x7f569c00b330 comp rx=0 tx=0).stop 2024-03-30T01:41:12.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.771+0000 7f56a8b69700 1 -- 172.21.15.31:0/3546470501 shutdown_connections 2024-03-30T01:41:12.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.771+0000 7f56a8b69700 1 --2- 172.21.15.31:0/3546470501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a406d340 0x7f56a4154d90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.771+0000 7f56a8b69700 1 -- 172.21.15.31:0/3546470501 >> 172.21.15.31:0/3546470501 conn(0x7f56a40c3910 msgr2=0x7f56a40c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:12.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.771+0000 7f56a8b69700 1 -- 172.21.15.31:0/3546470501 shutdown_connections 2024-03-30T01:41:12.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.771+0000 7f56a8b69700 1 -- 172.21.15.31:0/3546470501 wait complete. 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.772+0000 7f56a8b69700 1 Processor -- start 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.772+0000 7f56a8b69700 1 -- start start 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a8b69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56a406d340 0x7f56a41538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a8b69700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a40c1250 0x7f56a4156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a8b69700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56a4151c90 0x7f56a4157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a8b69700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a40bdee0 con 0x7f56a40c1250 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a8b69700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41576a0 con 0x7f56a406d340 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a8b69700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a40bfdf0 con 0x7f56a4151c90 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a40c1250 0x7f56a4156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a40c1250 0x7f56a4156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52090/0 (socket says 172.21.15.31:52090) 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a1d9b700 1 -- 172.21.15.31:0/492216307 learned_addr learned my addr 172.21.15.31:0/492216307 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a2d9d700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56a4151c90 0x7f56a4157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.773+0000 7f56a259c700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56a406d340 0x7f56a41538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f56a1d9b700 1 -- 172.21.15.31:0/492216307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56a4151c90 msgr2=0x7f56a4157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f56a1d9b700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56a4151c90 0x7f56a4157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f56a1d9b700 1 -- 172.21.15.31:0/492216307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56a406d340 msgr2=0x7f56a41538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f56a1d9b700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56a406d340 0x7f56a41538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f56a1d9b700 1 -- 172.21.15.31:0/492216307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c009a70 con 0x7f56a40c1250 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f56a2d9d700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56a4151c90 0x7f56a4157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f56a1d9b700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a40c1250 0x7f56a4156c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5694001eb0 tx=0x7f569400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f5692ffd700 1 -- 172.21.15.31:0/492216307 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694004c80 con 0x7f56a40c1250 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f5692ffd700 1 -- 172.21.15.31:0/492216307 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5694004de0 con 0x7f56a40c1250 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a40bf550 con 0x7f56a40c1250 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f5692ffd700 1 -- 172.21.15.31:0/492216307 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694007310 con 0x7f56a40c1250 2024-03-30T01:41:12.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.774+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a40bfb10 con 0x7f56a40c1250 2024-03-30T01:41:12.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.775+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a4066f90 con 0x7f56a40c1250 2024-03-30T01:41:12.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.776+0000 7f5692ffd700 1 -- 172.21.15.31:0/492216307 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5694007470 con 0x7f56a40c1250 2024-03-30T01:41:12.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.776+0000 7f5692ffd700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f568406ac90 0x7f568406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:12.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.776+0000 7f5692ffd700 1 -- 172.21.15.31:0/492216307 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5694089210 con 0x7f56a40c1250 2024-03-30T01:41:12.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.776+0000 7f56a259c700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f568406ac90 0x7f568406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:12.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.777+0000 7f56a259c700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f568406ac90 0x7f568406d150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5698007fd0 tx=0x7f5698009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:12.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.781+0000 7f5692ffd700 1 -- 172.21.15.31:0/492216307 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5694054190 con 0x7f56a40c1250 2024-03-30T01:41:13.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.997+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f56a40747d0 con 0x7f56a40c1250 2024-03-30T01:41:13.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.997+0000 7f5692ffd700 1 -- 172.21.15.31:0/492216307 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f56940577b0 con 0x7f56a40c1250 2024-03-30T01:41:13.004 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:13.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f568406ac90 msgr2=0x7f568406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f568406ac90 0x7f568406d150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5698007fd0 tx=0x7f5698009040 comp rx=0 tx=0).stop 2024-03-30T01:41:13.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a40c1250 msgr2=0x7f56a4156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a40c1250 0x7f56a4156c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5694001eb0 tx=0x7f569400c800 comp rx=0 tx=0).stop 2024-03-30T01:41:13.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 shutdown_connections 2024-03-30T01:41:13.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f568406ac90 0x7f568406d150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56a4151c90 0x7f56a4157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56a406d340 0x7f56a41538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 --2- 172.21.15.31:0/492216307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56a40c1250 0x7f56a4156c20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 >> 172.21.15.31:0/492216307 conn(0x7f56a40c3910 msgr2=0x7f56a406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:13.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 shutdown_connections 2024-03-30T01:41:13.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:12.999+0000 7f56a8b69700 1 -- 172.21.15.31:0/492216307 wait complete. 2024-03-30T01:41:13.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T01:41:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.149+0000 7efd8e129700 1 Processor -- start 2024-03-30T01:41:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.149+0000 7efd8e129700 1 -- start start 2024-03-30T01:41:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd8e129700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efd880c8310 0x7efd880c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd8e129700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd8e129700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd881556d0 con 0x7efd880c5df0 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd8e129700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd881559b0 con 0x7efd880c8310 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd8e129700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880c1f70 con 0x7efd880c2870 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd87fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd877fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efd880c8310 0x7efd880c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd86ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efd880c5df0 0x7efd88151d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:44536/0 (socket says 172.21.15.31:44536) 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd86ffd700 1 -- 172.21.15.31:0/2424763921 learned_addr learned my addr 172.21.15.31:0/2424763921 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.150+0000 7efd87fff700 1 --2- 172.21.15.31:0/2424763921 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:46774/0 (socket says 172.21.15.31:46774) 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.151+0000 7efd87fff700 1 -- 172.21.15.31:0/2424763921 >> v2:172.21.15.59:3300/0 conn(0x7efd880c8310 msgr2=0x7efd880c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.151+0000 7efd87fff700 1 --2- 172.21.15.31:0/2424763921 >> v2:172.21.15.59:3300/0 conn(0x7efd880c8310 0x7efd880c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.151+0000 7efd87fff700 1 -- 172.21.15.31:0/2424763921 >> v1:172.21.15.59:6789/0 conn(0x7efd880c5df0 legacy=0x7efd88151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.151+0000 7efd87fff700 1 -- 172.21.15.31:0/2424763921 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880c2250 con 0x7efd880c2870 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.151+0000 7efd877fe700 1 --2- 172.21.15.31:0/2424763921 >> v2:172.21.15.59:3300/0 conn(0x7efd880c8310 0x7efd880c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.151+0000 7efd87fff700 1 --2- 172.21.15.31:0/2424763921 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a6ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efd80009dc0 tx=0x7efd8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e89b0f0d1589c17d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.151+0000 7efd867fc700 1 -- 172.21.15.31:0/2424763921 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000e070 con 0x7efd880c2870 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.151+0000 7efd867fc700 1 -- 172.21.15.31:0/2424763921 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8000bae0 con 0x7efd880c2870 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.151+0000 7efd867fc700 1 -- 172.21.15.31:0/2424763921 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000be90 con 0x7efd880c2870 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.152+0000 7efd8e129700 1 -- 172.21.15.31:0/2424763921 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 msgr2=0x7efd880a6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.152+0000 7efd8e129700 1 --2- 172.21.15.31:0/2424763921 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a6ad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efd80009dc0 tx=0x7efd8000b330 comp rx=0 tx=0).stop 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.152+0000 7efd8e129700 1 -- 172.21.15.31:0/2424763921 shutdown_connections 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.152+0000 7efd8e129700 1 --2- 172.21.15.31:0/2424763921 >> v2:172.21.15.59:3300/0 conn(0x7efd880c8310 0x7efd880c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.152+0000 7efd8e129700 1 --2- 172.21.15.31:0/2424763921 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a6ad0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.152+0000 7efd8e129700 1 -- 172.21.15.31:0/2424763921 >> 172.21.15.31:0/2424763921 conn(0x7efd8806ab20 msgr2=0x7efd8806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.152+0000 7efd8e129700 1 -- 172.21.15.31:0/2424763921 shutdown_connections 2024-03-30T01:41:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.152+0000 7efd8e129700 1 -- 172.21.15.31:0/2424763921 wait complete. 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.153+0000 7efd8e129700 1 Processor -- start 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.153+0000 7efd8e129700 1 -- start start 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd8e129700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd8e129700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd880c5df0 0x7efd8814e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd8e129700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd880c8310 0x7efd8814e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd8e129700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd88156f20 con 0x7efd880c5df0 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd8e129700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd88157a50 con 0x7efd880c8310 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd8e129700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8814eda0 con 0x7efd880c2870 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd86ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd880c5df0 0x7efd8814e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd86ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd880c5df0 0x7efd8814e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52104/0 (socket says 172.21.15.31:52104) 2024-03-30T01:41:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd86ffd700 1 -- 172.21.15.31:0/2082130623 learned_addr learned my addr 172.21.15.31:0/2082130623 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd877fe700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd87fff700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd880c8310 0x7efd8814e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd86ffd700 1 -- 172.21.15.31:0/2082130623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 msgr2=0x7efd880a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd86ffd700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd86ffd700 1 -- 172.21.15.31:0/2082130623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd880c8310 msgr2=0x7efd8814e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd86ffd700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd880c8310 0x7efd8814e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.154+0000 7efd86ffd700 1 -- 172.21.15.31:0/2082130623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd780094a0 con 0x7efd880c5df0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.155+0000 7efd86ffd700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd880c5df0 0x7efd8814e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c520 tx=0x7efd7000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.155+0000 7efd7ffff700 1 -- 172.21.15.31:0/2082130623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70005d70 con 0x7efd880c5df0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.155+0000 7efd7ffff700 1 -- 172.21.15.31:0/2082130623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd700070d0 con 0x7efd880c5df0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.155+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd80009a70 con 0x7efd880c5df0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.155+0000 7efd7ffff700 1 -- 172.21.15.31:0/2082130623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70007440 con 0x7efd880c5df0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.155+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8806e690 con 0x7efd880c5df0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.156+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd88073760 con 0x7efd880c5df0 2024-03-30T01:41:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.156+0000 7efd7ffff700 1 -- 172.21.15.31:0/2082130623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efd700075a0 con 0x7efd880c5df0 2024-03-30T01:41:13.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.157+0000 7efd7ffff700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd6806ac90 0x7efd6806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.157+0000 7efd877fe700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd6806ac90 0x7efd6806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.157+0000 7efd7ffff700 1 -- 172.21.15.31:0/2082130623 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efd70088990 con 0x7efd880c5df0 2024-03-30T01:41:13.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.157+0000 7efd877fe700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd6806ac90 0x7efd6806d150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd880ad080 tx=0x7efd78009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:13.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.162+0000 7efd7ffff700 1 -- 172.21.15.31:0/2082130623 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efd70053890 con 0x7efd880c5df0 2024-03-30T01:41:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.393+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efd8804ea80 con 0x7efd880c5df0 2024-03-30T01:41:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.393+0000 7efd7ffff700 1 -- 172.21.15.31:0/2082130623 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7efd70056eb0 con 0x7efd880c5df0 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd6806ac90 msgr2=0x7efd6806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd6806ac90 0x7efd6806d150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd880ad080 tx=0x7efd78009210 comp rx=0 tx=0).stop 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd880c5df0 msgr2=0x7efd8814e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd880c5df0 0x7efd8814e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c520 tx=0x7efd7000c830 comp rx=0 tx=0).stop 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 shutdown_connections 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd6806ac90 0x7efd6806d150 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd880c8310 0x7efd8814e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd880c2870 0x7efd880a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 --2- 172.21.15.31:0/2082130623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd880c5df0 0x7efd8814e320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 >> 172.21.15.31:0/2082130623 conn(0x7efd8806ab20 msgr2=0x7efd88152320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 shutdown_connections 2024-03-30T01:41:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.396+0000 7efd8e129700 1 -- 172.21.15.31:0/2082130623 wait complete. 2024-03-30T01:41:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-03-30T01:41:13.416 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-03-30T01:41:13.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:13.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-30T01:41:13.466 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:ceph-fuse[54666]: fuse finished with error 0 and tester_r 0 2024-03-30T01:41:13.468 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T01:41:13.501 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-03-30T01:41:13.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:13.501 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-30T01:41:13.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:13.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.726+0000 7f32623e9700 1 Processor -- start 2024-03-30T01:41:13.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.726+0000 7f32623e9700 1 -- start start 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f32623e9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c06a700 0x7f325c0c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f32623e9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f325c0c4710 0x7f325c0c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f32623e9700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325c1547d0 con 0x7f325c0c3180 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f32623e9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325c154a90 con 0x7f325c0c4710 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f32623e9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325c154d70 con 0x7f325c06a700 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f325c0c3180 0x7f325c06eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:48678/0 (socket says 172.21.15.31:48678) 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325bfff700 1 -- 172.21.15.31:0/4091409955 learned_addr learned my addr 172.21.15.31:0/4091409955 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f3260986700 1 --2- 172.21.15.31:0/4091409955 >> v2:172.21.15.59:3300/0 conn(0x7f325c0c4710 0x7f325c0c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325b7fe700 1 --2- 172.21.15.31:0/4091409955 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c06a700 0x7f325c0c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325affd700 1 -- 172.21.15.31:0/4091409955 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015766930 0 0) 0x7f325c1547d0 con 0x7f325c0c3180 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325affd700 1 -- 172.21.15.31:0/4091409955 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32440035b0 con 0x7f325c0c3180 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325affd700 1 -- 172.21.15.31:0/4091409955 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4230334566 0 0) 0x7f32440035b0 con 0x7f325c0c3180 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325b7fe700 1 -- 172.21.15.31:0/4091409955 >> v2:172.21.15.59:3300/0 conn(0x7f325c0c4710 msgr2=0x7f325c0c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325b7fe700 1 --2- 172.21.15.31:0/4091409955 >> v2:172.21.15.59:3300/0 conn(0x7f325c0c4710 0x7f325c0c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325b7fe700 1 -- 172.21.15.31:0/4091409955 >> v1:172.21.15.31:6789/0 conn(0x7f325c0c3180 legacy=0x7f325c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f325b7fe700 1 -- 172.21.15.31:0/4091409955 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c155050 con 0x7f325c06a700 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.727+0000 7f3260986700 1 --2- 172.21.15.31:0/4091409955 >> v2:172.21.15.59:3300/0 conn(0x7f325c0c4710 0x7f325c0c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f325b7fe700 1 --2- 172.21.15.31:0/4091409955 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c06a700 0x7f325c0c0390 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3250009840 tx=0x7f3250009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f74c920daae6875 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f325affd700 1 -- 172.21.15.31:0/4091409955 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c001670 con 0x7f325c06a700 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f325affd700 1 -- 172.21.15.31:0/4091409955 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3250004030 con 0x7f325c06a700 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f325affd700 1 -- 172.21.15.31:0/4091409955 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500043c0 con 0x7f325c06a700 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f32623e9700 1 -- 172.21.15.31:0/4091409955 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c06a700 msgr2=0x7f325c0c0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f32623e9700 1 --2- 172.21.15.31:0/4091409955 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c06a700 0x7f325c0c0390 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3250009840 tx=0x7f3250009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f32623e9700 1 -- 172.21.15.31:0/4091409955 shutdown_connections 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f32623e9700 1 --2- 172.21.15.31:0/4091409955 >> v2:172.21.15.59:3300/0 conn(0x7f325c0c4710 0x7f325c0c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f32623e9700 1 --2- 172.21.15.31:0/4091409955 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c06a700 0x7f325c0c0390 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f32623e9700 1 -- 172.21.15.31:0/4091409955 >> 172.21.15.31:0/4091409955 conn(0x7f325c0c3a60 msgr2=0x7f325c0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:13.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f32623e9700 1 -- 172.21.15.31:0/4091409955 shutdown_connections 2024-03-30T01:41:13.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.728+0000 7f32623e9700 1 -- 172.21.15.31:0/4091409955 wait complete. 2024-03-30T01:41:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.730+0000 7f32623e9700 1 Processor -- start 2024-03-30T01:41:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.730+0000 7f32623e9700 1 -- start start 2024-03-30T01:41:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f32623e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f325c06a700 0x7f325c1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f32623e9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f325c0c3180 0x7f325c1ed060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f32623e9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c0c4710 0x7f325c1ed5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f32623e9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c148f60 con 0x7f325c0c3180 2024-03-30T01:41:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f32623e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c1edbc0 con 0x7f325c06a700 2024-03-30T01:41:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f32623e9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325c148910 con 0x7f325c0c4710 2024-03-30T01:41:13.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f325c0c3180 0x7f325c1ed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f325c0c3180 0x7f325c1ed060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52116/0 (socket says 172.21.15.31:52116) 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325b7fe700 1 -- 172.21.15.31:0/2127091458 learned_addr learned my addr 172.21.15.31:0/2127091458 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f3260986700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c0c4710 0x7f325c1ed5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325bfff700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f325c06a700 0x7f325c1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325b7fe700 1 -- 172.21.15.31:0/2127091458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c0c4710 msgr2=0x7f325c1ed5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325b7fe700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c0c4710 0x7f325c1ed5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325b7fe700 1 -- 172.21.15.31:0/2127091458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f325c06a700 msgr2=0x7f325c1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325b7fe700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f325c06a700 0x7f325c1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325b7fe700 1 -- 172.21.15.31:0/2127091458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3254009a70 con 0x7f325c0c3180 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f325b7fe700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f325c0c3180 0x7f325c1ed060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3250000c00 tx=0x7f3250004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f3258ff9700 1 -- 172.21.15.31:0/2127091458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3250004590 con 0x7f325c0c3180 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.731+0000 7f3258ff9700 1 -- 172.21.15.31:0/2127091458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325001f070 con 0x7f325c0c3180 2024-03-30T01:41:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.732+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32500094a0 con 0x7f325c0c3180 2024-03-30T01:41:13.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.732+0000 7f3258ff9700 1 -- 172.21.15.31:0/2127091458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3250011870 con 0x7f325c0c3180 2024-03-30T01:41:13.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.732+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c148620 con 0x7f325c0c3180 2024-03-30T01:41:13.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.733+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325c04ea80 con 0x7f325c0c3180 2024-03-30T01:41:13.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.733+0000 7f3258ff9700 1 -- 172.21.15.31:0/2127091458 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f32500119d0 con 0x7f325c0c3180 2024-03-30T01:41:13.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.734+0000 7f3258ff9700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f324806ac90 0x7f324806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:13.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.734+0000 7f3258ff9700 1 -- 172.21.15.31:0/2127091458 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3250021070 con 0x7f325c0c3180 2024-03-30T01:41:13.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.734+0000 7f325bfff700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f324806ac90 0x7f324806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:13.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.734+0000 7f325bfff700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f324806ac90 0x7f324806d150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f324c001eb0 tx=0x7f324c001c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:13.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.738+0000 7f3258ff9700 1 -- 172.21.15.31:0/2127091458 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3250055e70 con 0x7f325c0c3180 2024-03-30T01:41:13.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.957+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f325c0722d0 con 0x7f325c0c3180 2024-03-30T01:41:13.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.958+0000 7f3258ff9700 1 -- 172.21.15.31:0/2127091458 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f3250059490 con 0x7f325c0c3180 2024-03-30T01:41:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:13.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.960+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f324806ac90 msgr2=0x7f324806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.960+0000 7f32623e9700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f324806ac90 0x7f324806d150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f324c001eb0 tx=0x7f324c001c60 comp rx=0 tx=0).stop 2024-03-30T01:41:13.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f325c0c3180 msgr2=0x7f325c1ed060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:13.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f325c0c3180 0x7f325c1ed060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3250000c00 tx=0x7f3250004750 comp rx=0 tx=0).stop 2024-03-30T01:41:13.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 shutdown_connections 2024-03-30T01:41:13.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f324806ac90 0x7f324806d150 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f325c0c4710 0x7f325c1ed5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f325c06a700 0x7f325c1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 --2- 172.21.15.31:0/2127091458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f325c0c3180 0x7f325c1ed060 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:13.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 >> 172.21.15.31:0/2127091458 conn(0x7f325c0c3a60 msgr2=0x7f325c0c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:13.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 shutdown_connections 2024-03-30T01:41:13.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:13.961+0000 7f32623e9700 1 -- 172.21.15.31:0/2127091458 wait complete. 2024-03-30T01:41:13.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:14.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.112+0000 7ff35a2aa700 1 Processor -- start 2024-03-30T01:41:14.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.112+0000 7ff35a2aa700 1 -- start start 2024-03-30T01:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff35a2aa700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff35414fd30 0x7ff354156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff35a2aa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff354150110 0x7ff3540c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff35a2aa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3540c4d60 con 0x7ff3540c88c0 2024-03-30T01:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff35a2aa700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3540c5040 con 0x7ff35414fd30 2024-03-30T01:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff35a2aa700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3540c5320 con 0x7ff354150110 2024-03-30T01:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff353fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff35414fd30 0x7ff354156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff353fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff35414fd30 0x7ff354156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52120/0 (socket says 172.21.15.31:52120) 2024-03-30T01:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff353fff700 1 -- 172.21.15.31:0/1262140272 learned_addr learned my addr 172.21.15.31:0/1262140272 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff3537fe700 1 --2- 172.21.15.31:0/1262140272 >> v2:172.21.15.59:3300/0 conn(0x7ff354150110 0x7ff3540c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff353fff700 1 -- 172.21.15.31:0/1262140272 >> v2:172.21.15.59:3300/0 conn(0x7ff354150110 msgr2=0x7ff3540c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff353fff700 1 --2- 172.21.15.31:0/1262140272 >> v2:172.21.15.59:3300/0 conn(0x7ff354150110 0x7ff3540c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff353fff700 1 -- 172.21.15.31:0/1262140272 >> v1:172.21.15.59:6789/0 conn(0x7ff3540c88c0 legacy=0x7ff3540ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff353fff700 1 -- 172.21.15.31:0/1262140272 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540c5600 con 0x7ff35414fd30 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.113+0000 7ff353fff700 1 --2- 172.21.15.31:0/1262140272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 0x7ff354156800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff3540aac20 tx=0x7ff344009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1ee6a5eef75836 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff352ffd700 1 -- 172.21.15.31:0/1262140272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344004230 con 0x7ff35414fd30 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff352ffd700 1 -- 172.21.15.31:0/1262140272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff344004390 con 0x7ff35414fd30 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff352ffd700 1 -- 172.21.15.31:0/1262140272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3440036a0 con 0x7ff35414fd30 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff35a2aa700 1 -- 172.21.15.31:0/1262140272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 msgr2=0x7ff354156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff35a2aa700 1 --2- 172.21.15.31:0/1262140272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 0x7ff354156800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff3540aac20 tx=0x7ff344009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff35a2aa700 1 -- 172.21.15.31:0/1262140272 shutdown_connections 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff35a2aa700 1 --2- 172.21.15.31:0/1262140272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 0x7ff354156800 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff35a2aa700 1 --2- 172.21.15.31:0/1262140272 >> v2:172.21.15.59:3300/0 conn(0x7ff354150110 0x7ff3540c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff35a2aa700 1 -- 172.21.15.31:0/1262140272 >> 172.21.15.31:0/1262140272 conn(0x7ff35406ab20 msgr2=0x7ff35406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff35a2aa700 1 -- 172.21.15.31:0/1262140272 shutdown_connections 2024-03-30T01:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.114+0000 7ff35a2aa700 1 -- 172.21.15.31:0/1262140272 wait complete. 2024-03-30T01:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.115+0000 7ff35a2aa700 1 Processor -- start 2024-03-30T01:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.116+0000 7ff35a2aa700 1 -- start start 2024-03-30T01:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.116+0000 7ff35a2aa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3540c88c0 0x7ff35406ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.116+0000 7ff35a2aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 0x7ff3540c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.116+0000 7ff35a2aa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff354150110 0x7ff3541dcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.116+0000 7ff35a2aa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35406c630 con 0x7ff35414fd30 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.116+0000 7ff35a2aa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35406cf90 con 0x7ff354150110 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.116+0000 7ff35a2aa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff354143fc0 con 0x7ff3540c88c0 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff3537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 0x7ff3540c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff3537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 0x7ff3540c1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52130/0 (socket says 172.21.15.31:52130) 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff3537fe700 1 -- 172.21.15.31:0/3744095685 learned_addr learned my addr 172.21.15.31:0/3744095685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff358847700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff354150110 0x7ff3541dcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff353fff700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3540c88c0 0x7ff35406ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff3537fe700 1 -- 172.21.15.31:0/3744095685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3540c88c0 msgr2=0x7ff35406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff3537fe700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3540c88c0 0x7ff35406ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff3537fe700 1 -- 172.21.15.31:0/3744095685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff354150110 msgr2=0x7ff3541dcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff3537fe700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff354150110 0x7ff3541dcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff3537fe700 1 -- 172.21.15.31:0/3744095685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff344009550 con 0x7ff35414fd30 2024-03-30T01:41:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff353fff700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3540c88c0 0x7ff35406ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff3537fe700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 0x7ff3540c1470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d800 tx=0x7ff33c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff350ff9700 1 -- 172.21.15.31:0/3744095685 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0125f0 con 0x7ff35414fd30 2024-03-30T01:41:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff350ff9700 1 -- 172.21.15.31:0/3744095685 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c012750 con 0x7ff35414fd30 2024-03-30T01:41:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff350ff9700 1 -- 172.21.15.31:0/3744095685 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c012b00 con 0x7ff35414fd30 2024-03-30T01:41:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff354150750 con 0x7ff35414fd30 2024-03-30T01:41:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.117+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35406bc90 con 0x7ff35414fd30 2024-03-30T01:41:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.119+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff354073760 con 0x7ff35414fd30 2024-03-30T01:41:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.119+0000 7ff350ff9700 1 -- 172.21.15.31:0/3744095685 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff33c012c60 con 0x7ff35414fd30 2024-03-30T01:41:14.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.119+0000 7ff350ff9700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff33406ac90 0x7ff33406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.120+0000 7ff350ff9700 1 -- 172.21.15.31:0/3744095685 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff33c089910 con 0x7ff35414fd30 2024-03-30T01:41:14.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.120+0000 7ff353fff700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff33406ac90 0x7ff33406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.120+0000 7ff353fff700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff33406ac90 0x7ff33406d150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff344009c10 tx=0x7ff344008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:14.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.124+0000 7ff350ff9700 1 -- 172.21.15.31:0/3744095685 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff33c054890 con 0x7ff35414fd30 2024-03-30T01:41:14.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.344+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff35404ea80 con 0x7ff35414fd30 2024-03-30T01:41:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.346+0000 7ff350ff9700 1 -- 172.21.15.31:0/3744095685 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7ff33c057eb0 con 0x7ff35414fd30 2024-03-30T01:41:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:14.353 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff33406ac90 msgr2=0x7ff33406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff33406ac90 0x7ff33406d150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff344009c10 tx=0x7ff344008030 comp rx=0 tx=0).stop 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 msgr2=0x7ff3540c1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 0x7ff3540c1470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d800 tx=0x7ff33c00dbc0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 shutdown_connections 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff33406ac90 0x7ff33406d150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff354150110 0x7ff3541dcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3540c88c0 0x7ff35406ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 --2- 172.21.15.31:0/3744095685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35414fd30 0x7ff3540c1470 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 >> 172.21.15.31:0/3744095685 conn(0x7ff35406ab20 msgr2=0x7ff35406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:14.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 shutdown_connections 2024-03-30T01:41:14.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.348+0000 7ff35a2aa700 1 -- 172.21.15.31:0/3744095685 wait complete. 2024-03-30T01:41:14.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T01:41:14.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.503+0000 7ffaee41a700 1 Processor -- start 2024-03-30T01:41:14.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.503+0000 7ffaee41a700 1 -- start start 2024-03-30T01:41:14.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.504+0000 7ffaee41a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae8151e00 0x7ffae806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.504+0000 7ffaee41a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffae806cf70 0x7ffae806a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.504+0000 7ffaee41a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffae806ac40 0x7ffae8154860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.504+0000 7ffaee41a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae81521e0 con 0x7ffae806ac40 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.504+0000 7ffaee41a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae80acc90 con 0x7ffae806cf70 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.504+0000 7ffaee41a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80acf70 con 0x7ffae8151e00 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.504+0000 7ffaec9b7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffae806ac40 0x7ffae8154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.504+0000 7ffaec9b7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffae806ac40 0x7ffae8154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52138/0 (socket says 172.21.15.31:52138) 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.504+0000 7ffaec9b7700 1 -- 172.21.15.31:0/1893956418 learned_addr learned my addr 172.21.15.31:0/1893956418 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffae77fe700 1 --2- 172.21.15.31:0/1893956418 >> v2:172.21.15.59:3300/0 conn(0x7ffae806cf70 0x7ffae806a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffae7fff700 1 --2- 172.21.15.31:0/1893956418 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae8151e00 0x7ffae806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffaec9b7700 1 -- 172.21.15.31:0/1893956418 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae8151e00 msgr2=0x7ffae806e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffaec9b7700 1 --2- 172.21.15.31:0/1893956418 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae8151e00 0x7ffae806e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffaec9b7700 1 -- 172.21.15.31:0/1893956418 >> v2:172.21.15.59:3300/0 conn(0x7ffae806cf70 msgr2=0x7ffae806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffaec9b7700 1 --2- 172.21.15.31:0/1893956418 >> v2:172.21.15.59:3300/0 conn(0x7ffae806cf70 0x7ffae806a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffaec9b7700 1 -- 172.21.15.31:0/1893956418 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80c3150 con 0x7ffae806ac40 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffaec9b7700 1 --2- 172.21.15.31:0/1893956418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae806ac40 0x7ffae8154860 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffae80722c0 tx=0x7ffae000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b49cdfe67c08dc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffae6ffd700 1 -- 172.21.15.31:0/1893956418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000e070 con 0x7ffae806ac40 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffae6ffd700 1 -- 172.21.15.31:0/1893956418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000b910 con 0x7ffae806ac40 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.505+0000 7ffae6ffd700 1 -- 172.21.15.31:0/1893956418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000bcc0 con 0x7ffae806ac40 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.506+0000 7ffaee41a700 1 -- 172.21.15.31:0/1893956418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae806ac40 msgr2=0x7ffae8154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.506+0000 7ffaee41a700 1 --2- 172.21.15.31:0/1893956418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae806ac40 0x7ffae8154860 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffae80722c0 tx=0x7ffae000b330 comp rx=0 tx=0).stop 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.506+0000 7ffaee41a700 1 -- 172.21.15.31:0/1893956418 shutdown_connections 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.506+0000 7ffaee41a700 1 --2- 172.21.15.31:0/1893956418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae806ac40 0x7ffae8154860 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.506+0000 7ffaee41a700 1 --2- 172.21.15.31:0/1893956418 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae8151e00 0x7ffae806e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.506+0000 7ffaee41a700 1 --2- 172.21.15.31:0/1893956418 >> v2:172.21.15.59:3300/0 conn(0x7ffae806cf70 0x7ffae806a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.506+0000 7ffaee41a700 1 -- 172.21.15.31:0/1893956418 >> 172.21.15.31:0/1893956418 conn(0x7ffae80c3880 msgr2=0x7ffae80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.506+0000 7ffaee41a700 1 -- 172.21.15.31:0/1893956418 shutdown_connections 2024-03-30T01:41:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.506+0000 7ffaee41a700 1 -- 172.21.15.31:0/1893956418 wait complete. 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.507+0000 7ffaee41a700 1 Processor -- start 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.507+0000 7ffaee41a700 1 -- start start 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffaee41a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae806ac40 0x7ffae8154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffaee41a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae806cf70 0x7ffae80bf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffaee41a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae8151e00 0x7ffae81562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffaee41a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8157d90 con 0x7ffae8151e00 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffaee41a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8154a90 con 0x7ffae806ac40 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffaee41a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae8155a00 con 0x7ffae806cf70 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffaec9b7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae8151e00 0x7ffae81562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffaec9b7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae8151e00 0x7ffae81562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52146/0 (socket says 172.21.15.31:52146) 2024-03-30T01:41:14.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffaec9b7700 1 -- 172.21.15.31:0/2484646684 learned_addr learned my addr 172.21.15.31:0/2484646684 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffae77fe700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae806cf70 0x7ffae80bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.508+0000 7ffae7fff700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae806ac40 0x7ffae8154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffaec9b7700 1 -- 172.21.15.31:0/2484646684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae806cf70 msgr2=0x7ffae80bf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffaec9b7700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae806cf70 0x7ffae80bf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffaec9b7700 1 -- 172.21.15.31:0/2484646684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae806ac40 msgr2=0x7ffae8154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffaec9b7700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae806ac40 0x7ffae8154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffaec9b7700 1 -- 172.21.15.31:0/2484646684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae0009b30 con 0x7ffae8151e00 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffaec9b7700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae8151e00 0x7ffae81562e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffae8072520 tx=0x7ffae0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffae4ff9700 1 -- 172.21.15.31:0/2484646684 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae00099b0 con 0x7ffae8151e00 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffae4ff9700 1 -- 172.21.15.31:0/2484646684 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000ce80 con 0x7ffae8151e00 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffae4ff9700 1 -- 172.21.15.31:0/2484646684 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0004490 con 0x7ffae8151e00 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae81586e0 con 0x7ffae8151e00 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.509+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae8158b70 con 0x7ffae8151e00 2024-03-30T01:41:14.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.510+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae806eca0 con 0x7ffae8151e00 2024-03-30T01:41:14.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.511+0000 7ffae4ff9700 1 -- 172.21.15.31:0/2484646684 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffae00045f0 con 0x7ffae8151e00 2024-03-30T01:41:14.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.511+0000 7ffae4ff9700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffac806ac90 0x7ffac806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:14.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.511+0000 7ffae4ff9700 1 -- 172.21.15.31:0/2484646684 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffae008a8a0 con 0x7ffae8151e00 2024-03-30T01:41:14.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.511+0000 7ffae7fff700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffac806ac90 0x7ffac806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:14.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.512+0000 7ffae7fff700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffac806ac90 0x7ffac806d150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009b40 tx=0x7ffad8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:14.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.516+0000 7ffae4ff9700 1 -- 172.21.15.31:0/2484646684 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ffae0055820 con 0x7ffae8151e00 2024-03-30T01:41:14.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:14.777+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffae8158870 con 0x7ffae8151e00 2024-03-30T01:41:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.756+0000 7ffae4ff9700 1 -- 172.21.15.31:0/2484646684 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffae00200b0 con 0x7ffae8151e00 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffac806ac90 msgr2=0x7ffac806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffac806ac90 0x7ffac806d150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009b40 tx=0x7ffad8009210 comp rx=0 tx=0).stop 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae8151e00 msgr2=0x7ffae81562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae8151e00 0x7ffae81562e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffae8072520 tx=0x7ffae0004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 shutdown_connections 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffae8151e00 0x7ffae81562e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffac806ac90 0x7ffac806d150 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffae806ac40 0x7ffae8154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 --2- 172.21.15.31:0/2484646684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffae806cf70 0x7ffae80bf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:15.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 >> 172.21.15.31:0/2484646684 conn(0x7ffae80c3880 msgr2=0x7ffae806d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:15.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.759+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 shutdown_connections 2024-03-30T01:41:15.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.760+0000 7ffaee41a700 1 -- 172.21.15.31:0/2484646684 wait complete. 2024-03-30T01:41:15.783 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-30T01:41:15.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T01:41:15.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.924+0000 7fc538f76700 1 Processor -- start 2024-03-30T01:41:15.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.924+0000 7fc538f76700 1 -- start start 2024-03-30T01:41:15.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.925+0000 7fc538f76700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc53414e9a0 0x7fc53406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.925+0000 7fc538f76700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5340c31f0 con 0x7fc53406d340 2024-03-30T01:41:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.925+0000 7fc538f76700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53414ed80 con 0x7fc534154930 2024-03-30T01:41:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.925+0000 7fc538f76700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340c2220 con 0x7fc53414e9a0 2024-03-30T01:41:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.925+0000 7fc531d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc53406d340 0x7fc53406e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34430/0 (socket says 172.21.15.31:34430) 2024-03-30T01:41:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.925+0000 7fc531d9b700 1 -- 172.21.15.31:0/1673848617 learned_addr learned my addr 172.21.15.31:0/1673848617 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.925+0000 7fc53259c700 1 --2- 172.21.15.31:0/1673848617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc53414e9a0 0x7fc53406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.925+0000 7fc53159a700 1 -- 172.21.15.31:0/1673848617 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805047383 0 0) 0x7fc5340c31f0 con 0x7fc53406d340 2024-03-30T01:41:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.926+0000 7fc53159a700 1 -- 172.21.15.31:0/1673848617 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc51c0035b0 con 0x7fc53406d340 2024-03-30T01:41:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.926+0000 7fc53159a700 1 -- 172.21.15.31:0/1673848617 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272483519 0 0) 0x7fc51c0035b0 con 0x7fc53406d340 2024-03-30T01:41:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.926+0000 7fc53259c700 1 -- 172.21.15.31:0/1673848617 >> v1:172.21.15.59:6789/0 conn(0x7fc534154930 legacy=0x7fc534154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.926+0000 7fc53259c700 1 -- 172.21.15.31:0/1673848617 >> v1:172.21.15.31:6789/0 conn(0x7fc53406d340 legacy=0x7fc53406e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.926+0000 7fc53259c700 1 -- 172.21.15.31:0/1673848617 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5340a8790 con 0x7fc53414e9a0 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.926+0000 7fc53259c700 1 --2- 172.21.15.31:0/1673848617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc53414e9a0 0x7fc53406e050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc524009870 tx=0x7fc524009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8364d930164caa32 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.926+0000 7fc53159a700 1 -- 172.21.15.31:0/1673848617 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528001670 con 0x7fc53414e9a0 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.926+0000 7fc53159a700 1 -- 172.21.15.31:0/1673848617 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc524004030 con 0x7fc53414e9a0 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.926+0000 7fc53159a700 1 -- 172.21.15.31:0/1673848617 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5240043a0 con 0x7fc53414e9a0 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.927+0000 7fc538f76700 1 -- 172.21.15.31:0/1673848617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc53414e9a0 msgr2=0x7fc53406e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.927+0000 7fc538f76700 1 --2- 172.21.15.31:0/1673848617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc53414e9a0 0x7fc53406e050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc524009870 tx=0x7fc524009b80 comp rx=0 tx=0).stop 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.927+0000 7fc538f76700 1 -- 172.21.15.31:0/1673848617 shutdown_connections 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.927+0000 7fc538f76700 1 --2- 172.21.15.31:0/1673848617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc53414e9a0 0x7fc53406e050 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:15.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.927+0000 7fc538f76700 1 -- 172.21.15.31:0/1673848617 >> 172.21.15.31:0/1673848617 conn(0x7fc5340c3910 msgr2=0x7fc5340c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.927+0000 7fc538f76700 1 -- 172.21.15.31:0/1673848617 shutdown_connections 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.927+0000 7fc538f76700 1 -- 172.21.15.31:0/1673848617 wait complete. 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.928+0000 7fc538f76700 1 Processor -- start 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.928+0000 7fc538f76700 1 -- start start 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc538f76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc53406d340 0x7fc53414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc538f76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53414e9a0 0x7fc53414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc538f76700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc534154930 0x7fc534148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc538f76700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc534150230 con 0x7fc53414e9a0 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc538f76700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5341e8440 con 0x7fc53406d340 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc538f76700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc534150440 con 0x7fc534154930 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc531d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53414e9a0 0x7fc53414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc531d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53414e9a0 0x7fc53414a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36376/0 (socket says 172.21.15.31:36376) 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc531d9b700 1 -- 172.21.15.31:0/2191214671 learned_addr learned my addr 172.21.15.31:0/2191214671 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc53259c700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc53406d340 0x7fc53414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.929+0000 7fc532d9d700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc534154930 0x7fc534148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc531d9b700 1 -- 172.21.15.31:0/2191214671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc534154930 msgr2=0x7fc534148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc531d9b700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc534154930 0x7fc534148bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc531d9b700 1 -- 172.21.15.31:0/2191214671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc53406d340 msgr2=0x7fc53414fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc531d9b700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc53406d340 0x7fc53414fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc531d9b700 1 -- 172.21.15.31:0/2191214671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240094a0 con 0x7fc53414e9a0 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc531d9b700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53414e9a0 0x7fc53414a240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5280017a0 tx=0x7fc52800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc522ffd700 1 -- 172.21.15.31:0/2191214671 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528004c80 con 0x7fc53414e9a0 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc522ffd700 1 -- 172.21.15.31:0/2191214671 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc528004de0 con 0x7fc53414e9a0 2024-03-30T01:41:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5341e7680 con 0x7fc53414e9a0 2024-03-30T01:41:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc522ffd700 1 -- 172.21.15.31:0/2191214671 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528007310 con 0x7fc53414e9a0 2024-03-30T01:41:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.930+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5341e7c40 con 0x7fc53414e9a0 2024-03-30T01:41:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.931+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc534066f90 con 0x7fc53414e9a0 2024-03-30T01:41:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.932+0000 7fc522ffd700 1 -- 172.21.15.31:0/2191214671 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc528007470 con 0x7fc53414e9a0 2024-03-30T01:41:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.932+0000 7fc522ffd700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc51406ac90 0x7fc51406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.932+0000 7fc522ffd700 1 -- 172.21.15.31:0/2191214671 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc528089210 con 0x7fc53414e9a0 2024-03-30T01:41:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.932+0000 7fc53259c700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc51406ac90 0x7fc51406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.933+0000 7fc53259c700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc51406ac90 0x7fc51406d150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc524000c00 tx=0x7fc524008000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:15.937+0000 7fc522ffd700 1 -- 172.21.15.31:0/2191214671 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc528054250 con 0x7fc53414e9a0 2024-03-30T01:41:16.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.198+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc53404ea80 con 0x7fc53414e9a0 2024-03-30T01:41:16.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.760+0000 7fc522ffd700 1 -- 172.21.15.31:0/2191214671 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc528057870 con 0x7fc53414e9a0 2024-03-30T01:41:16.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc51406ac90 msgr2=0x7fc51406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc51406ac90 0x7fc51406d150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc524000c00 tx=0x7fc524008000 comp rx=0 tx=0).stop 2024-03-30T01:41:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53414e9a0 msgr2=0x7fc53414a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53414e9a0 0x7fc53414a240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5280017a0 tx=0x7fc52800ca30 comp rx=0 tx=0).stop 2024-03-30T01:41:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 shutdown_connections 2024-03-30T01:41:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc51406ac90 0x7fc51406d150 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc534154930 0x7fc534148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc53406d340 0x7fc53414fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 --2- 172.21.15.31:0/2191214671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53414e9a0 0x7fc53414a240 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 >> 172.21.15.31:0/2191214671 conn(0x7fc5340c3910 msgr2=0x7fc5340c3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 shutdown_connections 2024-03-30T01:41:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.764+0000 7fc538f76700 1 -- 172.21.15.31:0/2191214671 wait complete. 2024-03-30T01:41:16.786 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T01:41:16.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:16.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.920+0000 7f54c5bd0700 1 Processor -- start 2024-03-30T01:41:16.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.920+0000 7f54c5bd0700 1 -- start start 2024-03-30T01:41:16.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54c5bd0700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54c0154d90 0x7f54c0155170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:16.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54c5bd0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 0x7f54c0071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:16.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54c5bd0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f54c006bf80 0x7f54c01479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:16.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54c5bd0700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c01557d0 con 0x7f54c0154d90 2024-03-30T01:41:16.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54c5bd0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c0070f80 con 0x7f54c006bf80 2024-03-30T01:41:16.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54c5bd0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c0071260 con 0x7f54c00c0800 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bf7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54c0154d90 0x7f54c0155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bf7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54c0154d90 0x7f54c0155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36392/0 (socket says 172.21.15.31:36392) 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bf7fe700 1 -- 172.21.15.31:0/2249002881 learned_addr learned my addr 172.21.15.31:0/2249002881 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bffff700 1 --2- 172.21.15.31:0/2249002881 >> v2:172.21.15.59:3300/0 conn(0x7f54c006bf80 0x7f54c01479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54beffd700 1 --2- 172.21.15.31:0/2249002881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 0x7f54c0071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bf7fe700 1 -- 172.21.15.31:0/2249002881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 msgr2=0x7f54c0071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bf7fe700 1 --2- 172.21.15.31:0/2249002881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 0x7f54c0071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bf7fe700 1 -- 172.21.15.31:0/2249002881 >> v2:172.21.15.59:3300/0 conn(0x7f54c006bf80 msgr2=0x7f54c01479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bf7fe700 1 --2- 172.21.15.31:0/2249002881 >> v2:172.21.15.59:3300/0 conn(0x7f54c006bf80 0x7f54c01479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bf7fe700 1 -- 172.21.15.31:0/2249002881 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c0071540 con 0x7f54c0154d90 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54beffd700 1 --2- 172.21.15.31:0/2249002881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 0x7f54c0071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.921+0000 7f54bf7fe700 1 --2- 172.21.15.31:0/2249002881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 0x7f54c0155170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54c006b6c0 tx=0x7f54b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581c7b988b7c248f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:16.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54be7fc700 1 -- 172.21.15.31:0/2249002881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b000b870 con 0x7f54c0154d90 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54be7fc700 1 -- 172.21.15.31:0/2249002881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b000b9d0 con 0x7f54c0154d90 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54be7fc700 1 -- 172.21.15.31:0/2249002881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b000bd60 con 0x7f54c0154d90 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2249002881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 msgr2=0x7f54c0155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2249002881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 0x7f54c0155170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54c006b6c0 tx=0x7f54b0009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2249002881 shutdown_connections 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2249002881 >> v2:172.21.15.59:3300/0 conn(0x7f54c006bf80 0x7f54c01479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2249002881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 0x7f54c0155170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2249002881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 0x7f54c0071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2249002881 >> 172.21.15.31:0/2249002881 conn(0x7f54c006af80 msgr2=0x7f54c00c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2249002881 shutdown_connections 2024-03-30T01:41:16.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.922+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2249002881 wait complete. 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.924+0000 7f54c5bd0700 1 Processor -- start 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.924+0000 7f54c5bd0700 1 -- start start 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.924+0000 7f54c5bd0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54c006bf80 0x7f54c01432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.924+0000 7f54c5bd0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 0x7f54c01526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.924+0000 7f54c5bd0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 0x7f54c0150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54c5bd0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c01515e0 con 0x7f54c0154d90 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54c5bd0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c0152c00 con 0x7f54c006bf80 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54c5bd0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c014dcf0 con 0x7f54c00c0800 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 0x7f54c0150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 0x7f54c0150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36394/0 (socket says 172.21.15.31:36394) 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bffff700 1 -- 172.21.15.31:0/2681663788 learned_addr learned my addr 172.21.15.31:0/2681663788 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bf7fe700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54c006bf80 0x7f54c01432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54beffd700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 0x7f54c01526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bffff700 1 -- 172.21.15.31:0/2681663788 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 msgr2=0x7f54c01526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bffff700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 0x7f54c01526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bffff700 1 -- 172.21.15.31:0/2681663788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54c006bf80 msgr2=0x7f54c01432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bffff700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54c006bf80 0x7f54c01432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bffff700 1 -- 172.21.15.31:0/2681663788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b0009550 con 0x7f54c0154d90 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54bffff700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 0x7f54c0150aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54b8007e60 tx=0x7f54b800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54b7fff700 1 -- 172.21.15.31:0/2681663788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b800ed30 con 0x7f54c0154d90 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54b7fff700 1 -- 172.21.15.31:0/2681663788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b8014420 con 0x7f54c0154d90 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54b7fff700 1 -- 172.21.15.31:0/2681663788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80147d0 con 0x7f54c0154d90 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c00727a0 con 0x7f54c0154d90 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.925+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c014df90 con 0x7f54c0154d90 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.926+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c004ea80 con 0x7f54c0154d90 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.927+0000 7f54b7fff700 1 -- 172.21.15.31:0/2681663788 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f54b8014930 con 0x7f54c0154d90 2024-03-30T01:41:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.927+0000 7f54b7fff700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f54a006ac90 0x7f54a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.928+0000 7f54b7fff700 1 -- 172.21.15.31:0/2681663788 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f54b8089c20 con 0x7f54c0154d90 2024-03-30T01:41:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.928+0000 7f54bf7fe700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f54a006ac90 0x7f54a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.928+0000 7f54bf7fe700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f54a006ac90 0x7f54a006d150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54b0014040 tx=0x7f54b0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:16.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:16.932+0000 7f54b7fff700 1 -- 172.21.15.31:0/2681663788 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f54b8054bb0 con 0x7f54c0154d90 2024-03-30T01:41:17.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.195+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54c00c0c60 con 0x7f54c0154d90 2024-03-30T01:41:17.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.195+0000 7f54b7fff700 1 -- 172.21.15.31:0/2681663788 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f54b80581d0 con 0x7f54c0154d90 2024-03-30T01:41:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:17.202 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:16.755934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T01:41:17.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f54a006ac90 msgr2=0x7f54a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f54a006ac90 0x7f54a006d150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54b0014040 tx=0x7f54b0008030 comp rx=0 tx=0).stop 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 msgr2=0x7f54c0150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 0x7f54c0150aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54b8007e60 tx=0x7f54b800e3c0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 shutdown_connections 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54c0154d90 0x7f54c0150aa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f54a006ac90 0x7f54a006d150 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54c006bf80 0x7f54c01432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 --2- 172.21.15.31:0/2681663788 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54c00c0800 0x7f54c01526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 >> 172.21.15.31:0/2681663788 conn(0x7f54c006af80 msgr2=0x7f54c0146f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 shutdown_connections 2024-03-30T01:41:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.198+0000 7f54c5bd0700 1 -- 172.21.15.31:0/2681663788 wait complete. 2024-03-30T01:41:17.206 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-03-30T01:41:17.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:17.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.347+0000 7fce4c4a2700 1 Processor -- start 2024-03-30T01:41:17.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.347+0000 7fce4c4a2700 1 -- start start 2024-03-30T01:41:17.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4c4a2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fce440c8310 0x7fce440c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4c4a2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c5df0 0x7fce44151d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4c4a2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce441556d0 con 0x7fce440c2870 2024-03-30T01:41:17.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4c4a2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce441559b0 con 0x7fce440c8310 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4c4a2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce440c1f70 con 0x7fce440c5df0 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4aa3f700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fce440c2870 0x7fce440a6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34436/0 (socket says 172.21.15.31:34436) 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4aa3f700 1 -- 172.21.15.31:0/1248183023 learned_addr learned my addr 172.21.15.31:0/1248183023 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4a23e700 1 --2- 172.21.15.31:0/1248183023 >> v2:172.21.15.59:3300/0 conn(0x7fce440c8310 0x7fce440c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce49a3d700 1 --2- 172.21.15.31:0/1248183023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c5df0 0x7fce44151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4923c700 1 -- 172.21.15.31:0/1248183023 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126359798 0 0) 0x7fce441556d0 con 0x7fce440c2870 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.348+0000 7fce4923c700 1 -- 172.21.15.31:0/1248183023 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce300035b0 con 0x7fce440c2870 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.349+0000 7fce4a23e700 1 -- 172.21.15.31:0/1248183023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c5df0 msgr2=0x7fce44151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.349+0000 7fce4a23e700 1 --2- 172.21.15.31:0/1248183023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c5df0 0x7fce44151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.349+0000 7fce4a23e700 1 -- 172.21.15.31:0/1248183023 >> v1:172.21.15.31:6789/0 conn(0x7fce440c2870 legacy=0x7fce440a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.349+0000 7fce4a23e700 1 -- 172.21.15.31:0/1248183023 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440c2250 con 0x7fce440c8310 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.349+0000 7fce4a23e700 1 --2- 172.21.15.31:0/1248183023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce440c8310 0x7fce440c86f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fce40009840 tx=0x7fce40009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a478350a0de628c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.349+0000 7fce4923c700 1 -- 172.21.15.31:0/1248183023 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40011830 con 0x7fce440c8310 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.349+0000 7fce4923c700 1 -- 172.21.15.31:0/1248183023 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce40011990 con 0x7fce440c8310 2024-03-30T01:41:17.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.349+0000 7fce4923c700 1 -- 172.21.15.31:0/1248183023 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40011d20 con 0x7fce440c8310 2024-03-30T01:41:17.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.350+0000 7fce4c4a2700 1 -- 172.21.15.31:0/1248183023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce440c8310 msgr2=0x7fce440c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.350+0000 7fce4c4a2700 1 --2- 172.21.15.31:0/1248183023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce440c8310 0x7fce440c86f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fce40009840 tx=0x7fce40009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:17.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.350+0000 7fce4c4a2700 1 -- 172.21.15.31:0/1248183023 shutdown_connections 2024-03-30T01:41:17.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.350+0000 7fce4c4a2700 1 --2- 172.21.15.31:0/1248183023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce440c8310 0x7fce440c86f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.350+0000 7fce4c4a2700 1 --2- 172.21.15.31:0/1248183023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c5df0 0x7fce44151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.350+0000 7fce4c4a2700 1 -- 172.21.15.31:0/1248183023 >> 172.21.15.31:0/1248183023 conn(0x7fce4406ab20 msgr2=0x7fce4406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:17.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.350+0000 7fce4c4a2700 1 -- 172.21.15.31:0/1248183023 shutdown_connections 2024-03-30T01:41:17.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.350+0000 7fce4c4a2700 1 -- 172.21.15.31:0/1248183023 wait complete. 2024-03-30T01:41:17.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.351+0000 7fce4c4a2700 1 Processor -- start 2024-03-30T01:41:17.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.351+0000 7fce4c4a2700 1 -- start start 2024-03-30T01:41:17.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4c4a2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c2870 0x7fce44157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4c4a2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce440c5df0 0x7fce440c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4c4a2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce440c8310 0x7fce44150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4c4a2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440a6bc0 con 0x7fce440c8310 2024-03-30T01:41:17.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4c4a2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce44157ca0 con 0x7fce440c5df0 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4c4a2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce441e9790 con 0x7fce440c2870 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4aa3f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce440c8310 0x7fce44150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4aa3f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce440c8310 0x7fce44150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36400/0 (socket says 172.21.15.31:36400) 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4aa3f700 1 -- 172.21.15.31:0/2758751747 learned_addr learned my addr 172.21.15.31:0/2758751747 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce49a3d700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce440c5df0 0x7fce440c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4a23e700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c2870 0x7fce44157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4aa3f700 1 -- 172.21.15.31:0/2758751747 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c2870 msgr2=0x7fce44157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4aa3f700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c2870 0x7fce44157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4aa3f700 1 -- 172.21.15.31:0/2758751747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce440c5df0 msgr2=0x7fce440c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4aa3f700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce440c5df0 0x7fce440c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4aa3f700 1 -- 172.21.15.31:0/2758751747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce400094a0 con 0x7fce440c8310 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.352+0000 7fce4aa3f700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce440c8310 0x7fce44150540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00cb40 tx=0x7fce3c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.353+0000 7fce3affd700 1 -- 172.21.15.31:0/2758751747 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00ab50 con 0x7fce440c8310 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.353+0000 7fce3affd700 1 -- 172.21.15.31:0/2758751747 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c00acb0 con 0x7fce440c8310 2024-03-30T01:41:17.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.353+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce441e9a70 con 0x7fce440c8310 2024-03-30T01:41:17.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.353+0000 7fce3affd700 1 -- 172.21.15.31:0/2758751747 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c012620 con 0x7fce440c8310 2024-03-30T01:41:17.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.353+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce441ea030 con 0x7fce440c8310 2024-03-30T01:41:17.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.354+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce44073760 con 0x7fce440c8310 2024-03-30T01:41:17.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.354+0000 7fce3affd700 1 -- 172.21.15.31:0/2758751747 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fce3c01b3b0 con 0x7fce440c8310 2024-03-30T01:41:17.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.355+0000 7fce3affd700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce2806ac90 0x7fce2806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.355+0000 7fce3affd700 1 -- 172.21.15.31:0/2758751747 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fce3c0893b0 con 0x7fce440c8310 2024-03-30T01:41:17.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.355+0000 7fce4a23e700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce2806ac90 0x7fce2806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.355+0000 7fce4a23e700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce2806ac90 0x7fce2806d150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce40000c00 tx=0x7fce4001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:17.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.359+0000 7fce3affd700 1 -- 172.21.15.31:0/2758751747 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fce3c054330 con 0x7fce440c8310 2024-03-30T01:41:17.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.574+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce441ea550 con 0x7fce440c8310 2024-03-30T01:41:17.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.575+0000 7fce3affd700 1 -- 172.21.15.31:0/2758751747 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7fce441ea550 con 0x7fce440c8310 2024-03-30T01:41:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:17.582 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce2806ac90 msgr2=0x7fce2806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce2806ac90 0x7fce2806d150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fce40000c00 tx=0x7fce4001c040 comp rx=0 tx=0).stop 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce440c8310 msgr2=0x7fce44150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce440c8310 0x7fce44150540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00cb40 tx=0x7fce3c00cf00 comp rx=0 tx=0).stop 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 shutdown_connections 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce440c8310 0x7fce44150540 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce2806ac90 0x7fce2806d150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce440c2870 0x7fce44157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 --2- 172.21.15.31:0/2758751747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce440c5df0 0x7fce440c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 >> 172.21.15.31:0/2758751747 conn(0x7fce4406ab20 msgr2=0x7fce440ace90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 shutdown_connections 2024-03-30T01:41:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.578+0000 7fce4c4a2700 1 -- 172.21.15.31:0/2758751747 wait complete. 2024-03-30T01:41:17.599 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T01:41:17.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:41:17.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.728+0000 7f039d21e700 1 Processor -- start 2024-03-30T01:41:17.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.728+0000 7f039d21e700 1 -- start start 2024-03-30T01:41:17.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.729+0000 7f039d21e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 0x7f0398154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.729+0000 7f039d21e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0398071750 0x7f0398154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.729+0000 7f039d21e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0398072160 con 0x7f039814e5f0 2024-03-30T01:41:17.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.729+0000 7f039d21e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03980c2bd0 con 0x7f0398071750 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.729+0000 7f039d21e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03980c2e90 con 0x7f0398071240 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.729+0000 7f039759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0398071750 0x7f0398154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.729+0000 7f039759e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0398071750 0x7f0398154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36404/0 (socket says 172.21.15.31:36404) 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.729+0000 7f0396d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f039814e5f0 0x7f03980c3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34444/0 (socket says 172.21.15.31:34444) 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.729+0000 7f039759e700 1 -- 172.21.15.31:0/2784420257 learned_addr learned my addr 172.21.15.31:0/2784420257 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.730+0000 7f039659c700 1 --2- 172.21.15.31:0/2784420257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 0x7f0398154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.730+0000 7f039759e700 1 -- 172.21.15.31:0/2784420257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 msgr2=0x7f0398154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.730+0000 7f039759e700 1 --2- 172.21.15.31:0/2784420257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 0x7f0398154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.730+0000 7f039759e700 1 -- 172.21.15.31:0/2784420257 >> v1:172.21.15.31:6789/0 conn(0x7f039814e5f0 legacy=0x7f03980c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.730+0000 7f039759e700 1 -- 172.21.15.31:0/2784420257 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03980723b0 con 0x7f0398071750 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.730+0000 7f039759e700 1 --2- 172.21.15.31:0/2784420257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 0x7f0398154dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0390009e50 tx=0x7f039000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad31d2a08f6bf8f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:17.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.730+0000 7f0395d9b700 1 -- 172.21.15.31:0/2784420257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000e070 con 0x7f0398071750 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.730+0000 7f0395d9b700 1 -- 172.21.15.31:0/2784420257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039000b9c0 con 0x7f0398071750 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.730+0000 7f0395d9b700 1 -- 172.21.15.31:0/2784420257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000bd70 con 0x7f0398071750 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.731+0000 7f039d21e700 1 -- 172.21.15.31:0/2784420257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 msgr2=0x7f0398154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.731+0000 7f039d21e700 1 --2- 172.21.15.31:0/2784420257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 0x7f0398154dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0390009e50 tx=0x7f039000b330 comp rx=0 tx=0).stop 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.731+0000 7f039d21e700 1 -- 172.21.15.31:0/2784420257 shutdown_connections 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.731+0000 7f039d21e700 1 --2- 172.21.15.31:0/2784420257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 0x7f0398154dd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.731+0000 7f039d21e700 1 --2- 172.21.15.31:0/2784420257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 0x7f0398154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.731+0000 7f039d21e700 1 -- 172.21.15.31:0/2784420257 >> 172.21.15.31:0/2784420257 conn(0x7f039806ba20 msgr2=0x7f039806be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.731+0000 7f039d21e700 1 -- 172.21.15.31:0/2784420257 shutdown_connections 2024-03-30T01:41:17.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.731+0000 7f039d21e700 1 -- 172.21.15.31:0/2784420257 wait complete. 2024-03-30T01:41:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.733+0000 7f039d21e700 1 Processor -- start 2024-03-30T01:41:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.733+0000 7f039d21e700 1 -- start start 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039d21e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 0x7f0398153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039d21e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 0x7f0398149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039d21e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f039814e5f0 0x7f039814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039d21e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03980c6c40 con 0x7f0398071750 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039d21e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039814a650 con 0x7f039814e5f0 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039d21e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03981ed080 con 0x7f0398071240 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 0x7f0398149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 0x7f0398149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36408/0 (socket says 172.21.15.31:36408) 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039659c700 1 -- 172.21.15.31:0/4080132323 learned_addr learned my addr 172.21.15.31:0/4080132323 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:17.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039759e700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f039814e5f0 0x7f039814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f0396d9d700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 0x7f0398153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039659c700 1 -- 172.21.15.31:0/4080132323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 msgr2=0x7f0398153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039659c700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 0x7f0398153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039659c700 1 -- 172.21.15.31:0/4080132323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f039814e5f0 msgr2=0x7f039814a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039659c700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f039814e5f0 0x7f039814a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f039659c700 1 -- 172.21.15.31:0/4080132323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390009b30 con 0x7f0398071750 2024-03-30T01:41:17.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.734+0000 7f0396d9d700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 0x7f0398153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:17.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.735+0000 7f039659c700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 0x7f0398149bd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f038c00d7d0 tx=0x7f038c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:17.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.735+0000 7f037f7fe700 1 -- 172.21.15.31:0/4080132323 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0125f0 con 0x7f0398071750 2024-03-30T01:41:17.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.735+0000 7f037f7fe700 1 -- 172.21.15.31:0/4080132323 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c012750 con 0x7f0398071750 2024-03-30T01:41:17.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.735+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03981ea970 con 0x7f0398071750 2024-03-30T01:41:17.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.735+0000 7f037f7fe700 1 -- 172.21.15.31:0/4080132323 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c012ae0 con 0x7f0398071750 2024-03-30T01:41:17.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.735+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03981ed620 con 0x7f0398071750 2024-03-30T01:41:17.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.736+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039804ea80 con 0x7f0398071750 2024-03-30T01:41:17.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.736+0000 7f037f7fe700 1 -- 172.21.15.31:0/4080132323 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f038c012c40 con 0x7f0398071750 2024-03-30T01:41:17.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.737+0000 7f037f7fe700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f038406ac90 0x7f038406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:17.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.737+0000 7f0396d9d700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f038406ac90 0x7f038406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:17.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.737+0000 7f037f7fe700 1 -- 172.21.15.31:0/4080132323 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f038c0896b0 con 0x7f0398071750 2024-03-30T01:41:17.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.738+0000 7f0396d9d700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f038406ac90 0x7f038406d150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f60 tx=0x7f0388007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:17.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.742+0000 7f037f7fe700 1 -- 172.21.15.31:0/4080132323 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f038c0545a0 con 0x7f0398071750 2024-03-30T01:41:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.955+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03980c1440 con 0x7f038406ac90 2024-03-30T01:41:17.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.966+0000 7f037f7fe700 1 -- 172.21.15.31:0/4080132323 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345314 (secure 0 0 0) 0x7f0388005cd0 con 0x7f038406ac90 2024-03-30T01:41:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f038406ac90 msgr2=0x7f038406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f038406ac90 0x7f038406d150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f60 tx=0x7f0388007dd0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 msgr2=0x7f0398149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 0x7f0398149bd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f038c00d7d0 tx=0x7f038c00dae0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 shutdown_connections 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f038406ac90 0x7f038406d150 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f039814e5f0 0x7f039814a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0398071240 0x7f0398153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 --2- 172.21.15.31:0/4080132323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0398071750 0x7f0398149bd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 >> 172.21.15.31:0/4080132323 conn(0x7f039806ba20 msgr2=0x7f03980c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 shutdown_connections 2024-03-30T01:41:17.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:17.970+0000 7f039d21e700 1 -- 172.21.15.31:0/4080132323 wait complete. 2024-03-30T01:41:17.982 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:41:17.990 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-03-30T01:41:17.803630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594821,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219364,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755356,"statfs":{"total":773094113280,"available":772869484544,"internally_reserved":0,"allocated":3280896,"data_stored":1986913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2091,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002862"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965171+0000","last_change":"2024-03-30T01:40:58.441027+0000","last_active":"2024-03-30T01:41:02.965171+0000","last_peered":"2024-03-30T01:41:02.965171+0000","last_clean":"2024-03-30T01:41:02.965171+0000","last_became_active":"2024-03-30T01:40:58.440741+0000","last_became_peered":"2024-03-30T01:40:58.440741+0000","last_unstale":"2024-03-30T01:41:02.965171+0000","last_undegraded":"2024-03-30T01:41:02.965171+0000","last_fullsized":"2024-03-30T01:41:02.965171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868566+0000","last_change":"2024-03-30T01:40:58.426588+0000","last_active":"2024-03-30T01:41:02.868566+0000","last_peered":"2024-03-30T01:41:02.868566+0000","last_clean":"2024-03-30T01:41:02.868566+0000","last_became_active":"2024-03-30T01:40:58.426336+0000","last_became_peered":"2024-03-30T01:40:58.426336+0000","last_unstale":"2024-03-30T01:41:02.868566+0000","last_undegraded":"2024-03-30T01:41:02.868566+0000","last_fullsized":"2024-03-30T01:41:02.868566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869813+0000","last_change":"2024-03-30T01:40:58.434632+0000","last_active":"2024-03-30T01:41:02.869813+0000","last_peered":"2024-03-30T01:41:02.869813+0000","last_clean":"2024-03-30T01:41:02.869813+0000","last_became_active":"2024-03-30T01:40:58.434339+0000","last_became_peered":"2024-03-30T01:40:58.434339+0000","last_unstale":"2024-03-30T01:41:02.869813+0000","last_undegraded":"2024-03-30T01:41:02.869813+0000","last_fullsized":"2024-03-30T01:41:02.869813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:35.665153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868405+0000","last_change":"2024-03-30T01:40:58.428659+0000","last_active":"2024-03-30T01:41:02.868405+0000","last_peered":"2024-03-30T01:41:02.868405+0000","last_clean":"2024-03-30T01:41:02.868405+0000","last_became_active":"2024-03-30T01:40:58.426016+0000","last_became_peered":"2024-03-30T01:40:58.426016+0000","last_unstale":"2024-03-30T01:41:02.868405+0000","last_undegraded":"2024-03-30T01:41:02.868405+0000","last_fullsized":"2024-03-30T01:41:02.868405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964596+0000","last_change":"2024-03-30T01:40:58.440079+0000","last_active":"2024-03-30T01:41:02.964596+0000","last_peered":"2024-03-30T01:41:02.964596+0000","last_clean":"2024-03-30T01:41:02.964596+0000","last_became_active":"2024-03-30T01:40:58.439661+0000","last_became_peered":"2024-03-30T01:40:58.439661+0000","last_unstale":"2024-03-30T01:41:02.964596+0000","last_undegraded":"2024-03-30T01:41:02.964596+0000","last_fullsized":"2024-03-30T01:41:02.964596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868344+0000","last_change":"2024-03-30T01:40:58.426692+0000","last_active":"2024-03-30T01:41:02.868344+0000","last_peered":"2024-03-30T01:41:02.868344+0000","last_clean":"2024-03-30T01:41:02.868344+0000","last_became_active":"2024-03-30T01:40:58.426370+0000","last_became_peered":"2024-03-30T01:40:58.426370+0000","last_unstale":"2024-03-30T01:41:02.868344+0000","last_undegraded":"2024-03-30T01:41:02.868344+0000","last_fullsized":"2024-03-30T01:41:02.868344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:51.791864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964482+0000","last_change":"2024-03-30T01:40:58.440389+0000","last_active":"2024-03-30T01:41:02.964482+0000","last_peered":"2024-03-30T01:41:02.964482+0000","last_clean":"2024-03-30T01:41:02.964482+0000","last_became_active":"2024-03-30T01:40:58.439965+0000","last_became_peered":"2024-03-30T01:40:58.439965+0000","last_unstale":"2024-03-30T01:41:02.964482+0000","last_undegraded":"2024-03-30T01:41:02.964482+0000","last_fullsized":"2024-03-30T01:41:02.964482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:57.923292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868477+0000","last_change":"2024-03-30T01:40:58.429571+0000","last_active":"2024-03-30T01:41:02.868477+0000","last_peered":"2024-03-30T01:41:02.868477+0000","last_clean":"2024-03-30T01:41:02.868477+0000","last_became_active":"2024-03-30T01:40:58.429144+0000","last_became_peered":"2024-03-30T01:40:58.429144+0000","last_unstale":"2024-03-30T01:41:02.868477+0000","last_undegraded":"2024-03-30T01:41:02.868477+0000","last_fullsized":"2024-03-30T01:41:02.868477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965163+0000","last_change":"2024-03-30T01:40:58.441005+0000","last_active":"2024-03-30T01:41:02.965163+0000","last_peered":"2024-03-30T01:41:02.965163+0000","last_clean":"2024-03-30T01:41:02.965163+0000","last_became_active":"2024-03-30T01:40:58.440409+0000","last_became_peered":"2024-03-30T01:40:58.440409+0000","last_unstale":"2024-03-30T01:41:02.965163+0000","last_undegraded":"2024-03-30T01:41:02.965163+0000","last_fullsized":"2024-03-30T01:41:02.965163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:48:57.608199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872597+0000","last_change":"2024-03-30T01:40:58.429626+0000","last_active":"2024-03-30T01:41:02.872597+0000","last_peered":"2024-03-30T01:41:02.872597+0000","last_clean":"2024-03-30T01:41:02.872597+0000","last_became_active":"2024-03-30T01:40:58.428916+0000","last_became_peered":"2024-03-30T01:40:58.428916+0000","last_unstale":"2024-03-30T01:41:02.872597+0000","last_undegraded":"2024-03-30T01:41:02.872597+0000","last_fullsized":"2024-03-30T01:41:02.872597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872572+0000","last_change":"2024-03-30T01:40:58.430546+0000","last_active":"2024-03-30T01:41:02.872572+0000","last_peered":"2024-03-30T01:41:02.872572+0000","last_clean":"2024-03-30T01:41:02.872572+0000","last_became_active":"2024-03-30T01:40:58.430232+0000","last_became_peered":"2024-03-30T01:40:58.430232+0000","last_unstale":"2024-03-30T01:41:02.872572+0000","last_undegraded":"2024-03-30T01:41:02.872572+0000","last_fullsized":"2024-03-30T01:41:02.872572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864576+0000","last_change":"2024-03-30T01:40:58.422212+0000","last_active":"2024-03-30T01:41:02.864576+0000","last_peered":"2024-03-30T01:41:02.864576+0000","last_clean":"2024-03-30T01:41:02.864576+0000","last_became_active":"2024-03-30T01:40:58.421812+0000","last_became_peered":"2024-03-30T01:40:58.421812+0000","last_unstale":"2024-03-30T01:41:02.864576+0000","last_undegraded":"2024-03-30T01:41:02.864576+0000","last_fullsized":"2024-03-30T01:41:02.864576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:30.774569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868720+0000","last_change":"2024-03-30T01:40:58.418447+0000","last_active":"2024-03-30T01:41:02.868720+0000","last_peered":"2024-03-30T01:41:02.868720+0000","last_clean":"2024-03-30T01:41:02.868720+0000","last_became_active":"2024-03-30T01:40:58.418146+0000","last_became_peered":"2024-03-30T01:40:58.418146+0000","last_unstale":"2024-03-30T01:41:02.868720+0000","last_undegraded":"2024-03-30T01:41:02.868720+0000","last_fullsized":"2024-03-30T01:41:02.868720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869653+0000","last_change":"2024-03-30T01:40:58.430002+0000","last_active":"2024-03-30T01:41:02.869653+0000","last_peered":"2024-03-30T01:41:02.869653+0000","last_clean":"2024-03-30T01:41:02.869653+0000","last_became_active":"2024-03-30T01:40:58.429743+0000","last_became_peered":"2024-03-30T01:40:58.429743+0000","last_unstale":"2024-03-30T01:41:02.869653+0000","last_undegraded":"2024-03-30T01:41:02.869653+0000","last_fullsized":"2024-03-30T01:41:02.869653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:54.449729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868389+0000","last_change":"2024-03-30T01:40:58.425393+0000","last_active":"2024-03-30T01:41:02.868389+0000","last_peered":"2024-03-30T01:41:02.868389+0000","last_clean":"2024-03-30T01:41:02.868389+0000","last_became_active":"2024-03-30T01:40:58.424890+0000","last_became_peered":"2024-03-30T01:40:58.424890+0000","last_unstale":"2024-03-30T01:41:02.868389+0000","last_undegraded":"2024-03-30T01:41:02.868389+0000","last_fullsized":"2024-03-30T01:41:02.868389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868218+0000","last_change":"2024-03-30T01:40:58.426453+0000","last_active":"2024-03-30T01:41:02.868218+0000","last_peered":"2024-03-30T01:41:02.868218+0000","last_clean":"2024-03-30T01:41:02.868218+0000","last_became_active":"2024-03-30T01:40:58.425360+0000","last_became_peered":"2024-03-30T01:40:58.425360+0000","last_unstale":"2024-03-30T01:41:02.868218+0000","last_undegraded":"2024-03-30T01:41:02.868218+0000","last_fullsized":"2024-03-30T01:41:02.868218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:28.675589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868517+0000","last_change":"2024-03-30T01:40:58.433354+0000","last_active":"2024-03-30T01:41:02.868517+0000","last_peered":"2024-03-30T01:41:02.868517+0000","last_clean":"2024-03-30T01:41:02.868517+0000","last_became_active":"2024-03-30T01:40:58.433034+0000","last_became_peered":"2024-03-30T01:40:58.433034+0000","last_unstale":"2024-03-30T01:41:02.868517+0000","last_undegraded":"2024-03-30T01:41:02.868517+0000","last_fullsized":"2024-03-30T01:41:02.868517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:30.394391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868781+0000","last_change":"2024-03-30T01:40:58.433603+0000","last_active":"2024-03-30T01:41:02.868781+0000","last_peered":"2024-03-30T01:41:02.868781+0000","last_clean":"2024-03-30T01:41:02.868781+0000","last_became_active":"2024-03-30T01:40:58.433303+0000","last_became_peered":"2024-03-30T01:40:58.433303+0000","last_unstale":"2024-03-30T01:41:02.868781+0000","last_undegraded":"2024-03-30T01:41:02.868781+0000","last_fullsized":"2024-03-30T01:41:02.868781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:05:33.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965044+0000","last_change":"2024-03-30T01:40:58.436807+0000","last_active":"2024-03-30T01:41:02.965044+0000","last_peered":"2024-03-30T01:41:02.965044+0000","last_clean":"2024-03-30T01:41:02.965044+0000","last_became_active":"2024-03-30T01:40:58.436528+0000","last_became_peered":"2024-03-30T01:40:58.436528+0000","last_unstale":"2024-03-30T01:41:02.965044+0000","last_undegraded":"2024-03-30T01:41:02.965044+0000","last_fullsized":"2024-03-30T01:41:02.965044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:21.062997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972825+0000","last_change":"2024-03-30T01:41:02.970253+0000","last_active":"2024-03-30T01:41:02.972825+0000","last_peered":"2024-03-30T01:41:02.972825+0000","last_clean":"2024-03-30T01:41:02.972825+0000","last_became_active":"2024-03-30T01:41:02.969854+0000","last_became_peered":"2024-03-30T01:41:02.969854+0000","last_unstale":"2024-03-30T01:41:02.972825+0000","last_undegraded":"2024-03-30T01:41:02.972825+0000","last_fullsized":"2024-03-30T01:41:02.972825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:57:18.590955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.786878+0000","last_change":"2024-03-30T01:40:57.963998+0000","last_active":"2024-03-30T01:41:02.786878+0000","last_peered":"2024-03-30T01:41:02.786878+0000","last_clean":"2024-03-30T01:41:02.786878+0000","last_became_active":"2024-03-30T01:40:57.963535+0000","last_became_peered":"2024-03-30T01:40:57.963535+0000","last_unstale":"2024-03-30T01:41:02.786878+0000","last_undegraded":"2024-03-30T01:41:02.786878+0000","last_fullsized":"2024-03-30T01:41:02.786878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:40:49.324287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868907+0000","last_change":"2024-03-30T01:40:58.418336+0000","last_active":"2024-03-30T01:41:02.868907+0000","last_peered":"2024-03-30T01:41:02.868907+0000","last_clean":"2024-03-30T01:41:02.868907+0000","last_became_active":"2024-03-30T01:40:58.418057+0000","last_became_peered":"2024-03-30T01:40:58.418057+0000","last_unstale":"2024-03-30T01:41:02.868907+0000","last_undegraded":"2024-03-30T01:41:02.868907+0000","last_fullsized":"2024-03-30T01:41:02.868907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:52.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864399+0000","last_change":"2024-03-30T01:41:02.791579+0000","last_active":"2024-03-30T01:41:02.864399+0000","last_peered":"2024-03-30T01:41:02.864399+0000","last_clean":"2024-03-30T01:41:02.864399+0000","last_became_active":"2024-03-30T01:41:02.791102+0000","last_became_peered":"2024-03-30T01:41:02.791102+0000","last_unstale":"2024-03-30T01:41:02.864399+0000","last_undegraded":"2024-03-30T01:41:02.864399+0000","last_fullsized":"2024-03-30T01:41:02.864399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:34.215813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.776056+0000","last_change":"2024-03-30T01:40:57.963840+0000","last_active":"2024-03-30T01:41:02.776056+0000","last_peered":"2024-03-30T01:41:02.776056+0000","last_clean":"2024-03-30T01:41:02.776056+0000","last_became_active":"2024-03-30T01:40:57.962842+0000","last_became_peered":"2024-03-30T01:40:57.962842+0000","last_unstale":"2024-03-30T01:41:02.776056+0000","last_undegraded":"2024-03-30T01:41:02.776056+0000","last_fullsized":"2024-03-30T01:41:02.776056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869931+0000","last_change":"2024-03-30T01:40:58.435910+0000","last_active":"2024-03-30T01:41:02.869931+0000","last_peered":"2024-03-30T01:41:02.869931+0000","last_clean":"2024-03-30T01:41:02.869931+0000","last_became_active":"2024-03-30T01:40:58.435402+0000","last_became_peered":"2024-03-30T01:40:58.435402+0000","last_unstale":"2024-03-30T01:41:02.869931+0000","last_undegraded":"2024-03-30T01:41:02.869931+0000","last_fullsized":"2024-03-30T01:41:02.869931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864534+0000","last_change":"2024-03-30T01:41:02.789877+0000","last_active":"2024-03-30T01:41:02.864534+0000","last_peered":"2024-03-30T01:41:02.864534+0000","last_clean":"2024-03-30T01:41:02.864534+0000","last_became_active":"2024-03-30T01:41:02.789440+0000","last_became_peered":"2024-03-30T01:41:02.789440+0000","last_unstale":"2024-03-30T01:41:02.864534+0000","last_undegraded":"2024-03-30T01:41:02.864534+0000","last_fullsized":"2024-03-30T01:41:02.864534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:23.954119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964766+0000","last_change":"2024-03-30T01:40:57.968973+0000","last_active":"2024-03-30T01:41:02.964766+0000","last_peered":"2024-03-30T01:41:02.964766+0000","last_clean":"2024-03-30T01:41:02.964766+0000","last_became_active":"2024-03-30T01:40:57.968415+0000","last_became_peered":"2024-03-30T01:40:57.968415+0000","last_unstale":"2024-03-30T01:41:02.964766+0000","last_undegraded":"2024-03-30T01:41:02.964766+0000","last_fullsized":"2024-03-30T01:41:02.964766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868378+0000","last_change":"2024-03-30T01:40:58.431466+0000","last_active":"2024-03-30T01:41:02.868378+0000","last_peered":"2024-03-30T01:41:02.868378+0000","last_clean":"2024-03-30T01:41:02.868378+0000","last_became_active":"2024-03-30T01:40:58.431122+0000","last_became_peered":"2024-03-30T01:40:58.431122+0000","last_unstale":"2024-03-30T01:41:02.868378+0000","last_undegraded":"2024-03-30T01:41:02.868378+0000","last_fullsized":"2024-03-30T01:41:02.868378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:21.062997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864511+0000","last_change":"2024-03-30T01:41:02.789299+0000","last_active":"2024-03-30T01:41:02.864511+0000","last_peered":"2024-03-30T01:41:02.864511+0000","last_clean":"2024-03-30T01:41:02.864511+0000","last_became_active":"2024-03-30T01:41:02.788769+0000","last_became_peered":"2024-03-30T01:41:02.788769+0000","last_unstale":"2024-03-30T01:41:02.864511+0000","last_undegraded":"2024-03-30T01:41:02.864511+0000","last_fullsized":"2024-03-30T01:41:02.864511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:18.933245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964688+0000","last_change":"2024-03-30T01:40:57.970239+0000","last_active":"2024-03-30T01:41:02.964688+0000","last_peered":"2024-03-30T01:41:02.964688+0000","last_clean":"2024-03-30T01:41:02.964688+0000","last_became_active":"2024-03-30T01:40:57.969868+0000","last_became_peered":"2024-03-30T01:40:57.969868+0000","last_unstale":"2024-03-30T01:41:02.964688+0000","last_undegraded":"2024-03-30T01:41:02.964688+0000","last_fullsized":"2024-03-30T01:41:02.964688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:34.662924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868462+0000","last_change":"2024-03-30T01:40:58.431864+0000","last_active":"2024-03-30T01:41:02.868462+0000","last_peered":"2024-03-30T01:41:02.868462+0000","last_clean":"2024-03-30T01:41:02.868462+0000","last_became_active":"2024-03-30T01:40:58.431101+0000","last_became_peered":"2024-03-30T01:40:58.431101+0000","last_unstale":"2024-03-30T01:41:02.868462+0000","last_undegraded":"2024-03-30T01:41:02.868462+0000","last_fullsized":"2024-03-30T01:41:02.868462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:43.041425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868886+0000","last_change":"2024-03-30T01:41:02.797411+0000","last_active":"2024-03-30T01:41:02.868886+0000","last_peered":"2024-03-30T01:41:02.868886+0000","last_clean":"2024-03-30T01:41:02.868886+0000","last_became_active":"2024-03-30T01:41:02.797018+0000","last_became_peered":"2024-03-30T01:41:02.797018+0000","last_unstale":"2024-03-30T01:41:02.868886+0000","last_undegraded":"2024-03-30T01:41:02.868886+0000","last_fullsized":"2024-03-30T01:41:02.868886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:18:03.263600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.775328+0000","last_change":"2024-03-30T01:40:57.963372+0000","last_active":"2024-03-30T01:41:02.775328+0000","last_peered":"2024-03-30T01:41:02.775328+0000","last_clean":"2024-03-30T01:41:02.775328+0000","last_became_active":"2024-03-30T01:40:57.963033+0000","last_became_peered":"2024-03-30T01:40:57.963033+0000","last_unstale":"2024-03-30T01:41:02.775328+0000","last_undegraded":"2024-03-30T01:41:02.775328+0000","last_fullsized":"2024-03-30T01:41:02.775328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864523+0000","last_change":"2024-03-30T01:40:58.427338+0000","last_active":"2024-03-30T01:41:02.864523+0000","last_peered":"2024-03-30T01:41:02.864523+0000","last_clean":"2024-03-30T01:41:02.864523+0000","last_became_active":"2024-03-30T01:40:58.427023+0000","last_became_peered":"2024-03-30T01:40:58.427023+0000","last_unstale":"2024-03-30T01:41:02.864523+0000","last_undegraded":"2024-03-30T01:41:02.864523+0000","last_fullsized":"2024-03-30T01:41:02.864523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:35.665153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868735+0000","last_change":"2024-03-30T01:41:02.801387+0000","last_active":"2024-03-30T01:41:02.868735+0000","last_peered":"2024-03-30T01:41:02.868735+0000","last_clean":"2024-03-30T01:41:02.868735+0000","last_became_active":"2024-03-30T01:41:02.800907+0000","last_became_peered":"2024-03-30T01:41:02.800907+0000","last_unstale":"2024-03-30T01:41:02.868735+0000","last_undegraded":"2024-03-30T01:41:02.868735+0000","last_fullsized":"2024-03-30T01:41:02.868735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:43.218702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770418+0000","last_change":"2024-03-30T01:40:57.965707+0000","last_active":"2024-03-30T01:41:02.770418+0000","last_peered":"2024-03-30T01:41:02.770418+0000","last_clean":"2024-03-30T01:41:02.770418+0000","last_became_active":"2024-03-30T01:40:57.965371+0000","last_became_peered":"2024-03-30T01:40:57.965371+0000","last_unstale":"2024-03-30T01:41:02.770418+0000","last_undegraded":"2024-03-30T01:41:02.770418+0000","last_fullsized":"2024-03-30T01:41:02.770418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869759+0000","last_change":"2024-03-30T01:40:58.432647+0000","last_active":"2024-03-30T01:41:02.869759+0000","last_peered":"2024-03-30T01:41:02.869759+0000","last_clean":"2024-03-30T01:41:02.869759+0000","last_became_active":"2024-03-30T01:40:58.432292+0000","last_became_peered":"2024-03-30T01:40:58.432292+0000","last_unstale":"2024-03-30T01:41:02.869759+0000","last_undegraded":"2024-03-30T01:41:02.869759+0000","last_fullsized":"2024-03-30T01:41:02.869759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872608+0000","last_change":"2024-03-30T01:41:02.798539+0000","last_active":"2024-03-30T01:41:02.872608+0000","last_peered":"2024-03-30T01:41:02.872608+0000","last_clean":"2024-03-30T01:41:02.872608+0000","last_became_active":"2024-03-30T01:41:02.798146+0000","last_became_peered":"2024-03-30T01:41:02.798146+0000","last_unstale":"2024-03-30T01:41:02.872608+0000","last_undegraded":"2024-03-30T01:41:02.872608+0000","last_fullsized":"2024-03-30T01:41:02.872608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:18.933245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.773948+0000","last_change":"2024-03-30T01:40:57.959181+0000","last_active":"2024-03-30T01:41:02.773948+0000","last_peered":"2024-03-30T01:41:02.773948+0000","last_clean":"2024-03-30T01:41:02.773948+0000","last_became_active":"2024-03-30T01:40:57.958838+0000","last_became_peered":"2024-03-30T01:40:57.958838+0000","last_unstale":"2024-03-30T01:41:02.773948+0000","last_undegraded":"2024-03-30T01:41:02.773948+0000","last_fullsized":"2024-03-30T01:41:02.773948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:04.688535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868351+0000","last_change":"2024-03-30T01:40:58.429817+0000","last_active":"2024-03-30T01:41:02.868351+0000","last_peered":"2024-03-30T01:41:02.868351+0000","last_clean":"2024-03-30T01:41:02.868351+0000","last_became_active":"2024-03-30T01:40:58.429505+0000","last_became_peered":"2024-03-30T01:40:58.429505+0000","last_unstale":"2024-03-30T01:41:02.868351+0000","last_undegraded":"2024-03-30T01:41:02.868351+0000","last_fullsized":"2024-03-30T01:41:02.868351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:30:35.170155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.970928+0000","last_change":"2024-03-30T01:41:02.967897+0000","last_active":"2024-03-30T01:41:02.970928+0000","last_peered":"2024-03-30T01:41:02.970928+0000","last_clean":"2024-03-30T01:41:02.970928+0000","last_became_active":"2024-03-30T01:41:02.967524+0000","last_became_peered":"2024-03-30T01:41:02.967524+0000","last_unstale":"2024-03-30T01:41:02.970928+0000","last_undegraded":"2024-03-30T01:41:02.970928+0000","last_fullsized":"2024-03-30T01:41:02.970928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:58.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.778673+0000","last_change":"2024-03-30T01:40:57.964382+0000","last_active":"2024-03-30T01:41:02.778673+0000","last_peered":"2024-03-30T01:41:02.778673+0000","last_clean":"2024-03-30T01:41:02.778673+0000","last_became_active":"2024-03-30T01:40:57.963879+0000","last_became_peered":"2024-03-30T01:40:57.963879+0000","last_unstale":"2024-03-30T01:41:02.778673+0000","last_undegraded":"2024-03-30T01:41:02.778673+0000","last_fullsized":"2024-03-30T01:41:02.778673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868435+0000","last_change":"2024-03-30T01:40:58.425195+0000","last_active":"2024-03-30T01:41:02.868435+0000","last_peered":"2024-03-30T01:41:02.868435+0000","last_clean":"2024-03-30T01:41:02.868435+0000","last_became_active":"2024-03-30T01:40:58.424555+0000","last_became_peered":"2024-03-30T01:40:58.424555+0000","last_unstale":"2024-03-30T01:41:02.868435+0000","last_undegraded":"2024-03-30T01:41:02.868435+0000","last_fullsized":"2024-03-30T01:41:02.868435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972778+0000","last_change":"2024-03-30T01:41:02.968243+0000","last_active":"2024-03-30T01:41:02.972778+0000","last_peered":"2024-03-30T01:41:02.972778+0000","last_clean":"2024-03-30T01:41:02.972778+0000","last_became_active":"2024-03-30T01:41:02.967682+0000","last_became_peered":"2024-03-30T01:41:02.967682+0000","last_unstale":"2024-03-30T01:41:02.972778+0000","last_undegraded":"2024-03-30T01:41:02.972778+0000","last_fullsized":"2024-03-30T01:41:02.972778+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:40.634879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770699+0000","last_change":"2024-03-30T01:40:57.959485+0000","last_active":"2024-03-30T01:41:02.770699+0000","last_peered":"2024-03-30T01:41:02.770699+0000","last_clean":"2024-03-30T01:41:02.770699+0000","last_became_active":"2024-03-30T01:40:57.959180+0000","last_became_peered":"2024-03-30T01:40:57.959180+0000","last_unstale":"2024-03-30T01:41:02.770699+0000","last_undegraded":"2024-03-30T01:41:02.770699+0000","last_fullsized":"2024-03-30T01:41:02.770699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868505+0000","last_change":"2024-03-30T01:40:58.432524+0000","last_active":"2024-03-30T01:41:02.868505+0000","last_peered":"2024-03-30T01:41:02.868505+0000","last_clean":"2024-03-30T01:41:02.868505+0000","last_became_active":"2024-03-30T01:40:58.432225+0000","last_became_peered":"2024-03-30T01:40:58.432225+0000","last_unstale":"2024-03-30T01:41:02.868505+0000","last_undegraded":"2024-03-30T01:41:02.868505+0000","last_fullsized":"2024-03-30T01:41:02.868505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:30:35.170155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868329+0000","last_change":"2024-03-30T01:41:02.796380+0000","last_active":"2024-03-30T01:41:02.868329+0000","last_peered":"2024-03-30T01:41:02.868329+0000","last_clean":"2024-03-30T01:41:02.868329+0000","last_became_active":"2024-03-30T01:41:02.795984+0000","last_became_peered":"2024-03-30T01:41:02.795984+0000","last_unstale":"2024-03-30T01:41:02.868329+0000","last_undegraded":"2024-03-30T01:41:02.868329+0000","last_fullsized":"2024-03-30T01:41:02.868329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:13.399650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.781391+0000","last_change":"2024-03-30T01:40:57.958458+0000","last_active":"2024-03-30T01:41:02.781391+0000","last_peered":"2024-03-30T01:41:02.781391+0000","last_clean":"2024-03-30T01:41:02.781391+0000","last_became_active":"2024-03-30T01:40:57.957887+0000","last_became_peered":"2024-03-30T01:40:57.957887+0000","last_unstale":"2024-03-30T01:41:02.781391+0000","last_undegraded":"2024-03-30T01:41:02.781391+0000","last_fullsized":"2024-03-30T01:41:02.781391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869948+0000","last_change":"2024-03-30T01:40:58.436050+0000","last_active":"2024-03-30T01:41:02.869948+0000","last_peered":"2024-03-30T01:41:02.869948+0000","last_clean":"2024-03-30T01:41:02.869948+0000","last_became_active":"2024-03-30T01:40:58.435014+0000","last_became_peered":"2024-03-30T01:40:58.435014+0000","last_unstale":"2024-03-30T01:41:02.869948+0000","last_undegraded":"2024-03-30T01:41:02.869948+0000","last_fullsized":"2024-03-30T01:41:02.869948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:52.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864562+0000","last_change":"2024-03-30T01:41:02.784245+0000","last_active":"2024-03-30T01:41:02.864562+0000","last_peered":"2024-03-30T01:41:02.864562+0000","last_clean":"2024-03-30T01:41:02.864562+0000","last_became_active":"2024-03-30T01:41:02.783538+0000","last_became_peered":"2024-03-30T01:41:02.783538+0000","last_unstale":"2024-03-30T01:41:02.864562+0000","last_undegraded":"2024-03-30T01:41:02.864562+0000","last_fullsized":"2024-03-30T01:41:02.864562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:58.270483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.777921+0000","last_change":"2024-03-30T01:40:57.967602+0000","last_active":"2024-03-30T01:41:02.777921+0000","last_peered":"2024-03-30T01:41:02.777921+0000","last_clean":"2024-03-30T01:41:02.777921+0000","last_became_active":"2024-03-30T01:40:57.967239+0000","last_became_peered":"2024-03-30T01:40:57.967239+0000","last_unstale":"2024-03-30T01:41:02.777921+0000","last_undegraded":"2024-03-30T01:41:02.777921+0000","last_fullsized":"2024-03-30T01:41:02.777921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:36.274096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864600+0000","last_change":"2024-03-30T01:40:58.430901+0000","last_active":"2024-03-30T01:41:02.864600+0000","last_peered":"2024-03-30T01:41:02.864600+0000","last_clean":"2024-03-30T01:41:02.864600+0000","last_became_active":"2024-03-30T01:40:58.430595+0000","last_became_peered":"2024-03-30T01:40:58.430595+0000","last_unstale":"2024-03-30T01:41:02.864600+0000","last_undegraded":"2024-03-30T01:41:02.864600+0000","last_fullsized":"2024-03-30T01:41:02.864600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:37.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868468+0000","last_change":"2024-03-30T01:41:02.795440+0000","last_active":"2024-03-30T01:41:02.868468+0000","last_peered":"2024-03-30T01:41:02.868468+0000","last_clean":"2024-03-30T01:41:02.868468+0000","last_became_active":"2024-03-30T01:41:02.795052+0000","last_became_peered":"2024-03-30T01:41:02.795052+0000","last_unstale":"2024-03-30T01:41:02.868468+0000","last_undegraded":"2024-03-30T01:41:02.868468+0000","last_fullsized":"2024-03-30T01:41:02.868468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:23.954119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T01:41:07.910808+0000","last_change":"2024-03-30T01:40:57.966563+0000","last_active":"2024-03-30T01:41:07.910808+0000","last_peered":"2024-03-30T01:41:07.910808+0000","last_clean":"2024-03-30T01:41:07.910808+0000","last_became_active":"2024-03-30T01:40:57.966235+0000","last_became_peered":"2024-03-30T01:40:57.966235+0000","last_unstale":"2024-03-30T01:41:07.910808+0000","last_undegraded":"2024-03-30T01:41:07.910808+0000","last_fullsized":"2024-03-30T01:41:07.910808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3655,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964895+0000","last_change":"2024-03-30T01:40:58.436063+0000","last_active":"2024-03-30T01:41:02.964895+0000","last_peered":"2024-03-30T01:41:02.964895+0000","last_clean":"2024-03-30T01:41:02.964895+0000","last_became_active":"2024-03-30T01:40:58.435716+0000","last_became_peered":"2024-03-30T01:40:58.435716+0000","last_unstale":"2024-03-30T01:41:02.964895+0000","last_undegraded":"2024-03-30T01:41:02.964895+0000","last_fullsized":"2024-03-30T01:41:02.964895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:30.394391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.973943+0000","last_change":"2024-03-30T01:41:02.969718+0000","last_active":"2024-03-30T01:41:02.973943+0000","last_peered":"2024-03-30T01:41:02.973943+0000","last_clean":"2024-03-30T01:41:02.973943+0000","last_became_active":"2024-03-30T01:41:02.969246+0000","last_became_peered":"2024-03-30T01:41:02.969246+0000","last_unstale":"2024-03-30T01:41:02.973943+0000","last_undegraded":"2024-03-30T01:41:02.973943+0000","last_fullsized":"2024-03-30T01:41:02.973943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:34.173820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.786047+0000","last_change":"2024-03-30T01:40:57.971645+0000","last_active":"2024-03-30T01:41:02.786047+0000","last_peered":"2024-03-30T01:41:02.786047+0000","last_clean":"2024-03-30T01:41:02.786047+0000","last_became_active":"2024-03-30T01:40:57.971274+0000","last_became_peered":"2024-03-30T01:40:57.971274+0000","last_unstale":"2024-03-30T01:41:02.786047+0000","last_undegraded":"2024-03-30T01:41:02.786047+0000","last_fullsized":"2024-03-30T01:41:02.786047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:09.583893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869665+0000","last_change":"2024-03-30T01:40:58.431398+0000","last_active":"2024-03-30T01:41:02.869665+0000","last_peered":"2024-03-30T01:41:02.869665+0000","last_clean":"2024-03-30T01:41:02.869665+0000","last_became_active":"2024-03-30T01:40:58.431120+0000","last_became_peered":"2024-03-30T01:40:58.431120+0000","last_unstale":"2024-03-30T01:41:02.869665+0000","last_undegraded":"2024-03-30T01:41:02.869665+0000","last_fullsized":"2024-03-30T01:41:02.869665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968671+0000","last_change":"2024-03-30T01:41:02.966425+0000","last_active":"2024-03-30T01:41:02.968671+0000","last_peered":"2024-03-30T01:41:02.968671+0000","last_clean":"2024-03-30T01:41:02.968671+0000","last_became_active":"2024-03-30T01:41:02.965978+0000","last_became_peered":"2024-03-30T01:41:02.965978+0000","last_unstale":"2024-03-30T01:41:02.968671+0000","last_undegraded":"2024-03-30T01:41:02.968671+0000","last_fullsized":"2024-03-30T01:41:02.968671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780686+0000","last_change":"2024-03-30T01:40:57.956691+0000","last_active":"2024-03-30T01:41:02.780686+0000","last_peered":"2024-03-30T01:41:02.780686+0000","last_clean":"2024-03-30T01:41:02.780686+0000","last_became_active":"2024-03-30T01:40:57.956387+0000","last_became_peered":"2024-03-30T01:40:57.956387+0000","last_unstale":"2024-03-30T01:41:02.780686+0000","last_undegraded":"2024-03-30T01:41:02.780686+0000","last_fullsized":"2024-03-30T01:41:02.780686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964728+0000","last_change":"2024-03-30T01:40:58.435147+0000","last_active":"2024-03-30T01:41:02.964728+0000","last_peered":"2024-03-30T01:41:02.964728+0000","last_clean":"2024-03-30T01:41:02.964728+0000","last_became_active":"2024-03-30T01:40:58.434835+0000","last_became_peered":"2024-03-30T01:40:58.434835+0000","last_unstale":"2024-03-30T01:41:02.964728+0000","last_undegraded":"2024-03-30T01:41:02.964728+0000","last_fullsized":"2024-03-30T01:41:02.964728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:43.041425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972786+0000","last_change":"2024-03-30T01:41:02.968714+0000","last_active":"2024-03-30T01:41:02.972786+0000","last_peered":"2024-03-30T01:41:02.972786+0000","last_clean":"2024-03-30T01:41:02.972786+0000","last_became_active":"2024-03-30T01:41:02.967728+0000","last_became_peered":"2024-03-30T01:41:02.967728+0000","last_unstale":"2024-03-30T01:41:02.972786+0000","last_undegraded":"2024-03-30T01:41:02.972786+0000","last_fullsized":"2024-03-30T01:41:02.972786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:57:35.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770059+0000","last_change":"2024-03-30T01:40:57.964975+0000","last_active":"2024-03-30T01:41:02.770059+0000","last_peered":"2024-03-30T01:41:02.770059+0000","last_clean":"2024-03-30T01:41:02.770059+0000","last_became_active":"2024-03-30T01:40:57.964458+0000","last_became_peered":"2024-03-30T01:40:57.964458+0000","last_unstale":"2024-03-30T01:41:02.770059+0000","last_undegraded":"2024-03-30T01:41:02.770059+0000","last_fullsized":"2024-03-30T01:41:02.770059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872388+0000","last_change":"2024-03-30T01:40:58.425912+0000","last_active":"2024-03-30T01:41:02.872388+0000","last_peered":"2024-03-30T01:41:02.872388+0000","last_clean":"2024-03-30T01:41:02.872388+0000","last_became_active":"2024-03-30T01:40:58.425596+0000","last_became_peered":"2024-03-30T01:40:58.425596+0000","last_unstale":"2024-03-30T01:41:02.872388+0000","last_undegraded":"2024-03-30T01:41:02.872388+0000","last_fullsized":"2024-03-30T01:41:02.872388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869825+0000","last_change":"2024-03-30T01:41:02.793602+0000","last_active":"2024-03-30T01:41:02.869825+0000","last_peered":"2024-03-30T01:41:02.869825+0000","last_clean":"2024-03-30T01:41:02.869825+0000","last_became_active":"2024-03-30T01:41:02.793146+0000","last_became_peered":"2024-03-30T01:41:02.793146+0000","last_unstale":"2024-03-30T01:41:02.869825+0000","last_undegraded":"2024-03-30T01:41:02.869825+0000","last_fullsized":"2024-03-30T01:41:02.869825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782115+0000","last_change":"2024-03-30T01:40:57.962830+0000","last_active":"2024-03-30T01:41:02.782115+0000","last_peered":"2024-03-30T01:41:02.782115+0000","last_clean":"2024-03-30T01:41:02.782115+0000","last_became_active":"2024-03-30T01:40:57.961073+0000","last_became_peered":"2024-03-30T01:40:57.961073+0000","last_unstale":"2024-03-30T01:41:02.782115+0000","last_undegraded":"2024-03-30T01:41:02.782115+0000","last_fullsized":"2024-03-30T01:41:02.782115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964632+0000","last_change":"2024-03-30T01:40:58.435030+0000","last_active":"2024-03-30T01:41:02.964632+0000","last_peered":"2024-03-30T01:41:02.964632+0000","last_clean":"2024-03-30T01:41:02.964632+0000","last_became_active":"2024-03-30T01:40:58.434660+0000","last_became_peered":"2024-03-30T01:40:58.434660+0000","last_unstale":"2024-03-30T01:41:02.964632+0000","last_undegraded":"2024-03-30T01:41:02.964632+0000","last_fullsized":"2024-03-30T01:41:02.964632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:37.385758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864431+0000","last_change":"2024-03-30T01:41:02.796839+0000","last_active":"2024-03-30T01:41:02.864431+0000","last_peered":"2024-03-30T01:41:02.864431+0000","last_clean":"2024-03-30T01:41:02.864431+0000","last_became_active":"2024-03-30T01:41:02.796400+0000","last_became_peered":"2024-03-30T01:41:02.796400+0000","last_unstale":"2024-03-30T01:41:02.864431+0000","last_undegraded":"2024-03-30T01:41:02.864431+0000","last_fullsized":"2024-03-30T01:41:02.864431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:06:05.412311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.775105+0000","last_change":"2024-03-30T01:40:57.963776+0000","last_active":"2024-03-30T01:41:02.775105+0000","last_peered":"2024-03-30T01:41:02.775105+0000","last_clean":"2024-03-30T01:41:02.775105+0000","last_became_active":"2024-03-30T01:40:57.963473+0000","last_became_peered":"2024-03-30T01:40:57.963473+0000","last_unstale":"2024-03-30T01:41:02.775105+0000","last_undegraded":"2024-03-30T01:41:02.775105+0000","last_fullsized":"2024-03-30T01:41:02.775105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:09.583893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964956+0000","last_change":"2024-03-30T01:40:58.440136+0000","last_active":"2024-03-30T01:41:02.964956+0000","last_peered":"2024-03-30T01:41:02.964956+0000","last_clean":"2024-03-30T01:41:02.964956+0000","last_became_active":"2024-03-30T01:40:58.439831+0000","last_became_peered":"2024-03-30T01:40:58.439831+0000","last_unstale":"2024-03-30T01:41:02.964956+0000","last_undegraded":"2024-03-30T01:41:02.964956+0000","last_fullsized":"2024-03-30T01:41:02.964956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:28.675589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868455+0000","last_change":"2024-03-30T01:41:02.790100+0000","last_active":"2024-03-30T01:41:02.868455+0000","last_peered":"2024-03-30T01:41:02.868455+0000","last_clean":"2024-03-30T01:41:02.868455+0000","last_became_active":"2024-03-30T01:41:02.789660+0000","last_became_peered":"2024-03-30T01:41:02.789660+0000","last_unstale":"2024-03-30T01:41:02.868455+0000","last_undegraded":"2024-03-30T01:41:02.868455+0000","last_fullsized":"2024-03-30T01:41:02.868455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:22:34.848382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774905+0000","last_change":"2024-03-30T01:40:57.962434+0000","last_active":"2024-03-30T01:41:02.774905+0000","last_peered":"2024-03-30T01:41:02.774905+0000","last_clean":"2024-03-30T01:41:02.774905+0000","last_became_active":"2024-03-30T01:40:57.961987+0000","last_became_peered":"2024-03-30T01:40:57.961987+0000","last_unstale":"2024-03-30T01:41:02.774905+0000","last_undegraded":"2024-03-30T01:41:02.774905+0000","last_fullsized":"2024-03-30T01:41:02.774905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:42.056286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868476+0000","last_change":"2024-03-30T01:40:58.419381+0000","last_active":"2024-03-30T01:41:02.868476+0000","last_peered":"2024-03-30T01:41:02.868476+0000","last_clean":"2024-03-30T01:41:02.868476+0000","last_became_active":"2024-03-30T01:40:58.419079+0000","last_became_peered":"2024-03-30T01:40:58.419079+0000","last_unstale":"2024-03-30T01:41:02.868476+0000","last_undegraded":"2024-03-30T01:41:02.868476+0000","last_fullsized":"2024-03-30T01:41:02.868476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868479+0000","last_change":"2024-03-30T01:41:02.795347+0000","last_active":"2024-03-30T01:41:02.868479+0000","last_peered":"2024-03-30T01:41:02.868479+0000","last_clean":"2024-03-30T01:41:02.868479+0000","last_became_active":"2024-03-30T01:41:02.794998+0000","last_became_peered":"2024-03-30T01:41:02.794998+0000","last_unstale":"2024-03-30T01:41:02.868479+0000","last_undegraded":"2024-03-30T01:41:02.868479+0000","last_fullsized":"2024-03-30T01:41:02.868479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:19.384530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780572+0000","last_change":"2024-03-30T01:40:57.967506+0000","last_active":"2024-03-30T01:41:02.780572+0000","last_peered":"2024-03-30T01:41:02.780572+0000","last_clean":"2024-03-30T01:41:02.780572+0000","last_became_active":"2024-03-30T01:40:57.967219+0000","last_became_peered":"2024-03-30T01:40:57.967219+0000","last_unstale":"2024-03-30T01:41:02.780572+0000","last_undegraded":"2024-03-30T01:41:02.780572+0000","last_fullsized":"2024-03-30T01:41:02.780572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864547+0000","last_change":"2024-03-30T01:40:58.434945+0000","last_active":"2024-03-30T01:41:02.864547+0000","last_peered":"2024-03-30T01:41:02.864547+0000","last_clean":"2024-03-30T01:41:02.864547+0000","last_became_active":"2024-03-30T01:40:58.434344+0000","last_became_peered":"2024-03-30T01:40:58.434344+0000","last_unstale":"2024-03-30T01:41:02.864547+0000","last_undegraded":"2024-03-30T01:41:02.864547+0000","last_fullsized":"2024-03-30T01:41:02.864547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:43.058516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868577+0000","last_change":"2024-03-30T01:41:02.796689+0000","last_active":"2024-03-30T01:41:02.868577+0000","last_peered":"2024-03-30T01:41:02.868577+0000","last_clean":"2024-03-30T01:41:02.868577+0000","last_became_active":"2024-03-30T01:41:02.796285+0000","last_became_peered":"2024-03-30T01:41:02.796285+0000","last_unstale":"2024-03-30T01:41:02.868577+0000","last_undegraded":"2024-03-30T01:41:02.868577+0000","last_fullsized":"2024-03-30T01:41:02.868577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868364+0000","last_change":"2024-03-30T01:40:58.426059+0000","last_active":"2024-03-30T01:41:02.868364+0000","last_peered":"2024-03-30T01:41:02.868364+0000","last_clean":"2024-03-30T01:41:02.868364+0000","last_became_active":"2024-03-30T01:40:58.424722+0000","last_became_peered":"2024-03-30T01:40:58.424722+0000","last_unstale":"2024-03-30T01:41:02.868364+0000","last_undegraded":"2024-03-30T01:41:02.868364+0000","last_fullsized":"2024-03-30T01:41:02.868364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782192+0000","last_change":"2024-03-30T01:40:57.964575+0000","last_active":"2024-03-30T01:41:02.782192+0000","last_peered":"2024-03-30T01:41:02.782192+0000","last_clean":"2024-03-30T01:41:02.782192+0000","last_became_active":"2024-03-30T01:40:57.962016+0000","last_became_peered":"2024-03-30T01:40:57.962016+0000","last_unstale":"2024-03-30T01:41:02.782192+0000","last_undegraded":"2024-03-30T01:41:02.782192+0000","last_fullsized":"2024-03-30T01:41:02.782192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:04.688535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.974026+0000","last_change":"2024-03-30T01:41:02.969088+0000","last_active":"2024-03-30T01:41:02.974026+0000","last_peered":"2024-03-30T01:41:02.974026+0000","last_clean":"2024-03-30T01:41:02.974026+0000","last_became_active":"2024-03-30T01:41:02.968624+0000","last_became_peered":"2024-03-30T01:41:02.968624+0000","last_unstale":"2024-03-30T01:41:02.974026+0000","last_undegraded":"2024-03-30T01:41:02.974026+0000","last_fullsized":"2024-03-30T01:41:02.974026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:34.265123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868700+0000","last_change":"2024-03-30T01:40:58.429787+0000","last_active":"2024-03-30T01:41:02.868700+0000","last_peered":"2024-03-30T01:41:02.868700+0000","last_clean":"2024-03-30T01:41:02.868700+0000","last_became_active":"2024-03-30T01:40:58.429037+0000","last_became_peered":"2024-03-30T01:40:58.429037+0000","last_unstale":"2024-03-30T01:41:02.868700+0000","last_undegraded":"2024-03-30T01:41:02.868700+0000","last_fullsized":"2024-03-30T01:41:02.868700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:54.449729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780784+0000","last_change":"2024-03-30T01:40:57.954275+0000","last_active":"2024-03-30T01:41:02.780784+0000","last_peered":"2024-03-30T01:41:02.780784+0000","last_clean":"2024-03-30T01:41:02.780784+0000","last_became_active":"2024-03-30T01:40:57.953987+0000","last_became_peered":"2024-03-30T01:40:57.953987+0000","last_unstale":"2024-03-30T01:41:02.780784+0000","last_undegraded":"2024-03-30T01:41:02.780784+0000","last_fullsized":"2024-03-30T01:41:02.780784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964197+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:41:02.964197+0000","last_peered":"2024-03-30T01:41:02.964197+0000","last_clean":"2024-03-30T01:41:02.964197+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:41:02.964197+0000","last_undegraded":"2024-03-30T01:41:02.964197+0000","last_fullsized":"2024-03-30T01:41:02.964197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972468+0000","last_change":"2024-03-30T01:41:02.969064+0000","last_active":"2024-03-30T01:41:02.972468+0000","last_peered":"2024-03-30T01:41:02.972468+0000","last_clean":"2024-03-30T01:41:02.972468+0000","last_became_active":"2024-03-30T01:41:02.968333+0000","last_became_peered":"2024-03-30T01:41:02.968333+0000","last_unstale":"2024-03-30T01:41:02.972468+0000","last_undegraded":"2024-03-30T01:41:02.972468+0000","last_fullsized":"2024-03-30T01:41:02.972468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:34.173820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869624+0000","last_change":"2024-03-30T01:40:58.432432+0000","last_active":"2024-03-30T01:41:02.869624+0000","last_peered":"2024-03-30T01:41:02.869624+0000","last_clean":"2024-03-30T01:41:02.869624+0000","last_became_active":"2024-03-30T01:40:58.432158+0000","last_became_peered":"2024-03-30T01:40:58.432158+0000","last_unstale":"2024-03-30T01:41:02.869624+0000","last_undegraded":"2024-03-30T01:41:02.869624+0000","last_fullsized":"2024-03-30T01:41:02.869624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964478+0000","last_change":"2024-03-30T01:40:57.966931+0000","last_active":"2024-03-30T01:41:02.964478+0000","last_peered":"2024-03-30T01:41:02.964478+0000","last_clean":"2024-03-30T01:41:02.964478+0000","last_became_active":"2024-03-30T01:40:57.966589+0000","last_became_peered":"2024-03-30T01:40:57.966589+0000","last_unstale":"2024-03-30T01:41:02.964478+0000","last_undegraded":"2024-03-30T01:41:02.964478+0000","last_fullsized":"2024-03-30T01:41:02.964478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:15.934995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968515+0000","last_change":"2024-03-30T01:41:02.966273+0000","last_active":"2024-03-30T01:41:02.968515+0000","last_peered":"2024-03-30T01:41:02.968515+0000","last_clean":"2024-03-30T01:41:02.968515+0000","last_became_active":"2024-03-30T01:41:02.965857+0000","last_became_peered":"2024-03-30T01:41:02.965857+0000","last_unstale":"2024-03-30T01:41:02.968515+0000","last_undegraded":"2024-03-30T01:41:02.968515+0000","last_fullsized":"2024-03-30T01:41:02.968515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:58.270483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964736+0000","last_change":"2024-03-30T01:40:57.964957+0000","last_active":"2024-03-30T01:41:02.964736+0000","last_peered":"2024-03-30T01:41:02.964736+0000","last_clean":"2024-03-30T01:41:02.964736+0000","last_became_active":"2024-03-30T01:40:57.963985+0000","last_became_peered":"2024-03-30T01:40:57.963985+0000","last_unstale":"2024-03-30T01:41:02.964736+0000","last_undegraded":"2024-03-30T01:41:02.964736+0000","last_fullsized":"2024-03-30T01:41:02.964736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:05:32.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868496+0000","last_change":"2024-03-30T01:40:58.425295+0000","last_active":"2024-03-30T01:41:02.868496+0000","last_peered":"2024-03-30T01:41:02.868496+0000","last_clean":"2024-03-30T01:41:02.868496+0000","last_became_active":"2024-03-30T01:40:58.424967+0000","last_became_peered":"2024-03-30T01:40:58.424967+0000","last_unstale":"2024-03-30T01:41:02.868496+0000","last_undegraded":"2024-03-30T01:41:02.868496+0000","last_fullsized":"2024-03-30T01:41:02.868496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.975320+0000","last_change":"2024-03-30T01:41:02.970612+0000","last_active":"2024-03-30T01:41:02.975320+0000","last_peered":"2024-03-30T01:41:02.975320+0000","last_clean":"2024-03-30T01:41:02.975320+0000","last_became_active":"2024-03-30T01:41:02.969823+0000","last_became_peered":"2024-03-30T01:41:02.969823+0000","last_unstale":"2024-03-30T01:41:02.975320+0000","last_undegraded":"2024-03-30T01:41:02.975320+0000","last_fullsized":"2024-03-30T01:41:02.975320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:25:35.463484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T01:41:03.568727+0000","last_change":"2024-03-30T01:40:57.968553+0000","last_active":"2024-03-30T01:41:03.568727+0000","last_peered":"2024-03-30T01:41:03.568727+0000","last_clean":"2024-03-30T01:41:03.568727+0000","last_became_active":"2024-03-30T01:40:57.968315+0000","last_became_peered":"2024-03-30T01:40:57.968315+0000","last_unstale":"2024-03-30T01:41:03.568727+0000","last_undegraded":"2024-03-30T01:41:03.568727+0000","last_fullsized":"2024-03-30T01:41:03.568727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:34.662924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869030+0000","last_change":"2024-03-30T01:40:58.429594+0000","last_active":"2024-03-30T01:41:02.869030+0000","last_peered":"2024-03-30T01:41:02.869030+0000","last_clean":"2024-03-30T01:41:02.869030+0000","last_became_active":"2024-03-30T01:40:58.429296+0000","last_became_peered":"2024-03-30T01:40:58.429296+0000","last_unstale":"2024-03-30T01:41:02.869030+0000","last_undegraded":"2024-03-30T01:41:02.869030+0000","last_fullsized":"2024-03-30T01:41:02.869030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.974419+0000","last_change":"2024-03-30T01:41:02.970239+0000","last_active":"2024-03-30T01:41:02.974419+0000","last_peered":"2024-03-30T01:41:02.974419+0000","last_clean":"2024-03-30T01:41:02.974419+0000","last_became_active":"2024-03-30T01:41:02.969823+0000","last_became_peered":"2024-03-30T01:41:02.969823+0000","last_unstale":"2024-03-30T01:41:02.974419+0000","last_undegraded":"2024-03-30T01:41:02.974419+0000","last_fullsized":"2024-03-30T01:41:02.974419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:00.285894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770593+0000","last_change":"2024-03-30T01:40:57.963709+0000","last_active":"2024-03-30T01:41:02.770593+0000","last_peered":"2024-03-30T01:41:02.770593+0000","last_clean":"2024-03-30T01:41:02.770593+0000","last_became_active":"2024-03-30T01:40:57.963374+0000","last_became_peered":"2024-03-30T01:40:57.963374+0000","last_unstale":"2024-03-30T01:41:02.770593+0000","last_undegraded":"2024-03-30T01:41:02.770593+0000","last_fullsized":"2024-03-30T01:41:02.770593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869920+0000","last_change":"2024-03-30T01:40:58.430636+0000","last_active":"2024-03-30T01:41:02.869920+0000","last_peered":"2024-03-30T01:41:02.869920+0000","last_clean":"2024-03-30T01:41:02.869920+0000","last_became_active":"2024-03-30T01:40:58.430400+0000","last_became_peered":"2024-03-30T01:40:58.430400+0000","last_unstale":"2024-03-30T01:41:02.869920+0000","last_undegraded":"2024-03-30T01:41:02.869920+0000","last_fullsized":"2024-03-30T01:41:02.869920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:44:58.613670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864624+0000","last_change":"2024-03-30T01:41:02.788933+0000","last_active":"2024-03-30T01:41:02.864624+0000","last_peered":"2024-03-30T01:41:02.864624+0000","last_clean":"2024-03-30T01:41:02.864624+0000","last_became_active":"2024-03-30T01:41:02.788466+0000","last_became_peered":"2024-03-30T01:41:02.788466+0000","last_unstale":"2024-03-30T01:41:02.864624+0000","last_undegraded":"2024-03-30T01:41:02.864624+0000","last_fullsized":"2024-03-30T01:41:02.864624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964182+0000","last_change":"2024-03-30T01:40:57.962449+0000","last_active":"2024-03-30T01:41:02.964182+0000","last_peered":"2024-03-30T01:41:02.964182+0000","last_clean":"2024-03-30T01:41:02.964182+0000","last_became_active":"2024-03-30T01:40:57.962192+0000","last_became_peered":"2024-03-30T01:40:57.962192+0000","last_unstale":"2024-03-30T01:41:02.964182+0000","last_undegraded":"2024-03-30T01:41:02.964182+0000","last_fullsized":"2024-03-30T01:41:02.964182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:36.274096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864497+0000","last_change":"2024-03-30T01:40:58.420796+0000","last_active":"2024-03-30T01:41:02.864497+0000","last_peered":"2024-03-30T01:41:02.864497+0000","last_clean":"2024-03-30T01:41:02.864497+0000","last_became_active":"2024-03-30T01:40:58.420441+0000","last_became_peered":"2024-03-30T01:40:58.420441+0000","last_unstale":"2024-03-30T01:41:02.864497+0000","last_undegraded":"2024-03-30T01:41:02.864497+0000","last_fullsized":"2024-03-30T01:41:02.864497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:16.937225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.969938+0000","last_change":"2024-03-30T01:41:02.966462+0000","last_active":"2024-03-30T01:41:02.969938+0000","last_peered":"2024-03-30T01:41:02.969938+0000","last_clean":"2024-03-30T01:41:02.969938+0000","last_became_active":"2024-03-30T01:41:02.966056+0000","last_became_peered":"2024-03-30T01:41:02.966056+0000","last_unstale":"2024-03-30T01:41:02.969938+0000","last_undegraded":"2024-03-30T01:41:02.969938+0000","last_fullsized":"2024-03-30T01:41:02.969938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:29.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774672+0000","last_change":"2024-03-30T01:40:57.967648+0000","last_active":"2024-03-30T01:41:02.774672+0000","last_peered":"2024-03-30T01:41:02.774672+0000","last_clean":"2024-03-30T01:41:02.774672+0000","last_became_active":"2024-03-30T01:40:57.967362+0000","last_became_peered":"2024-03-30T01:40:57.967362+0000","last_unstale":"2024-03-30T01:41:02.774672+0000","last_undegraded":"2024-03-30T01:41:02.774672+0000","last_fullsized":"2024-03-30T01:41:02.774672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868291+0000","last_change":"2024-03-30T01:40:58.432296+0000","last_active":"2024-03-30T01:41:02.868291+0000","last_peered":"2024-03-30T01:41:02.868291+0000","last_clean":"2024-03-30T01:41:02.868291+0000","last_became_active":"2024-03-30T01:40:58.432045+0000","last_became_peered":"2024-03-30T01:40:58.432045+0000","last_unstale":"2024-03-30T01:41:02.868291+0000","last_undegraded":"2024-03-30T01:41:02.868291+0000","last_fullsized":"2024-03-30T01:41:02.868291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864586+0000","last_change":"2024-03-30T01:41:02.796892+0000","last_active":"2024-03-30T01:41:02.864586+0000","last_peered":"2024-03-30T01:41:02.864586+0000","last_clean":"2024-03-30T01:41:02.864586+0000","last_became_active":"2024-03-30T01:41:02.796347+0000","last_became_peered":"2024-03-30T01:41:02.796347+0000","last_unstale":"2024-03-30T01:41:02.864586+0000","last_undegraded":"2024-03-30T01:41:02.864586+0000","last_fullsized":"2024-03-30T01:41:02.864586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:20.847856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774635+0000","last_change":"2024-03-30T01:40:57.969391+0000","last_active":"2024-03-30T01:41:02.774635+0000","last_peered":"2024-03-30T01:41:02.774635+0000","last_clean":"2024-03-30T01:41:02.774635+0000","last_became_active":"2024-03-30T01:40:57.969107+0000","last_became_peered":"2024-03-30T01:40:57.969107+0000","last_unstale":"2024-03-30T01:41:02.774635+0000","last_undegraded":"2024-03-30T01:41:02.774635+0000","last_fullsized":"2024-03-30T01:41:02.774635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868769+0000","last_change":"2024-03-30T01:40:58.430179+0000","last_active":"2024-03-30T01:41:02.868769+0000","last_peered":"2024-03-30T01:41:02.868769+0000","last_clean":"2024-03-30T01:41:02.868769+0000","last_became_active":"2024-03-30T01:40:58.429919+0000","last_became_peered":"2024-03-30T01:40:58.429919+0000","last_unstale":"2024-03-30T01:41:02.868769+0000","last_undegraded":"2024-03-30T01:41:02.868769+0000","last_fullsized":"2024-03-30T01:41:02.868769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864611+0000","last_change":"2024-03-30T01:41:02.792167+0000","last_active":"2024-03-30T01:41:02.864611+0000","last_peered":"2024-03-30T01:41:02.864611+0000","last_clean":"2024-03-30T01:41:02.864611+0000","last_became_active":"2024-03-30T01:41:02.791805+0000","last_became_peered":"2024-03-30T01:41:02.791805+0000","last_unstale":"2024-03-30T01:41:02.864611+0000","last_undegraded":"2024-03-30T01:41:02.864611+0000","last_fullsized":"2024-03-30T01:41:02.864611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964766+0000","last_change":"2024-03-30T01:40:57.964733+0000","last_active":"2024-03-30T01:41:02.964766+0000","last_peered":"2024-03-30T01:41:02.964766+0000","last_clean":"2024-03-30T01:41:02.964766+0000","last_became_active":"2024-03-30T01:40:57.964430+0000","last_became_peered":"2024-03-30T01:40:57.964430+0000","last_unstale":"2024-03-30T01:41:02.964766+0000","last_undegraded":"2024-03-30T01:41:02.964766+0000","last_fullsized":"2024-03-30T01:41:02.964766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:42.056286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872480+0000","last_change":"2024-03-30T01:40:58.420811+0000","last_active":"2024-03-30T01:41:02.872480+0000","last_peered":"2024-03-30T01:41:02.872480+0000","last_clean":"2024-03-30T01:41:02.872480+0000","last_became_active":"2024-03-30T01:40:58.420491+0000","last_became_peered":"2024-03-30T01:40:58.420491+0000","last_unstale":"2024-03-30T01:41:02.872480+0000","last_undegraded":"2024-03-30T01:41:02.872480+0000","last_fullsized":"2024-03-30T01:41:02.872480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968342+0000","last_change":"2024-03-30T01:41:02.966275+0000","last_active":"2024-03-30T01:41:02.968342+0000","last_peered":"2024-03-30T01:41:02.968342+0000","last_clean":"2024-03-30T01:41:02.968342+0000","last_became_active":"2024-03-30T01:41:02.965934+0000","last_became_peered":"2024-03-30T01:41:02.965934+0000","last_unstale":"2024-03-30T01:41:02.968342+0000","last_undegraded":"2024-03-30T01:41:02.968342+0000","last_fullsized":"2024-03-30T01:41:02.968342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:39.954419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.783325+0000","last_change":"2024-03-30T01:40:57.962835+0000","last_active":"2024-03-30T01:41:02.783325+0000","last_peered":"2024-03-30T01:41:02.783325+0000","last_clean":"2024-03-30T01:41:02.783325+0000","last_became_active":"2024-03-30T01:40:57.961552+0000","last_became_peered":"2024-03-30T01:40:57.961552+0000","last_unstale":"2024-03-30T01:41:02.783325+0000","last_undegraded":"2024-03-30T01:41:02.783325+0000","last_fullsized":"2024-03-30T01:41:02.783325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964874+0000","last_change":"2024-03-30T01:40:58.432622+0000","last_active":"2024-03-30T01:41:02.964874+0000","last_peered":"2024-03-30T01:41:02.964874+0000","last_clean":"2024-03-30T01:41:02.964874+0000","last_became_active":"2024-03-30T01:40:58.432336+0000","last_became_peered":"2024-03-30T01:40:58.432336+0000","last_unstale":"2024-03-30T01:41:02.964874+0000","last_undegraded":"2024-03-30T01:41:02.964874+0000","last_fullsized":"2024-03-30T01:41:02.964874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:24:50.793081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869687+0000","last_change":"2024-03-30T01:41:02.799702+0000","last_active":"2024-03-30T01:41:02.869687+0000","last_peered":"2024-03-30T01:41:02.869687+0000","last_clean":"2024-03-30T01:41:02.869687+0000","last_became_active":"2024-03-30T01:41:02.799189+0000","last_became_peered":"2024-03-30T01:41:02.799189+0000","last_unstale":"2024-03-30T01:41:02.869687+0000","last_undegraded":"2024-03-30T01:41:02.869687+0000","last_fullsized":"2024-03-30T01:41:02.869687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.777920+0000","last_change":"2024-03-30T01:40:57.964577+0000","last_active":"2024-03-30T01:41:02.777920+0000","last_peered":"2024-03-30T01:41:02.777920+0000","last_clean":"2024-03-30T01:41:02.777920+0000","last_became_active":"2024-03-30T01:40:57.964277+0000","last_became_peered":"2024-03-30T01:40:57.964277+0000","last_unstale":"2024-03-30T01:41:02.777920+0000","last_undegraded":"2024-03-30T01:41:02.777920+0000","last_fullsized":"2024-03-30T01:41:02.777920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:15.934995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868392+0000","last_change":"2024-03-30T01:40:58.428923+0000","last_active":"2024-03-30T01:41:02.868392+0000","last_peered":"2024-03-30T01:41:02.868392+0000","last_clean":"2024-03-30T01:41:02.868392+0000","last_became_active":"2024-03-30T01:40:58.428204+0000","last_became_peered":"2024-03-30T01:40:58.428204+0000","last_unstale":"2024-03-30T01:41:02.868392+0000","last_undegraded":"2024-03-30T01:41:02.868392+0000","last_fullsized":"2024-03-30T01:41:02.868392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.970293+0000","last_change":"2024-03-30T01:41:02.968043+0000","last_active":"2024-03-30T01:41:02.970293+0000","last_peered":"2024-03-30T01:41:02.970293+0000","last_clean":"2024-03-30T01:41:02.970293+0000","last_became_active":"2024-03-30T01:41:02.967586+0000","last_became_peered":"2024-03-30T01:41:02.967586+0000","last_unstale":"2024-03-30T01:41:02.970293+0000","last_undegraded":"2024-03-30T01:41:02.970293+0000","last_fullsized":"2024-03-30T01:41:02.970293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:29.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782245+0000","last_change":"2024-03-30T01:40:57.962668+0000","last_active":"2024-03-30T01:41:02.782245+0000","last_peered":"2024-03-30T01:41:02.782245+0000","last_clean":"2024-03-30T01:41:02.782245+0000","last_became_active":"2024-03-30T01:40:57.962255+0000","last_became_peered":"2024-03-30T01:40:57.962255+0000","last_unstale":"2024-03-30T01:41:02.782245+0000","last_undegraded":"2024-03-30T01:41:02.782245+0000","last_fullsized":"2024-03-30T01:41:02.782245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965085+0000","last_change":"2024-03-30T01:40:58.441825+0000","last_active":"2024-03-30T01:41:02.965085+0000","last_peered":"2024-03-30T01:41:02.965085+0000","last_clean":"2024-03-30T01:41:02.965085+0000","last_became_active":"2024-03-30T01:40:58.441575+0000","last_became_peered":"2024-03-30T01:40:58.441575+0000","last_unstale":"2024-03-30T01:41:02.965085+0000","last_undegraded":"2024-03-30T01:41:02.965085+0000","last_fullsized":"2024-03-30T01:41:02.965085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:51.791864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872404+0000","last_change":"2024-03-30T01:40:58.425945+0000","last_active":"2024-03-30T01:41:02.872404+0000","last_peered":"2024-03-30T01:41:02.872404+0000","last_clean":"2024-03-30T01:41:02.872404+0000","last_became_active":"2024-03-30T01:40:58.425607+0000","last_became_peered":"2024-03-30T01:40:58.425607+0000","last_unstale":"2024-03-30T01:41:02.872404+0000","last_undegraded":"2024-03-30T01:41:02.872404+0000","last_fullsized":"2024-03-30T01:41:02.872404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868697+0000","last_change":"2024-03-30T01:40:58.427012+0000","last_active":"2024-03-30T01:41:02.868697+0000","last_peered":"2024-03-30T01:41:02.868697+0000","last_clean":"2024-03-30T01:41:02.868697+0000","last_became_active":"2024-03-30T01:40:58.426750+0000","last_became_peered":"2024-03-30T01:40:58.426750+0000","last_unstale":"2024-03-30T01:41:02.868697+0000","last_undegraded":"2024-03-30T01:41:02.868697+0000","last_fullsized":"2024-03-30T01:41:02.868697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:37.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868317+0000","last_change":"2024-03-30T01:40:58.426120+0000","last_active":"2024-03-30T01:41:02.868317+0000","last_peered":"2024-03-30T01:41:02.868317+0000","last_clean":"2024-03-30T01:41:02.868317+0000","last_became_active":"2024-03-30T01:40:58.425776+0000","last_became_peered":"2024-03-30T01:40:58.425776+0000","last_unstale":"2024-03-30T01:41:02.868317+0000","last_undegraded":"2024-03-30T01:41:02.868317+0000","last_fullsized":"2024-03-30T01:41:02.868317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869771+0000","last_change":"2024-03-30T01:40:58.436516+0000","last_active":"2024-03-30T01:41:02.869771+0000","last_peered":"2024-03-30T01:41:02.869771+0000","last_clean":"2024-03-30T01:41:02.869771+0000","last_became_active":"2024-03-30T01:40:58.436247+0000","last_became_peered":"2024-03-30T01:40:58.436247+0000","last_unstale":"2024-03-30T01:41:02.869771+0000","last_undegraded":"2024-03-30T01:41:02.869771+0000","last_fullsized":"2024-03-30T01:41:02.869771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868868+0000","last_change":"2024-03-30T01:40:58.415507+0000","last_active":"2024-03-30T01:41:02.868868+0000","last_peered":"2024-03-30T01:41:02.868868+0000","last_clean":"2024-03-30T01:41:02.868868+0000","last_became_active":"2024-03-30T01:40:58.415197+0000","last_became_peered":"2024-03-30T01:40:58.415197+0000","last_unstale":"2024-03-30T01:41:02.868868+0000","last_undegraded":"2024-03-30T01:41:02.868868+0000","last_fullsized":"2024-03-30T01:41:02.868868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868506+0000","last_change":"2024-03-30T01:40:58.425962+0000","last_active":"2024-03-30T01:41:02.868506+0000","last_peered":"2024-03-30T01:41:02.868506+0000","last_clean":"2024-03-30T01:41:02.868506+0000","last_became_active":"2024-03-30T01:40:58.425667+0000","last_became_peered":"2024-03-30T01:40:58.425667+0000","last_unstale":"2024-03-30T01:41:02.868506+0000","last_undegraded":"2024-03-30T01:41:02.868506+0000","last_fullsized":"2024-03-30T01:41:02.868506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872585+0000","last_change":"2024-03-30T01:40:58.429802+0000","last_active":"2024-03-30T01:41:02.872585+0000","last_peered":"2024-03-30T01:41:02.872585+0000","last_clean":"2024-03-30T01:41:02.872585+0000","last_became_active":"2024-03-30T01:40:58.429296+0000","last_became_peered":"2024-03-30T01:40:58.429296+0000","last_unstale":"2024-03-30T01:41:02.872585+0000","last_undegraded":"2024-03-30T01:41:02.872585+0000","last_fullsized":"2024-03-30T01:41:02.872585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869051+0000","last_change":"2024-03-30T01:40:58.420564+0000","last_active":"2024-03-30T01:41:02.869051+0000","last_peered":"2024-03-30T01:41:02.869051+0000","last_clean":"2024-03-30T01:41:02.869051+0000","last_became_active":"2024-03-30T01:40:58.420286+0000","last_became_peered":"2024-03-30T01:40:58.420286+0000","last_unstale":"2024-03-30T01:41:02.869051+0000","last_undegraded":"2024-03-30T01:41:02.869051+0000","last_fullsized":"2024-03-30T01:41:02.869051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868288+0000","last_change":"2024-03-30T01:40:58.418904+0000","last_active":"2024-03-30T01:41:02.868288+0000","last_peered":"2024-03-30T01:41:02.868288+0000","last_clean":"2024-03-30T01:41:02.868288+0000","last_became_active":"2024-03-30T01:40:58.418351+0000","last_became_peered":"2024-03-30T01:40:58.418351+0000","last_unstale":"2024-03-30T01:41:02.868288+0000","last_undegraded":"2024-03-30T01:41:02.868288+0000","last_fullsized":"2024-03-30T01:41:02.868288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868595+0000","last_change":"2024-03-30T01:40:58.416220+0000","last_active":"2024-03-30T01:41:02.868595+0000","last_peered":"2024-03-30T01:41:02.868595+0000","last_clean":"2024-03-30T01:41:02.868595+0000","last_became_active":"2024-03-30T01:40:58.415937+0000","last_became_peered":"2024-03-30T01:40:58.415937+0000","last_unstale":"2024-03-30T01:41:02.868595+0000","last_undegraded":"2024-03-30T01:41:02.868595+0000","last_fullsized":"2024-03-30T01:41:02.868595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868332+0000","last_change":"2024-03-30T01:40:58.433268+0000","last_active":"2024-03-30T01:41:02.868332+0000","last_peered":"2024-03-30T01:41:02.868332+0000","last_clean":"2024-03-30T01:41:02.868332+0000","last_became_active":"2024-03-30T01:40:58.432963+0000","last_became_peered":"2024-03-30T01:40:58.432963+0000","last_unstale":"2024-03-30T01:41:02.868332+0000","last_undegraded":"2024-03-30T01:41:02.868332+0000","last_fullsized":"2024-03-30T01:41:02.868332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869784+0000","last_change":"2024-03-30T01:40:58.440462+0000","last_active":"2024-03-30T01:41:02.869784+0000","last_peered":"2024-03-30T01:41:02.869784+0000","last_clean":"2024-03-30T01:41:02.869784+0000","last_became_active":"2024-03-30T01:40:58.440144+0000","last_became_peered":"2024-03-30T01:40:58.440144+0000","last_unstale":"2024-03-30T01:41:02.869784+0000","last_undegraded":"2024-03-30T01:41:02.869784+0000","last_fullsized":"2024-03-30T01:41:02.869784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872499+0000","last_change":"2024-03-30T01:40:58.422820+0000","last_active":"2024-03-30T01:41:02.872499+0000","last_peered":"2024-03-30T01:41:02.872499+0000","last_clean":"2024-03-30T01:41:02.872499+0000","last_became_active":"2024-03-30T01:40:58.420834+0000","last_became_peered":"2024-03-30T01:40:58.420834+0000","last_unstale":"2024-03-30T01:41:02.872499+0000","last_undegraded":"2024-03-30T01:41:02.872499+0000","last_fullsized":"2024-03-30T01:41:02.872499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4453,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":856064,"data_stored":689817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:41:17.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:41:18.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.163+0000 7f6a5da6a700 1 Processor -- start 2024-03-30T01:41:18.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.163+0000 7f6a5da6a700 1 -- start start 2024-03-30T01:41:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.164+0000 7f6a5da6a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a5806b170 0x7f6a58155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.164+0000 7f6a5da6a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a580bd7f0 0x7f6a5806b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.164+0000 7f6a5da6a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580c8d10 con 0x7f6a58154760 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.164+0000 7f6a5da6a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5806bef0 con 0x7f6a5806b170 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.164+0000 7f6a5da6a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580c98b0 con 0x7f6a580bd7f0 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.164+0000 7f6a577fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a5806b170 0x7f6a58155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.164+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a580bd7f0 0x7f6a5806b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.164+0000 7f6a577fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a5806b170 0x7f6a58155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38664/0 (socket says 172.21.15.31:38664) 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.164+0000 7f6a577fe700 1 -- 172.21.15.31:0/4238888664 learned_addr learned my addr 172.21.15.31:0/4238888664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.165+0000 7f6a577fe700 1 -- 172.21.15.31:0/4238888664 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a580bd7f0 msgr2=0x7f6a5806b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.165+0000 7f6a577fe700 1 --2- 172.21.15.31:0/4238888664 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a580bd7f0 0x7f6a5806b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.165+0000 7f6a577fe700 1 -- 172.21.15.31:0/4238888664 >> v1:172.21.15.59:6789/0 conn(0x7f6a58154760 legacy=0x7f6a580c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.165+0000 7f6a567fc700 1 -- 172.21.15.31:0/4238888664 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291782590 0 0) 0x7f6a580c8d10 con 0x7f6a58154760 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.165+0000 7f6a577fe700 1 -- 172.21.15.31:0/4238888664 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580c8100 con 0x7f6a5806b170 2024-03-30T01:41:18.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.165+0000 7f6a577fe700 1 --2- 172.21.15.31:0/4238888664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a5806b170 0x7f6a58155c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a480098f0 tx=0x7f6a48009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669d6263b2d90cc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.165+0000 7f6a567fc700 1 -- 172.21.15.31:0/4238888664 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48011830 con 0x7f6a5806b170 2024-03-30T01:41:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.165+0000 7f6a567fc700 1 -- 172.21.15.31:0/4238888664 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a48011990 con 0x7f6a5806b170 2024-03-30T01:41:18.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.165+0000 7f6a567fc700 1 -- 172.21.15.31:0/4238888664 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48011d20 con 0x7f6a5806b170 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.166+0000 7f6a5da6a700 1 -- 172.21.15.31:0/4238888664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a5806b170 msgr2=0x7f6a58155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.166+0000 7f6a5da6a700 1 --2- 172.21.15.31:0/4238888664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a5806b170 0x7f6a58155c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a480098f0 tx=0x7f6a48009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.166+0000 7f6a5da6a700 1 -- 172.21.15.31:0/4238888664 shutdown_connections 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.166+0000 7f6a5da6a700 1 --2- 172.21.15.31:0/4238888664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a5806b170 0x7f6a58155c50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.166+0000 7f6a5da6a700 1 --2- 172.21.15.31:0/4238888664 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a580bd7f0 0x7f6a5806b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.166+0000 7f6a5da6a700 1 -- 172.21.15.31:0/4238888664 >> 172.21.15.31:0/4238888664 conn(0x7f6a580ca0d0 msgr2=0x7f6a580ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.166+0000 7f6a5da6a700 1 -- 172.21.15.31:0/4238888664 shutdown_connections 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.166+0000 7f6a5da6a700 1 -- 172.21.15.31:0/4238888664 wait complete. 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.167+0000 7f6a5da6a700 1 Processor -- start 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.167+0000 7f6a5da6a700 1 -- start start 2024-03-30T01:41:18.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a5da6a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a5806b170 0x7f6a580bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a5da6a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a580bd7f0 0x7f6a580c0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a5da6a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a58154760 0x7f6a58156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a5da6a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a581ea2e0 con 0x7f6a58154760 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a5da6a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a581e9730 con 0x7f6a580bd7f0 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a5da6a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580c25a0 con 0x7f6a5806b170 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a57fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a58154760 0x7f6a58156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a57fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a58154760 0x7f6a58156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36422/0 (socket says 172.21.15.31:36422) 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a57fff700 1 -- 172.21.15.31:0/808557902 learned_addr learned my addr 172.21.15.31:0/808557902 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a56ffd700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a580bd7f0 0x7f6a580c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a577fe700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a5806b170 0x7f6a580bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a57fff700 1 -- 172.21.15.31:0/808557902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a5806b170 msgr2=0x7f6a580bdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a57fff700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a5806b170 0x7f6a580bdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a57fff700 1 -- 172.21.15.31:0/808557902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a580bd7f0 msgr2=0x7f6a580c0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a57fff700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a580bd7f0 0x7f6a580c0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.168+0000 7f6a57fff700 1 -- 172.21.15.31:0/808557902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a48009550 con 0x7f6a58154760 2024-03-30T01:41:18.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.169+0000 7f6a57fff700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a58154760 0x7f6a58156c60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a50001560 tx=0x7f6a5000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:18.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.169+0000 7f6a3ffff700 1 -- 172.21.15.31:0/808557902 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50003e30 con 0x7f6a58154760 2024-03-30T01:41:18.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.169+0000 7f6a3ffff700 1 -- 172.21.15.31:0/808557902 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a50005070 con 0x7f6a58154760 2024-03-30T01:41:18.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.169+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a581ea700 con 0x7f6a58154760 2024-03-30T01:41:18.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.169+0000 7f6a3ffff700 1 -- 172.21.15.31:0/808557902 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50005420 con 0x7f6a58154760 2024-03-30T01:41:18.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.169+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a580c0d80 con 0x7f6a58154760 2024-03-30T01:41:18.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.170+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5804ea80 con 0x7f6a58154760 2024-03-30T01:41:18.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.170+0000 7f6a3ffff700 1 -- 172.21.15.31:0/808557902 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6a50005580 con 0x7f6a58154760 2024-03-30T01:41:18.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.171+0000 7f6a3ffff700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a4406ac90 0x7f6a4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.171+0000 7f6a3ffff700 1 -- 172.21.15.31:0/808557902 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6a50088cb0 con 0x7f6a58154760 2024-03-30T01:41:18.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.171+0000 7f6a577fe700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a4406ac90 0x7f6a4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.171+0000 7f6a577fe700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a4406ac90 0x7f6a4406d150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a48014040 tx=0x7f6a48008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:18.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.176+0000 7f6a3ffff700 1 -- 172.21.15.31:0/808557902 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6a50053b20 con 0x7f6a58154760 2024-03-30T01:41:18.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.386+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a580c89a0 con 0x7f6a4406ac90 2024-03-30T01:41:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.396+0000 7f6a3ffff700 1 -- 172.21.15.31:0/808557902 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345314 (secure 0 0 0) 0x7f6a48011c30 con 0x7f6a4406ac90 2024-03-30T01:41:18.407 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.400+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a4406ac90 msgr2=0x7f6a4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.400+0000 7f6a5da6a700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a4406ac90 0x7f6a4406d150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a48014040 tx=0x7f6a48008030 comp rx=0 tx=0).stop 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.400+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a58154760 msgr2=0x7f6a58156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.400+0000 7f6a5da6a700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a58154760 0x7f6a58156c60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a50001560 tx=0x7f6a5000cb50 comp rx=0 tx=0).stop 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.400+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 shutdown_connections 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.400+0000 7f6a5da6a700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a58154760 0x7f6a58156c60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.400+0000 7f6a5da6a700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a4406ac90 0x7f6a4406d150 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.400+0000 7f6a5da6a700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a5806b170 0x7f6a580bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.400+0000 7f6a5da6a700 1 --2- 172.21.15.31:0/808557902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a580bd7f0 0x7f6a580c0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.401+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 >> 172.21.15.31:0/808557902 conn(0x7f6a580ca0d0 msgr2=0x7f6a5806b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.401+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 shutdown_connections 2024-03-30T01:41:18.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.401+0000 7f6a5da6a700 1 -- 172.21.15.31:0/808557902 wait complete. 2024-03-30T01:41:18.411 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:41:18.421 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-03-30T01:41:17.803630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594821,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219364,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755356,"statfs":{"total":773094113280,"available":772869484544,"internally_reserved":0,"allocated":3280896,"data_stored":1986913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2091,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002862"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965171+0000","last_change":"2024-03-30T01:40:58.441027+0000","last_active":"2024-03-30T01:41:02.965171+0000","last_peered":"2024-03-30T01:41:02.965171+0000","last_clean":"2024-03-30T01:41:02.965171+0000","last_became_active":"2024-03-30T01:40:58.440741+0000","last_became_peered":"2024-03-30T01:40:58.440741+0000","last_unstale":"2024-03-30T01:41:02.965171+0000","last_undegraded":"2024-03-30T01:41:02.965171+0000","last_fullsized":"2024-03-30T01:41:02.965171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868566+0000","last_change":"2024-03-30T01:40:58.426588+0000","last_active":"2024-03-30T01:41:02.868566+0000","last_peered":"2024-03-30T01:41:02.868566+0000","last_clean":"2024-03-30T01:41:02.868566+0000","last_became_active":"2024-03-30T01:40:58.426336+0000","last_became_peered":"2024-03-30T01:40:58.426336+0000","last_unstale":"2024-03-30T01:41:02.868566+0000","last_undegraded":"2024-03-30T01:41:02.868566+0000","last_fullsized":"2024-03-30T01:41:02.868566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869813+0000","last_change":"2024-03-30T01:40:58.434632+0000","last_active":"2024-03-30T01:41:02.869813+0000","last_peered":"2024-03-30T01:41:02.869813+0000","last_clean":"2024-03-30T01:41:02.869813+0000","last_became_active":"2024-03-30T01:40:58.434339+0000","last_became_peered":"2024-03-30T01:40:58.434339+0000","last_unstale":"2024-03-30T01:41:02.869813+0000","last_undegraded":"2024-03-30T01:41:02.869813+0000","last_fullsized":"2024-03-30T01:41:02.869813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:35.665153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868405+0000","last_change":"2024-03-30T01:40:58.428659+0000","last_active":"2024-03-30T01:41:02.868405+0000","last_peered":"2024-03-30T01:41:02.868405+0000","last_clean":"2024-03-30T01:41:02.868405+0000","last_became_active":"2024-03-30T01:40:58.426016+0000","last_became_peered":"2024-03-30T01:40:58.426016+0000","last_unstale":"2024-03-30T01:41:02.868405+0000","last_undegraded":"2024-03-30T01:41:02.868405+0000","last_fullsized":"2024-03-30T01:41:02.868405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964596+0000","last_change":"2024-03-30T01:40:58.440079+0000","last_active":"2024-03-30T01:41:02.964596+0000","last_peered":"2024-03-30T01:41:02.964596+0000","last_clean":"2024-03-30T01:41:02.964596+0000","last_became_active":"2024-03-30T01:40:58.439661+0000","last_became_peered":"2024-03-30T01:40:58.439661+0000","last_unstale":"2024-03-30T01:41:02.964596+0000","last_undegraded":"2024-03-30T01:41:02.964596+0000","last_fullsized":"2024-03-30T01:41:02.964596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868344+0000","last_change":"2024-03-30T01:40:58.426692+0000","last_active":"2024-03-30T01:41:02.868344+0000","last_peered":"2024-03-30T01:41:02.868344+0000","last_clean":"2024-03-30T01:41:02.868344+0000","last_became_active":"2024-03-30T01:40:58.426370+0000","last_became_peered":"2024-03-30T01:40:58.426370+0000","last_unstale":"2024-03-30T01:41:02.868344+0000","last_undegraded":"2024-03-30T01:41:02.868344+0000","last_fullsized":"2024-03-30T01:41:02.868344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:51.791864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964482+0000","last_change":"2024-03-30T01:40:58.440389+0000","last_active":"2024-03-30T01:41:02.964482+0000","last_peered":"2024-03-30T01:41:02.964482+0000","last_clean":"2024-03-30T01:41:02.964482+0000","last_became_active":"2024-03-30T01:40:58.439965+0000","last_became_peered":"2024-03-30T01:40:58.439965+0000","last_unstale":"2024-03-30T01:41:02.964482+0000","last_undegraded":"2024-03-30T01:41:02.964482+0000","last_fullsized":"2024-03-30T01:41:02.964482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:57.923292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868477+0000","last_change":"2024-03-30T01:40:58.429571+0000","last_active":"2024-03-30T01:41:02.868477+0000","last_peered":"2024-03-30T01:41:02.868477+0000","last_clean":"2024-03-30T01:41:02.868477+0000","last_became_active":"2024-03-30T01:40:58.429144+0000","last_became_peered":"2024-03-30T01:40:58.429144+0000","last_unstale":"2024-03-30T01:41:02.868477+0000","last_undegraded":"2024-03-30T01:41:02.868477+0000","last_fullsized":"2024-03-30T01:41:02.868477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965163+0000","last_change":"2024-03-30T01:40:58.441005+0000","last_active":"2024-03-30T01:41:02.965163+0000","last_peered":"2024-03-30T01:41:02.965163+0000","last_clean":"2024-03-30T01:41:02.965163+0000","last_became_active":"2024-03-30T01:40:58.440409+0000","last_became_peered":"2024-03-30T01:40:58.440409+0000","last_unstale":"2024-03-30T01:41:02.965163+0000","last_undegraded":"2024-03-30T01:41:02.965163+0000","last_fullsized":"2024-03-30T01:41:02.965163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:48:57.608199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872597+0000","last_change":"2024-03-30T01:40:58.429626+0000","last_active":"2024-03-30T01:41:02.872597+0000","last_peered":"2024-03-30T01:41:02.872597+0000","last_clean":"2024-03-30T01:41:02.872597+0000","last_became_active":"2024-03-30T01:40:58.428916+0000","last_became_peered":"2024-03-30T01:40:58.428916+0000","last_unstale":"2024-03-30T01:41:02.872597+0000","last_undegraded":"2024-03-30T01:41:02.872597+0000","last_fullsized":"2024-03-30T01:41:02.872597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872572+0000","last_change":"2024-03-30T01:40:58.430546+0000","last_active":"2024-03-30T01:41:02.872572+0000","last_peered":"2024-03-30T01:41:02.872572+0000","last_clean":"2024-03-30T01:41:02.872572+0000","last_became_active":"2024-03-30T01:40:58.430232+0000","last_became_peered":"2024-03-30T01:40:58.430232+0000","last_unstale":"2024-03-30T01:41:02.872572+0000","last_undegraded":"2024-03-30T01:41:02.872572+0000","last_fullsized":"2024-03-30T01:41:02.872572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864576+0000","last_change":"2024-03-30T01:40:58.422212+0000","last_active":"2024-03-30T01:41:02.864576+0000","last_peered":"2024-03-30T01:41:02.864576+0000","last_clean":"2024-03-30T01:41:02.864576+0000","last_became_active":"2024-03-30T01:40:58.421812+0000","last_became_peered":"2024-03-30T01:40:58.421812+0000","last_unstale":"2024-03-30T01:41:02.864576+0000","last_undegraded":"2024-03-30T01:41:02.864576+0000","last_fullsized":"2024-03-30T01:41:02.864576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:30.774569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868720+0000","last_change":"2024-03-30T01:40:58.418447+0000","last_active":"2024-03-30T01:41:02.868720+0000","last_peered":"2024-03-30T01:41:02.868720+0000","last_clean":"2024-03-30T01:41:02.868720+0000","last_became_active":"2024-03-30T01:40:58.418146+0000","last_became_peered":"2024-03-30T01:40:58.418146+0000","last_unstale":"2024-03-30T01:41:02.868720+0000","last_undegraded":"2024-03-30T01:41:02.868720+0000","last_fullsized":"2024-03-30T01:41:02.868720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869653+0000","last_change":"2024-03-30T01:40:58.430002+0000","last_active":"2024-03-30T01:41:02.869653+0000","last_peered":"2024-03-30T01:41:02.869653+0000","last_clean":"2024-03-30T01:41:02.869653+0000","last_became_active":"2024-03-30T01:40:58.429743+0000","last_became_peered":"2024-03-30T01:40:58.429743+0000","last_unstale":"2024-03-30T01:41:02.869653+0000","last_undegraded":"2024-03-30T01:41:02.869653+0000","last_fullsized":"2024-03-30T01:41:02.869653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:54.449729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868389+0000","last_change":"2024-03-30T01:40:58.425393+0000","last_active":"2024-03-30T01:41:02.868389+0000","last_peered":"2024-03-30T01:41:02.868389+0000","last_clean":"2024-03-30T01:41:02.868389+0000","last_became_active":"2024-03-30T01:40:58.424890+0000","last_became_peered":"2024-03-30T01:40:58.424890+0000","last_unstale":"2024-03-30T01:41:02.868389+0000","last_undegraded":"2024-03-30T01:41:02.868389+0000","last_fullsized":"2024-03-30T01:41:02.868389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868218+0000","last_change":"2024-03-30T01:40:58.426453+0000","last_active":"2024-03-30T01:41:02.868218+0000","last_peered":"2024-03-30T01:41:02.868218+0000","last_clean":"2024-03-30T01:41:02.868218+0000","last_became_active":"2024-03-30T01:40:58.425360+0000","last_became_peered":"2024-03-30T01:40:58.425360+0000","last_unstale":"2024-03-30T01:41:02.868218+0000","last_undegraded":"2024-03-30T01:41:02.868218+0000","last_fullsized":"2024-03-30T01:41:02.868218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:28.675589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868517+0000","last_change":"2024-03-30T01:40:58.433354+0000","last_active":"2024-03-30T01:41:02.868517+0000","last_peered":"2024-03-30T01:41:02.868517+0000","last_clean":"2024-03-30T01:41:02.868517+0000","last_became_active":"2024-03-30T01:40:58.433034+0000","last_became_peered":"2024-03-30T01:40:58.433034+0000","last_unstale":"2024-03-30T01:41:02.868517+0000","last_undegraded":"2024-03-30T01:41:02.868517+0000","last_fullsized":"2024-03-30T01:41:02.868517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:30.394391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868781+0000","last_change":"2024-03-30T01:40:58.433603+0000","last_active":"2024-03-30T01:41:02.868781+0000","last_peered":"2024-03-30T01:41:02.868781+0000","last_clean":"2024-03-30T01:41:02.868781+0000","last_became_active":"2024-03-30T01:40:58.433303+0000","last_became_peered":"2024-03-30T01:40:58.433303+0000","last_unstale":"2024-03-30T01:41:02.868781+0000","last_undegraded":"2024-03-30T01:41:02.868781+0000","last_fullsized":"2024-03-30T01:41:02.868781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:05:33.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965044+0000","last_change":"2024-03-30T01:40:58.436807+0000","last_active":"2024-03-30T01:41:02.965044+0000","last_peered":"2024-03-30T01:41:02.965044+0000","last_clean":"2024-03-30T01:41:02.965044+0000","last_became_active":"2024-03-30T01:40:58.436528+0000","last_became_peered":"2024-03-30T01:40:58.436528+0000","last_unstale":"2024-03-30T01:41:02.965044+0000","last_undegraded":"2024-03-30T01:41:02.965044+0000","last_fullsized":"2024-03-30T01:41:02.965044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:21.062997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972825+0000","last_change":"2024-03-30T01:41:02.970253+0000","last_active":"2024-03-30T01:41:02.972825+0000","last_peered":"2024-03-30T01:41:02.972825+0000","last_clean":"2024-03-30T01:41:02.972825+0000","last_became_active":"2024-03-30T01:41:02.969854+0000","last_became_peered":"2024-03-30T01:41:02.969854+0000","last_unstale":"2024-03-30T01:41:02.972825+0000","last_undegraded":"2024-03-30T01:41:02.972825+0000","last_fullsized":"2024-03-30T01:41:02.972825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:57:18.590955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.786878+0000","last_change":"2024-03-30T01:40:57.963998+0000","last_active":"2024-03-30T01:41:02.786878+0000","last_peered":"2024-03-30T01:41:02.786878+0000","last_clean":"2024-03-30T01:41:02.786878+0000","last_became_active":"2024-03-30T01:40:57.963535+0000","last_became_peered":"2024-03-30T01:40:57.963535+0000","last_unstale":"2024-03-30T01:41:02.786878+0000","last_undegraded":"2024-03-30T01:41:02.786878+0000","last_fullsized":"2024-03-30T01:41:02.786878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:40:49.324287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868907+0000","last_change":"2024-03-30T01:40:58.418336+0000","last_active":"2024-03-30T01:41:02.868907+0000","last_peered":"2024-03-30T01:41:02.868907+0000","last_clean":"2024-03-30T01:41:02.868907+0000","last_became_active":"2024-03-30T01:40:58.418057+0000","last_became_peered":"2024-03-30T01:40:58.418057+0000","last_unstale":"2024-03-30T01:41:02.868907+0000","last_undegraded":"2024-03-30T01:41:02.868907+0000","last_fullsized":"2024-03-30T01:41:02.868907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:52.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864399+0000","last_change":"2024-03-30T01:41:02.791579+0000","last_active":"2024-03-30T01:41:02.864399+0000","last_peered":"2024-03-30T01:41:02.864399+0000","last_clean":"2024-03-30T01:41:02.864399+0000","last_became_active":"2024-03-30T01:41:02.791102+0000","last_became_peered":"2024-03-30T01:41:02.791102+0000","last_unstale":"2024-03-30T01:41:02.864399+0000","last_undegraded":"2024-03-30T01:41:02.864399+0000","last_fullsized":"2024-03-30T01:41:02.864399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:34.215813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.776056+0000","last_change":"2024-03-30T01:40:57.963840+0000","last_active":"2024-03-30T01:41:02.776056+0000","last_peered":"2024-03-30T01:41:02.776056+0000","last_clean":"2024-03-30T01:41:02.776056+0000","last_became_active":"2024-03-30T01:40:57.962842+0000","last_became_peered":"2024-03-30T01:40:57.962842+0000","last_unstale":"2024-03-30T01:41:02.776056+0000","last_undegraded":"2024-03-30T01:41:02.776056+0000","last_fullsized":"2024-03-30T01:41:02.776056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869931+0000","last_change":"2024-03-30T01:40:58.435910+0000","last_active":"2024-03-30T01:41:02.869931+0000","last_peered":"2024-03-30T01:41:02.869931+0000","last_clean":"2024-03-30T01:41:02.869931+0000","last_became_active":"2024-03-30T01:40:58.435402+0000","last_became_peered":"2024-03-30T01:40:58.435402+0000","last_unstale":"2024-03-30T01:41:02.869931+0000","last_undegraded":"2024-03-30T01:41:02.869931+0000","last_fullsized":"2024-03-30T01:41:02.869931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864534+0000","last_change":"2024-03-30T01:41:02.789877+0000","last_active":"2024-03-30T01:41:02.864534+0000","last_peered":"2024-03-30T01:41:02.864534+0000","last_clean":"2024-03-30T01:41:02.864534+0000","last_became_active":"2024-03-30T01:41:02.789440+0000","last_became_peered":"2024-03-30T01:41:02.789440+0000","last_unstale":"2024-03-30T01:41:02.864534+0000","last_undegraded":"2024-03-30T01:41:02.864534+0000","last_fullsized":"2024-03-30T01:41:02.864534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:23.954119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964766+0000","last_change":"2024-03-30T01:40:57.968973+0000","last_active":"2024-03-30T01:41:02.964766+0000","last_peered":"2024-03-30T01:41:02.964766+0000","last_clean":"2024-03-30T01:41:02.964766+0000","last_became_active":"2024-03-30T01:40:57.968415+0000","last_became_peered":"2024-03-30T01:40:57.968415+0000","last_unstale":"2024-03-30T01:41:02.964766+0000","last_undegraded":"2024-03-30T01:41:02.964766+0000","last_fullsized":"2024-03-30T01:41:02.964766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868378+0000","last_change":"2024-03-30T01:40:58.431466+0000","last_active":"2024-03-30T01:41:02.868378+0000","last_peered":"2024-03-30T01:41:02.868378+0000","last_clean":"2024-03-30T01:41:02.868378+0000","last_became_active":"2024-03-30T01:40:58.431122+0000","last_became_peered":"2024-03-30T01:40:58.431122+0000","last_unstale":"2024-03-30T01:41:02.868378+0000","last_undegraded":"2024-03-30T01:41:02.868378+0000","last_fullsized":"2024-03-30T01:41:02.868378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:21.062997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864511+0000","last_change":"2024-03-30T01:41:02.789299+0000","last_active":"2024-03-30T01:41:02.864511+0000","last_peered":"2024-03-30T01:41:02.864511+0000","last_clean":"2024-03-30T01:41:02.864511+0000","last_became_active":"2024-03-30T01:41:02.788769+0000","last_became_peered":"2024-03-30T01:41:02.788769+0000","last_unstale":"2024-03-30T01:41:02.864511+0000","last_undegraded":"2024-03-30T01:41:02.864511+0000","last_fullsized":"2024-03-30T01:41:02.864511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:18.933245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964688+0000","last_change":"2024-03-30T01:40:57.970239+0000","last_active":"2024-03-30T01:41:02.964688+0000","last_peered":"2024-03-30T01:41:02.964688+0000","last_clean":"2024-03-30T01:41:02.964688+0000","last_became_active":"2024-03-30T01:40:57.969868+0000","last_became_peered":"2024-03-30T01:40:57.969868+0000","last_unstale":"2024-03-30T01:41:02.964688+0000","last_undegraded":"2024-03-30T01:41:02.964688+0000","last_fullsized":"2024-03-30T01:41:02.964688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:34.662924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868462+0000","last_change":"2024-03-30T01:40:58.431864+0000","last_active":"2024-03-30T01:41:02.868462+0000","last_peered":"2024-03-30T01:41:02.868462+0000","last_clean":"2024-03-30T01:41:02.868462+0000","last_became_active":"2024-03-30T01:40:58.431101+0000","last_became_peered":"2024-03-30T01:40:58.431101+0000","last_unstale":"2024-03-30T01:41:02.868462+0000","last_undegraded":"2024-03-30T01:41:02.868462+0000","last_fullsized":"2024-03-30T01:41:02.868462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:43.041425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868886+0000","last_change":"2024-03-30T01:41:02.797411+0000","last_active":"2024-03-30T01:41:02.868886+0000","last_peered":"2024-03-30T01:41:02.868886+0000","last_clean":"2024-03-30T01:41:02.868886+0000","last_became_active":"2024-03-30T01:41:02.797018+0000","last_became_peered":"2024-03-30T01:41:02.797018+0000","last_unstale":"2024-03-30T01:41:02.868886+0000","last_undegraded":"2024-03-30T01:41:02.868886+0000","last_fullsized":"2024-03-30T01:41:02.868886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:18:03.263600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.775328+0000","last_change":"2024-03-30T01:40:57.963372+0000","last_active":"2024-03-30T01:41:02.775328+0000","last_peered":"2024-03-30T01:41:02.775328+0000","last_clean":"2024-03-30T01:41:02.775328+0000","last_became_active":"2024-03-30T01:40:57.963033+0000","last_became_peered":"2024-03-30T01:40:57.963033+0000","last_unstale":"2024-03-30T01:41:02.775328+0000","last_undegraded":"2024-03-30T01:41:02.775328+0000","last_fullsized":"2024-03-30T01:41:02.775328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864523+0000","last_change":"2024-03-30T01:40:58.427338+0000","last_active":"2024-03-30T01:41:02.864523+0000","last_peered":"2024-03-30T01:41:02.864523+0000","last_clean":"2024-03-30T01:41:02.864523+0000","last_became_active":"2024-03-30T01:40:58.427023+0000","last_became_peered":"2024-03-30T01:40:58.427023+0000","last_unstale":"2024-03-30T01:41:02.864523+0000","last_undegraded":"2024-03-30T01:41:02.864523+0000","last_fullsized":"2024-03-30T01:41:02.864523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:35.665153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868735+0000","last_change":"2024-03-30T01:41:02.801387+0000","last_active":"2024-03-30T01:41:02.868735+0000","last_peered":"2024-03-30T01:41:02.868735+0000","last_clean":"2024-03-30T01:41:02.868735+0000","last_became_active":"2024-03-30T01:41:02.800907+0000","last_became_peered":"2024-03-30T01:41:02.800907+0000","last_unstale":"2024-03-30T01:41:02.868735+0000","last_undegraded":"2024-03-30T01:41:02.868735+0000","last_fullsized":"2024-03-30T01:41:02.868735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:43.218702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770418+0000","last_change":"2024-03-30T01:40:57.965707+0000","last_active":"2024-03-30T01:41:02.770418+0000","last_peered":"2024-03-30T01:41:02.770418+0000","last_clean":"2024-03-30T01:41:02.770418+0000","last_became_active":"2024-03-30T01:40:57.965371+0000","last_became_peered":"2024-03-30T01:40:57.965371+0000","last_unstale":"2024-03-30T01:41:02.770418+0000","last_undegraded":"2024-03-30T01:41:02.770418+0000","last_fullsized":"2024-03-30T01:41:02.770418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869759+0000","last_change":"2024-03-30T01:40:58.432647+0000","last_active":"2024-03-30T01:41:02.869759+0000","last_peered":"2024-03-30T01:41:02.869759+0000","last_clean":"2024-03-30T01:41:02.869759+0000","last_became_active":"2024-03-30T01:40:58.432292+0000","last_became_peered":"2024-03-30T01:40:58.432292+0000","last_unstale":"2024-03-30T01:41:02.869759+0000","last_undegraded":"2024-03-30T01:41:02.869759+0000","last_fullsized":"2024-03-30T01:41:02.869759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872608+0000","last_change":"2024-03-30T01:41:02.798539+0000","last_active":"2024-03-30T01:41:02.872608+0000","last_peered":"2024-03-30T01:41:02.872608+0000","last_clean":"2024-03-30T01:41:02.872608+0000","last_became_active":"2024-03-30T01:41:02.798146+0000","last_became_peered":"2024-03-30T01:41:02.798146+0000","last_unstale":"2024-03-30T01:41:02.872608+0000","last_undegraded":"2024-03-30T01:41:02.872608+0000","last_fullsized":"2024-03-30T01:41:02.872608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:18.933245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.773948+0000","last_change":"2024-03-30T01:40:57.959181+0000","last_active":"2024-03-30T01:41:02.773948+0000","last_peered":"2024-03-30T01:41:02.773948+0000","last_clean":"2024-03-30T01:41:02.773948+0000","last_became_active":"2024-03-30T01:40:57.958838+0000","last_became_peered":"2024-03-30T01:40:57.958838+0000","last_unstale":"2024-03-30T01:41:02.773948+0000","last_undegraded":"2024-03-30T01:41:02.773948+0000","last_fullsized":"2024-03-30T01:41:02.773948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:04.688535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868351+0000","last_change":"2024-03-30T01:40:58.429817+0000","last_active":"2024-03-30T01:41:02.868351+0000","last_peered":"2024-03-30T01:41:02.868351+0000","last_clean":"2024-03-30T01:41:02.868351+0000","last_became_active":"2024-03-30T01:40:58.429505+0000","last_became_peered":"2024-03-30T01:40:58.429505+0000","last_unstale":"2024-03-30T01:41:02.868351+0000","last_undegraded":"2024-03-30T01:41:02.868351+0000","last_fullsized":"2024-03-30T01:41:02.868351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:30:35.170155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.970928+0000","last_change":"2024-03-30T01:41:02.967897+0000","last_active":"2024-03-30T01:41:02.970928+0000","last_peered":"2024-03-30T01:41:02.970928+0000","last_clean":"2024-03-30T01:41:02.970928+0000","last_became_active":"2024-03-30T01:41:02.967524+0000","last_became_peered":"2024-03-30T01:41:02.967524+0000","last_unstale":"2024-03-30T01:41:02.970928+0000","last_undegraded":"2024-03-30T01:41:02.970928+0000","last_fullsized":"2024-03-30T01:41:02.970928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:58.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.778673+0000","last_change":"2024-03-30T01:40:57.964382+0000","last_active":"2024-03-30T01:41:02.778673+0000","last_peered":"2024-03-30T01:41:02.778673+0000","last_clean":"2024-03-30T01:41:02.778673+0000","last_became_active":"2024-03-30T01:40:57.963879+0000","last_became_peered":"2024-03-30T01:40:57.963879+0000","last_unstale":"2024-03-30T01:41:02.778673+0000","last_undegraded":"2024-03-30T01:41:02.778673+0000","last_fullsized":"2024-03-30T01:41:02.778673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868435+0000","last_change":"2024-03-30T01:40:58.425195+0000","last_active":"2024-03-30T01:41:02.868435+0000","last_peered":"2024-03-30T01:41:02.868435+0000","last_clean":"2024-03-30T01:41:02.868435+0000","last_became_active":"2024-03-30T01:40:58.424555+0000","last_became_peered":"2024-03-30T01:40:58.424555+0000","last_unstale":"2024-03-30T01:41:02.868435+0000","last_undegraded":"2024-03-30T01:41:02.868435+0000","last_fullsized":"2024-03-30T01:41:02.868435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972778+0000","last_change":"2024-03-30T01:41:02.968243+0000","last_active":"2024-03-30T01:41:02.972778+0000","last_peered":"2024-03-30T01:41:02.972778+0000","last_clean":"2024-03-30T01:41:02.972778+0000","last_became_active":"2024-03-30T01:41:02.967682+0000","last_became_peered":"2024-03-30T01:41:02.967682+0000","last_unstale":"2024-03-30T01:41:02.972778+0000","last_undegraded":"2024-03-30T01:41:02.972778+0000","last_fullsized":"2024-03-30T01:41:02.972778+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:40.634879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770699+0000","last_change":"2024-03-30T01:40:57.959485+0000","last_active":"2024-03-30T01:41:02.770699+0000","last_peered":"2024-03-30T01:41:02.770699+0000","last_clean":"2024-03-30T01:41:02.770699+0000","last_became_active":"2024-03-30T01:40:57.959180+0000","last_became_peered":"2024-03-30T01:40:57.959180+0000","last_unstale":"2024-03-30T01:41:02.770699+0000","last_undegraded":"2024-03-30T01:41:02.770699+0000","last_fullsized":"2024-03-30T01:41:02.770699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868505+0000","last_change":"2024-03-30T01:40:58.432524+0000","last_active":"2024-03-30T01:41:02.868505+0000","last_peered":"2024-03-30T01:41:02.868505+0000","last_clean":"2024-03-30T01:41:02.868505+0000","last_became_active":"2024-03-30T01:40:58.432225+0000","last_became_peered":"2024-03-30T01:40:58.432225+0000","last_unstale":"2024-03-30T01:41:02.868505+0000","last_undegraded":"2024-03-30T01:41:02.868505+0000","last_fullsized":"2024-03-30T01:41:02.868505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:30:35.170155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868329+0000","last_change":"2024-03-30T01:41:02.796380+0000","last_active":"2024-03-30T01:41:02.868329+0000","last_peered":"2024-03-30T01:41:02.868329+0000","last_clean":"2024-03-30T01:41:02.868329+0000","last_became_active":"2024-03-30T01:41:02.795984+0000","last_became_peered":"2024-03-30T01:41:02.795984+0000","last_unstale":"2024-03-30T01:41:02.868329+0000","last_undegraded":"2024-03-30T01:41:02.868329+0000","last_fullsized":"2024-03-30T01:41:02.868329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:13.399650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.781391+0000","last_change":"2024-03-30T01:40:57.958458+0000","last_active":"2024-03-30T01:41:02.781391+0000","last_peered":"2024-03-30T01:41:02.781391+0000","last_clean":"2024-03-30T01:41:02.781391+0000","last_became_active":"2024-03-30T01:40:57.957887+0000","last_became_peered":"2024-03-30T01:40:57.957887+0000","last_unstale":"2024-03-30T01:41:02.781391+0000","last_undegraded":"2024-03-30T01:41:02.781391+0000","last_fullsized":"2024-03-30T01:41:02.781391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869948+0000","last_change":"2024-03-30T01:40:58.436050+0000","last_active":"2024-03-30T01:41:02.869948+0000","last_peered":"2024-03-30T01:41:02.869948+0000","last_clean":"2024-03-30T01:41:02.869948+0000","last_became_active":"2024-03-30T01:40:58.435014+0000","last_became_peered":"2024-03-30T01:40:58.435014+0000","last_unstale":"2024-03-30T01:41:02.869948+0000","last_undegraded":"2024-03-30T01:41:02.869948+0000","last_fullsized":"2024-03-30T01:41:02.869948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:52.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864562+0000","last_change":"2024-03-30T01:41:02.784245+0000","last_active":"2024-03-30T01:41:02.864562+0000","last_peered":"2024-03-30T01:41:02.864562+0000","last_clean":"2024-03-30T01:41:02.864562+0000","last_became_active":"2024-03-30T01:41:02.783538+0000","last_became_peered":"2024-03-30T01:41:02.783538+0000","last_unstale":"2024-03-30T01:41:02.864562+0000","last_undegraded":"2024-03-30T01:41:02.864562+0000","last_fullsized":"2024-03-30T01:41:02.864562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:58.270483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.777921+0000","last_change":"2024-03-30T01:40:57.967602+0000","last_active":"2024-03-30T01:41:02.777921+0000","last_peered":"2024-03-30T01:41:02.777921+0000","last_clean":"2024-03-30T01:41:02.777921+0000","last_became_active":"2024-03-30T01:40:57.967239+0000","last_became_peered":"2024-03-30T01:40:57.967239+0000","last_unstale":"2024-03-30T01:41:02.777921+0000","last_undegraded":"2024-03-30T01:41:02.777921+0000","last_fullsized":"2024-03-30T01:41:02.777921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:36.274096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864600+0000","last_change":"2024-03-30T01:40:58.430901+0000","last_active":"2024-03-30T01:41:02.864600+0000","last_peered":"2024-03-30T01:41:02.864600+0000","last_clean":"2024-03-30T01:41:02.864600+0000","last_became_active":"2024-03-30T01:40:58.430595+0000","last_became_peered":"2024-03-30T01:40:58.430595+0000","last_unstale":"2024-03-30T01:41:02.864600+0000","last_undegraded":"2024-03-30T01:41:02.864600+0000","last_fullsized":"2024-03-30T01:41:02.864600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:37.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868468+0000","last_change":"2024-03-30T01:41:02.795440+0000","last_active":"2024-03-30T01:41:02.868468+0000","last_peered":"2024-03-30T01:41:02.868468+0000","last_clean":"2024-03-30T01:41:02.868468+0000","last_became_active":"2024-03-30T01:41:02.795052+0000","last_became_peered":"2024-03-30T01:41:02.795052+0000","last_unstale":"2024-03-30T01:41:02.868468+0000","last_undegraded":"2024-03-30T01:41:02.868468+0000","last_fullsized":"2024-03-30T01:41:02.868468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:23.954119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T01:41:07.910808+0000","last_change":"2024-03-30T01:40:57.966563+0000","last_active":"2024-03-30T01:41:07.910808+0000","last_peered":"2024-03-30T01:41:07.910808+0000","last_clean":"2024-03-30T01:41:07.910808+0000","last_became_active":"2024-03-30T01:40:57.966235+0000","last_became_peered":"2024-03-30T01:40:57.966235+0000","last_unstale":"2024-03-30T01:41:07.910808+0000","last_undegraded":"2024-03-30T01:41:07.910808+0000","last_fullsized":"2024-03-30T01:41:07.910808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3655,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964895+0000","last_change":"2024-03-30T01:40:58.436063+0000","last_active":"2024-03-30T01:41:02.964895+0000","last_peered":"2024-03-30T01:41:02.964895+0000","last_clean":"2024-03-30T01:41:02.964895+0000","last_became_active":"2024-03-30T01:40:58.435716+0000","last_became_peered":"2024-03-30T01:40:58.435716+0000","last_unstale":"2024-03-30T01:41:02.964895+0000","last_undegraded":"2024-03-30T01:41:02.964895+0000","last_fullsized":"2024-03-30T01:41:02.964895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:30.394391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.973943+0000","last_change":"2024-03-30T01:41:02.969718+0000","last_active":"2024-03-30T01:41:02.973943+0000","last_peered":"2024-03-30T01:41:02.973943+0000","last_clean":"2024-03-30T01:41:02.973943+0000","last_became_active":"2024-03-30T01:41:02.969246+0000","last_became_peered":"2024-03-30T01:41:02.969246+0000","last_unstale":"2024-03-30T01:41:02.973943+0000","last_undegraded":"2024-03-30T01:41:02.973943+0000","last_fullsized":"2024-03-30T01:41:02.973943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:34.173820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.786047+0000","last_change":"2024-03-30T01:40:57.971645+0000","last_active":"2024-03-30T01:41:02.786047+0000","last_peered":"2024-03-30T01:41:02.786047+0000","last_clean":"2024-03-30T01:41:02.786047+0000","last_became_active":"2024-03-30T01:40:57.971274+0000","last_became_peered":"2024-03-30T01:40:57.971274+0000","last_unstale":"2024-03-30T01:41:02.786047+0000","last_undegraded":"2024-03-30T01:41:02.786047+0000","last_fullsized":"2024-03-30T01:41:02.786047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:09.583893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869665+0000","last_change":"2024-03-30T01:40:58.431398+0000","last_active":"2024-03-30T01:41:02.869665+0000","last_peered":"2024-03-30T01:41:02.869665+0000","last_clean":"2024-03-30T01:41:02.869665+0000","last_became_active":"2024-03-30T01:40:58.431120+0000","last_became_peered":"2024-03-30T01:40:58.431120+0000","last_unstale":"2024-03-30T01:41:02.869665+0000","last_undegraded":"2024-03-30T01:41:02.869665+0000","last_fullsized":"2024-03-30T01:41:02.869665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968671+0000","last_change":"2024-03-30T01:41:02.966425+0000","last_active":"2024-03-30T01:41:02.968671+0000","last_peered":"2024-03-30T01:41:02.968671+0000","last_clean":"2024-03-30T01:41:02.968671+0000","last_became_active":"2024-03-30T01:41:02.965978+0000","last_became_peered":"2024-03-30T01:41:02.965978+0000","last_unstale":"2024-03-30T01:41:02.968671+0000","last_undegraded":"2024-03-30T01:41:02.968671+0000","last_fullsized":"2024-03-30T01:41:02.968671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780686+0000","last_change":"2024-03-30T01:40:57.956691+0000","last_active":"2024-03-30T01:41:02.780686+0000","last_peered":"2024-03-30T01:41:02.780686+0000","last_clean":"2024-03-30T01:41:02.780686+0000","last_became_active":"2024-03-30T01:40:57.956387+0000","last_became_peered":"2024-03-30T01:40:57.956387+0000","last_unstale":"2024-03-30T01:41:02.780686+0000","last_undegraded":"2024-03-30T01:41:02.780686+0000","last_fullsized":"2024-03-30T01:41:02.780686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964728+0000","last_change":"2024-03-30T01:40:58.435147+0000","last_active":"2024-03-30T01:41:02.964728+0000","last_peered":"2024-03-30T01:41:02.964728+0000","last_clean":"2024-03-30T01:41:02.964728+0000","last_became_active":"2024-03-30T01:40:58.434835+0000","last_became_peered":"2024-03-30T01:40:58.434835+0000","last_unstale":"2024-03-30T01:41:02.964728+0000","last_undegraded":"2024-03-30T01:41:02.964728+0000","last_fullsized":"2024-03-30T01:41:02.964728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:43.041425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972786+0000","last_change":"2024-03-30T01:41:02.968714+0000","last_active":"2024-03-30T01:41:02.972786+0000","last_peered":"2024-03-30T01:41:02.972786+0000","last_clean":"2024-03-30T01:41:02.972786+0000","last_became_active":"2024-03-30T01:41:02.967728+0000","last_became_peered":"2024-03-30T01:41:02.967728+0000","last_unstale":"2024-03-30T01:41:02.972786+0000","last_undegraded":"2024-03-30T01:41:02.972786+0000","last_fullsized":"2024-03-30T01:41:02.972786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:57:35.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770059+0000","last_change":"2024-03-30T01:40:57.964975+0000","last_active":"2024-03-30T01:41:02.770059+0000","last_peered":"2024-03-30T01:41:02.770059+0000","last_clean":"2024-03-30T01:41:02.770059+0000","last_became_active":"2024-03-30T01:40:57.964458+0000","last_became_peered":"2024-03-30T01:40:57.964458+0000","last_unstale":"2024-03-30T01:41:02.770059+0000","last_undegraded":"2024-03-30T01:41:02.770059+0000","last_fullsized":"2024-03-30T01:41:02.770059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872388+0000","last_change":"2024-03-30T01:40:58.425912+0000","last_active":"2024-03-30T01:41:02.872388+0000","last_peered":"2024-03-30T01:41:02.872388+0000","last_clean":"2024-03-30T01:41:02.872388+0000","last_became_active":"2024-03-30T01:40:58.425596+0000","last_became_peered":"2024-03-30T01:40:58.425596+0000","last_unstale":"2024-03-30T01:41:02.872388+0000","last_undegraded":"2024-03-30T01:41:02.872388+0000","last_fullsized":"2024-03-30T01:41:02.872388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869825+0000","last_change":"2024-03-30T01:41:02.793602+0000","last_active":"2024-03-30T01:41:02.869825+0000","last_peered":"2024-03-30T01:41:02.869825+0000","last_clean":"2024-03-30T01:41:02.869825+0000","last_became_active":"2024-03-30T01:41:02.793146+0000","last_became_peered":"2024-03-30T01:41:02.793146+0000","last_unstale":"2024-03-30T01:41:02.869825+0000","last_undegraded":"2024-03-30T01:41:02.869825+0000","last_fullsized":"2024-03-30T01:41:02.869825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782115+0000","last_change":"2024-03-30T01:40:57.962830+0000","last_active":"2024-03-30T01:41:02.782115+0000","last_peered":"2024-03-30T01:41:02.782115+0000","last_clean":"2024-03-30T01:41:02.782115+0000","last_became_active":"2024-03-30T01:40:57.961073+0000","last_became_peered":"2024-03-30T01:40:57.961073+0000","last_unstale":"2024-03-30T01:41:02.782115+0000","last_undegraded":"2024-03-30T01:41:02.782115+0000","last_fullsized":"2024-03-30T01:41:02.782115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964632+0000","last_change":"2024-03-30T01:40:58.435030+0000","last_active":"2024-03-30T01:41:02.964632+0000","last_peered":"2024-03-30T01:41:02.964632+0000","last_clean":"2024-03-30T01:41:02.964632+0000","last_became_active":"2024-03-30T01:40:58.434660+0000","last_became_peered":"2024-03-30T01:40:58.434660+0000","last_unstale":"2024-03-30T01:41:02.964632+0000","last_undegraded":"2024-03-30T01:41:02.964632+0000","last_fullsized":"2024-03-30T01:41:02.964632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:37.385758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864431+0000","last_change":"2024-03-30T01:41:02.796839+0000","last_active":"2024-03-30T01:41:02.864431+0000","last_peered":"2024-03-30T01:41:02.864431+0000","last_clean":"2024-03-30T01:41:02.864431+0000","last_became_active":"2024-03-30T01:41:02.796400+0000","last_became_peered":"2024-03-30T01:41:02.796400+0000","last_unstale":"2024-03-30T01:41:02.864431+0000","last_undegraded":"2024-03-30T01:41:02.864431+0000","last_fullsized":"2024-03-30T01:41:02.864431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:06:05.412311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.775105+0000","last_change":"2024-03-30T01:40:57.963776+0000","last_active":"2024-03-30T01:41:02.775105+0000","last_peered":"2024-03-30T01:41:02.775105+0000","last_clean":"2024-03-30T01:41:02.775105+0000","last_became_active":"2024-03-30T01:40:57.963473+0000","last_became_peered":"2024-03-30T01:40:57.963473+0000","last_unstale":"2024-03-30T01:41:02.775105+0000","last_undegraded":"2024-03-30T01:41:02.775105+0000","last_fullsized":"2024-03-30T01:41:02.775105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:09.583893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964956+0000","last_change":"2024-03-30T01:40:58.440136+0000","last_active":"2024-03-30T01:41:02.964956+0000","last_peered":"2024-03-30T01:41:02.964956+0000","last_clean":"2024-03-30T01:41:02.964956+0000","last_became_active":"2024-03-30T01:40:58.439831+0000","last_became_peered":"2024-03-30T01:40:58.439831+0000","last_unstale":"2024-03-30T01:41:02.964956+0000","last_undegraded":"2024-03-30T01:41:02.964956+0000","last_fullsized":"2024-03-30T01:41:02.964956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:28.675589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868455+0000","last_change":"2024-03-30T01:41:02.790100+0000","last_active":"2024-03-30T01:41:02.868455+0000","last_peered":"2024-03-30T01:41:02.868455+0000","last_clean":"2024-03-30T01:41:02.868455+0000","last_became_active":"2024-03-30T01:41:02.789660+0000","last_became_peered":"2024-03-30T01:41:02.789660+0000","last_unstale":"2024-03-30T01:41:02.868455+0000","last_undegraded":"2024-03-30T01:41:02.868455+0000","last_fullsized":"2024-03-30T01:41:02.868455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:22:34.848382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774905+0000","last_change":"2024-03-30T01:40:57.962434+0000","last_active":"2024-03-30T01:41:02.774905+0000","last_peered":"2024-03-30T01:41:02.774905+0000","last_clean":"2024-03-30T01:41:02.774905+0000","last_became_active":"2024-03-30T01:40:57.961987+0000","last_became_peered":"2024-03-30T01:40:57.961987+0000","last_unstale":"2024-03-30T01:41:02.774905+0000","last_undegraded":"2024-03-30T01:41:02.774905+0000","last_fullsized":"2024-03-30T01:41:02.774905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:42.056286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868476+0000","last_change":"2024-03-30T01:40:58.419381+0000","last_active":"2024-03-30T01:41:02.868476+0000","last_peered":"2024-03-30T01:41:02.868476+0000","last_clean":"2024-03-30T01:41:02.868476+0000","last_became_active":"2024-03-30T01:40:58.419079+0000","last_became_peered":"2024-03-30T01:40:58.419079+0000","last_unstale":"2024-03-30T01:41:02.868476+0000","last_undegraded":"2024-03-30T01:41:02.868476+0000","last_fullsized":"2024-03-30T01:41:02.868476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868479+0000","last_change":"2024-03-30T01:41:02.795347+0000","last_active":"2024-03-30T01:41:02.868479+0000","last_peered":"2024-03-30T01:41:02.868479+0000","last_clean":"2024-03-30T01:41:02.868479+0000","last_became_active":"2024-03-30T01:41:02.794998+0000","last_became_peered":"2024-03-30T01:41:02.794998+0000","last_unstale":"2024-03-30T01:41:02.868479+0000","last_undegraded":"2024-03-30T01:41:02.868479+0000","last_fullsized":"2024-03-30T01:41:02.868479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:19.384530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780572+0000","last_change":"2024-03-30T01:40:57.967506+0000","last_active":"2024-03-30T01:41:02.780572+0000","last_peered":"2024-03-30T01:41:02.780572+0000","last_clean":"2024-03-30T01:41:02.780572+0000","last_became_active":"2024-03-30T01:40:57.967219+0000","last_became_peered":"2024-03-30T01:40:57.967219+0000","last_unstale":"2024-03-30T01:41:02.780572+0000","last_undegraded":"2024-03-30T01:41:02.780572+0000","last_fullsized":"2024-03-30T01:41:02.780572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864547+0000","last_change":"2024-03-30T01:40:58.434945+0000","last_active":"2024-03-30T01:41:02.864547+0000","last_peered":"2024-03-30T01:41:02.864547+0000","last_clean":"2024-03-30T01:41:02.864547+0000","last_became_active":"2024-03-30T01:40:58.434344+0000","last_became_peered":"2024-03-30T01:40:58.434344+0000","last_unstale":"2024-03-30T01:41:02.864547+0000","last_undegraded":"2024-03-30T01:41:02.864547+0000","last_fullsized":"2024-03-30T01:41:02.864547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:43.058516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868577+0000","last_change":"2024-03-30T01:41:02.796689+0000","last_active":"2024-03-30T01:41:02.868577+0000","last_peered":"2024-03-30T01:41:02.868577+0000","last_clean":"2024-03-30T01:41:02.868577+0000","last_became_active":"2024-03-30T01:41:02.796285+0000","last_became_peered":"2024-03-30T01:41:02.796285+0000","last_unstale":"2024-03-30T01:41:02.868577+0000","last_undegraded":"2024-03-30T01:41:02.868577+0000","last_fullsized":"2024-03-30T01:41:02.868577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868364+0000","last_change":"2024-03-30T01:40:58.426059+0000","last_active":"2024-03-30T01:41:02.868364+0000","last_peered":"2024-03-30T01:41:02.868364+0000","last_clean":"2024-03-30T01:41:02.868364+0000","last_became_active":"2024-03-30T01:40:58.424722+0000","last_became_peered":"2024-03-30T01:40:58.424722+0000","last_unstale":"2024-03-30T01:41:02.868364+0000","last_undegraded":"2024-03-30T01:41:02.868364+0000","last_fullsized":"2024-03-30T01:41:02.868364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782192+0000","last_change":"2024-03-30T01:40:57.964575+0000","last_active":"2024-03-30T01:41:02.782192+0000","last_peered":"2024-03-30T01:41:02.782192+0000","last_clean":"2024-03-30T01:41:02.782192+0000","last_became_active":"2024-03-30T01:40:57.962016+0000","last_became_peered":"2024-03-30T01:40:57.962016+0000","last_unstale":"2024-03-30T01:41:02.782192+0000","last_undegraded":"2024-03-30T01:41:02.782192+0000","last_fullsized":"2024-03-30T01:41:02.782192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:04.688535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.974026+0000","last_change":"2024-03-30T01:41:02.969088+0000","last_active":"2024-03-30T01:41:02.974026+0000","last_peered":"2024-03-30T01:41:02.974026+0000","last_clean":"2024-03-30T01:41:02.974026+0000","last_became_active":"2024-03-30T01:41:02.968624+0000","last_became_peered":"2024-03-30T01:41:02.968624+0000","last_unstale":"2024-03-30T01:41:02.974026+0000","last_undegraded":"2024-03-30T01:41:02.974026+0000","last_fullsized":"2024-03-30T01:41:02.974026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:34.265123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868700+0000","last_change":"2024-03-30T01:40:58.429787+0000","last_active":"2024-03-30T01:41:02.868700+0000","last_peered":"2024-03-30T01:41:02.868700+0000","last_clean":"2024-03-30T01:41:02.868700+0000","last_became_active":"2024-03-30T01:40:58.429037+0000","last_became_peered":"2024-03-30T01:40:58.429037+0000","last_unstale":"2024-03-30T01:41:02.868700+0000","last_undegraded":"2024-03-30T01:41:02.868700+0000","last_fullsized":"2024-03-30T01:41:02.868700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:54.449729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780784+0000","last_change":"2024-03-30T01:40:57.954275+0000","last_active":"2024-03-30T01:41:02.780784+0000","last_peered":"2024-03-30T01:41:02.780784+0000","last_clean":"2024-03-30T01:41:02.780784+0000","last_became_active":"2024-03-30T01:40:57.953987+0000","last_became_peered":"2024-03-30T01:40:57.953987+0000","last_unstale":"2024-03-30T01:41:02.780784+0000","last_undegraded":"2024-03-30T01:41:02.780784+0000","last_fullsized":"2024-03-30T01:41:02.780784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964197+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:41:02.964197+0000","last_peered":"2024-03-30T01:41:02.964197+0000","last_clean":"2024-03-30T01:41:02.964197+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:41:02.964197+0000","last_undegraded":"2024-03-30T01:41:02.964197+0000","last_fullsized":"2024-03-30T01:41:02.964197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972468+0000","last_change":"2024-03-30T01:41:02.969064+0000","last_active":"2024-03-30T01:41:02.972468+0000","last_peered":"2024-03-30T01:41:02.972468+0000","last_clean":"2024-03-30T01:41:02.972468+0000","last_became_active":"2024-03-30T01:41:02.968333+0000","last_became_peered":"2024-03-30T01:41:02.968333+0000","last_unstale":"2024-03-30T01:41:02.972468+0000","last_undegraded":"2024-03-30T01:41:02.972468+0000","last_fullsized":"2024-03-30T01:41:02.972468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:34.173820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869624+0000","last_change":"2024-03-30T01:40:58.432432+0000","last_active":"2024-03-30T01:41:02.869624+0000","last_peered":"2024-03-30T01:41:02.869624+0000","last_clean":"2024-03-30T01:41:02.869624+0000","last_became_active":"2024-03-30T01:40:58.432158+0000","last_became_peered":"2024-03-30T01:40:58.432158+0000","last_unstale":"2024-03-30T01:41:02.869624+0000","last_undegraded":"2024-03-30T01:41:02.869624+0000","last_fullsized":"2024-03-30T01:41:02.869624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964478+0000","last_change":"2024-03-30T01:40:57.966931+0000","last_active":"2024-03-30T01:41:02.964478+0000","last_peered":"2024-03-30T01:41:02.964478+0000","last_clean":"2024-03-30T01:41:02.964478+0000","last_became_active":"2024-03-30T01:40:57.966589+0000","last_became_peered":"2024-03-30T01:40:57.966589+0000","last_unstale":"2024-03-30T01:41:02.964478+0000","last_undegraded":"2024-03-30T01:41:02.964478+0000","last_fullsized":"2024-03-30T01:41:02.964478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:15.934995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968515+0000","last_change":"2024-03-30T01:41:02.966273+0000","last_active":"2024-03-30T01:41:02.968515+0000","last_peered":"2024-03-30T01:41:02.968515+0000","last_clean":"2024-03-30T01:41:02.968515+0000","last_became_active":"2024-03-30T01:41:02.965857+0000","last_became_peered":"2024-03-30T01:41:02.965857+0000","last_unstale":"2024-03-30T01:41:02.968515+0000","last_undegraded":"2024-03-30T01:41:02.968515+0000","last_fullsized":"2024-03-30T01:41:02.968515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:58.270483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964736+0000","last_change":"2024-03-30T01:40:57.964957+0000","last_active":"2024-03-30T01:41:02.964736+0000","last_peered":"2024-03-30T01:41:02.964736+0000","last_clean":"2024-03-30T01:41:02.964736+0000","last_became_active":"2024-03-30T01:40:57.963985+0000","last_became_peered":"2024-03-30T01:40:57.963985+0000","last_unstale":"2024-03-30T01:41:02.964736+0000","last_undegraded":"2024-03-30T01:41:02.964736+0000","last_fullsized":"2024-03-30T01:41:02.964736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:05:32.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868496+0000","last_change":"2024-03-30T01:40:58.425295+0000","last_active":"2024-03-30T01:41:02.868496+0000","last_peered":"2024-03-30T01:41:02.868496+0000","last_clean":"2024-03-30T01:41:02.868496+0000","last_became_active":"2024-03-30T01:40:58.424967+0000","last_became_peered":"2024-03-30T01:40:58.424967+0000","last_unstale":"2024-03-30T01:41:02.868496+0000","last_undegraded":"2024-03-30T01:41:02.868496+0000","last_fullsized":"2024-03-30T01:41:02.868496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.975320+0000","last_change":"2024-03-30T01:41:02.970612+0000","last_active":"2024-03-30T01:41:02.975320+0000","last_peered":"2024-03-30T01:41:02.975320+0000","last_clean":"2024-03-30T01:41:02.975320+0000","last_became_active":"2024-03-30T01:41:02.969823+0000","last_became_peered":"2024-03-30T01:41:02.969823+0000","last_unstale":"2024-03-30T01:41:02.975320+0000","last_undegraded":"2024-03-30T01:41:02.975320+0000","last_fullsized":"2024-03-30T01:41:02.975320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:25:35.463484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T01:41:03.568727+0000","last_change":"2024-03-30T01:40:57.968553+0000","last_active":"2024-03-30T01:41:03.568727+0000","last_peered":"2024-03-30T01:41:03.568727+0000","last_clean":"2024-03-30T01:41:03.568727+0000","last_became_active":"2024-03-30T01:40:57.968315+0000","last_became_peered":"2024-03-30T01:40:57.968315+0000","last_unstale":"2024-03-30T01:41:03.568727+0000","last_undegraded":"2024-03-30T01:41:03.568727+0000","last_fullsized":"2024-03-30T01:41:03.568727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:34.662924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869030+0000","last_change":"2024-03-30T01:40:58.429594+0000","last_active":"2024-03-30T01:41:02.869030+0000","last_peered":"2024-03-30T01:41:02.869030+0000","last_clean":"2024-03-30T01:41:02.869030+0000","last_became_active":"2024-03-30T01:40:58.429296+0000","last_became_peered":"2024-03-30T01:40:58.429296+0000","last_unstale":"2024-03-30T01:41:02.869030+0000","last_undegraded":"2024-03-30T01:41:02.869030+0000","last_fullsized":"2024-03-30T01:41:02.869030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.974419+0000","last_change":"2024-03-30T01:41:02.970239+0000","last_active":"2024-03-30T01:41:02.974419+0000","last_peered":"2024-03-30T01:41:02.974419+0000","last_clean":"2024-03-30T01:41:02.974419+0000","last_became_active":"2024-03-30T01:41:02.969823+0000","last_became_peered":"2024-03-30T01:41:02.969823+0000","last_unstale":"2024-03-30T01:41:02.974419+0000","last_undegraded":"2024-03-30T01:41:02.974419+0000","last_fullsized":"2024-03-30T01:41:02.974419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:00.285894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770593+0000","last_change":"2024-03-30T01:40:57.963709+0000","last_active":"2024-03-30T01:41:02.770593+0000","last_peered":"2024-03-30T01:41:02.770593+0000","last_clean":"2024-03-30T01:41:02.770593+0000","last_became_active":"2024-03-30T01:40:57.963374+0000","last_became_peered":"2024-03-30T01:40:57.963374+0000","last_unstale":"2024-03-30T01:41:02.770593+0000","last_undegraded":"2024-03-30T01:41:02.770593+0000","last_fullsized":"2024-03-30T01:41:02.770593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869920+0000","last_change":"2024-03-30T01:40:58.430636+0000","last_active":"2024-03-30T01:41:02.869920+0000","last_peered":"2024-03-30T01:41:02.869920+0000","last_clean":"2024-03-30T01:41:02.869920+0000","last_became_active":"2024-03-30T01:40:58.430400+0000","last_became_peered":"2024-03-30T01:40:58.430400+0000","last_unstale":"2024-03-30T01:41:02.869920+0000","last_undegraded":"2024-03-30T01:41:02.869920+0000","last_fullsized":"2024-03-30T01:41:02.869920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:44:58.613670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864624+0000","last_change":"2024-03-30T01:41:02.788933+0000","last_active":"2024-03-30T01:41:02.864624+0000","last_peered":"2024-03-30T01:41:02.864624+0000","last_clean":"2024-03-30T01:41:02.864624+0000","last_became_active":"2024-03-30T01:41:02.788466+0000","last_became_peered":"2024-03-30T01:41:02.788466+0000","last_unstale":"2024-03-30T01:41:02.864624+0000","last_undegraded":"2024-03-30T01:41:02.864624+0000","last_fullsized":"2024-03-30T01:41:02.864624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964182+0000","last_change":"2024-03-30T01:40:57.962449+0000","last_active":"2024-03-30T01:41:02.964182+0000","last_peered":"2024-03-30T01:41:02.964182+0000","last_clean":"2024-03-30T01:41:02.964182+0000","last_became_active":"2024-03-30T01:40:57.962192+0000","last_became_peered":"2024-03-30T01:40:57.962192+0000","last_unstale":"2024-03-30T01:41:02.964182+0000","last_undegraded":"2024-03-30T01:41:02.964182+0000","last_fullsized":"2024-03-30T01:41:02.964182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:36.274096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864497+0000","last_change":"2024-03-30T01:40:58.420796+0000","last_active":"2024-03-30T01:41:02.864497+0000","last_peered":"2024-03-30T01:41:02.864497+0000","last_clean":"2024-03-30T01:41:02.864497+0000","last_became_active":"2024-03-30T01:40:58.420441+0000","last_became_peered":"2024-03-30T01:40:58.420441+0000","last_unstale":"2024-03-30T01:41:02.864497+0000","last_undegraded":"2024-03-30T01:41:02.864497+0000","last_fullsized":"2024-03-30T01:41:02.864497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:16.937225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.969938+0000","last_change":"2024-03-30T01:41:02.966462+0000","last_active":"2024-03-30T01:41:02.969938+0000","last_peered":"2024-03-30T01:41:02.969938+0000","last_clean":"2024-03-30T01:41:02.969938+0000","last_became_active":"2024-03-30T01:41:02.966056+0000","last_became_peered":"2024-03-30T01:41:02.966056+0000","last_unstale":"2024-03-30T01:41:02.969938+0000","last_undegraded":"2024-03-30T01:41:02.969938+0000","last_fullsized":"2024-03-30T01:41:02.969938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:29.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774672+0000","last_change":"2024-03-30T01:40:57.967648+0000","last_active":"2024-03-30T01:41:02.774672+0000","last_peered":"2024-03-30T01:41:02.774672+0000","last_clean":"2024-03-30T01:41:02.774672+0000","last_became_active":"2024-03-30T01:40:57.967362+0000","last_became_peered":"2024-03-30T01:40:57.967362+0000","last_unstale":"2024-03-30T01:41:02.774672+0000","last_undegraded":"2024-03-30T01:41:02.774672+0000","last_fullsized":"2024-03-30T01:41:02.774672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868291+0000","last_change":"2024-03-30T01:40:58.432296+0000","last_active":"2024-03-30T01:41:02.868291+0000","last_peered":"2024-03-30T01:41:02.868291+0000","last_clean":"2024-03-30T01:41:02.868291+0000","last_became_active":"2024-03-30T01:40:58.432045+0000","last_became_peered":"2024-03-30T01:40:58.432045+0000","last_unstale":"2024-03-30T01:41:02.868291+0000","last_undegraded":"2024-03-30T01:41:02.868291+0000","last_fullsized":"2024-03-30T01:41:02.868291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864586+0000","last_change":"2024-03-30T01:41:02.796892+0000","last_active":"2024-03-30T01:41:02.864586+0000","last_peered":"2024-03-30T01:41:02.864586+0000","last_clean":"2024-03-30T01:41:02.864586+0000","last_became_active":"2024-03-30T01:41:02.796347+0000","last_became_peered":"2024-03-30T01:41:02.796347+0000","last_unstale":"2024-03-30T01:41:02.864586+0000","last_undegraded":"2024-03-30T01:41:02.864586+0000","last_fullsized":"2024-03-30T01:41:02.864586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:20.847856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774635+0000","last_change":"2024-03-30T01:40:57.969391+0000","last_active":"2024-03-30T01:41:02.774635+0000","last_peered":"2024-03-30T01:41:02.774635+0000","last_clean":"2024-03-30T01:41:02.774635+0000","last_became_active":"2024-03-30T01:40:57.969107+0000","last_became_peered":"2024-03-30T01:40:57.969107+0000","last_unstale":"2024-03-30T01:41:02.774635+0000","last_undegraded":"2024-03-30T01:41:02.774635+0000","last_fullsized":"2024-03-30T01:41:02.774635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868769+0000","last_change":"2024-03-30T01:40:58.430179+0000","last_active":"2024-03-30T01:41:02.868769+0000","last_peered":"2024-03-30T01:41:02.868769+0000","last_clean":"2024-03-30T01:41:02.868769+0000","last_became_active":"2024-03-30T01:40:58.429919+0000","last_became_peered":"2024-03-30T01:40:58.429919+0000","last_unstale":"2024-03-30T01:41:02.868769+0000","last_undegraded":"2024-03-30T01:41:02.868769+0000","last_fullsized":"2024-03-30T01:41:02.868769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864611+0000","last_change":"2024-03-30T01:41:02.792167+0000","last_active":"2024-03-30T01:41:02.864611+0000","last_peered":"2024-03-30T01:41:02.864611+0000","last_clean":"2024-03-30T01:41:02.864611+0000","last_became_active":"2024-03-30T01:41:02.791805+0000","last_became_peered":"2024-03-30T01:41:02.791805+0000","last_unstale":"2024-03-30T01:41:02.864611+0000","last_undegraded":"2024-03-30T01:41:02.864611+0000","last_fullsized":"2024-03-30T01:41:02.864611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964766+0000","last_change":"2024-03-30T01:40:57.964733+0000","last_active":"2024-03-30T01:41:02.964766+0000","last_peered":"2024-03-30T01:41:02.964766+0000","last_clean":"2024-03-30T01:41:02.964766+0000","last_became_active":"2024-03-30T01:40:57.964430+0000","last_became_peered":"2024-03-30T01:40:57.964430+0000","last_unstale":"2024-03-30T01:41:02.964766+0000","last_undegraded":"2024-03-30T01:41:02.964766+0000","last_fullsized":"2024-03-30T01:41:02.964766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:42.056286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872480+0000","last_change":"2024-03-30T01:40:58.420811+0000","last_active":"2024-03-30T01:41:02.872480+0000","last_peered":"2024-03-30T01:41:02.872480+0000","last_clean":"2024-03-30T01:41:02.872480+0000","last_became_active":"2024-03-30T01:40:58.420491+0000","last_became_peered":"2024-03-30T01:40:58.420491+0000","last_unstale":"2024-03-30T01:41:02.872480+0000","last_undegraded":"2024-03-30T01:41:02.872480+0000","last_fullsized":"2024-03-30T01:41:02.872480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968342+0000","last_change":"2024-03-30T01:41:02.966275+0000","last_active":"2024-03-30T01:41:02.968342+0000","last_peered":"2024-03-30T01:41:02.968342+0000","last_clean":"2024-03-30T01:41:02.968342+0000","last_became_active":"2024-03-30T01:41:02.965934+0000","last_became_peered":"2024-03-30T01:41:02.965934+0000","last_unstale":"2024-03-30T01:41:02.968342+0000","last_undegraded":"2024-03-30T01:41:02.968342+0000","last_fullsized":"2024-03-30T01:41:02.968342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:39.954419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.783325+0000","last_change":"2024-03-30T01:40:57.962835+0000","last_active":"2024-03-30T01:41:02.783325+0000","last_peered":"2024-03-30T01:41:02.783325+0000","last_clean":"2024-03-30T01:41:02.783325+0000","last_became_active":"2024-03-30T01:40:57.961552+0000","last_became_peered":"2024-03-30T01:40:57.961552+0000","last_unstale":"2024-03-30T01:41:02.783325+0000","last_undegraded":"2024-03-30T01:41:02.783325+0000","last_fullsized":"2024-03-30T01:41:02.783325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964874+0000","last_change":"2024-03-30T01:40:58.432622+0000","last_active":"2024-03-30T01:41:02.964874+0000","last_peered":"2024-03-30T01:41:02.964874+0000","last_clean":"2024-03-30T01:41:02.964874+0000","last_became_active":"2024-03-30T01:40:58.432336+0000","last_became_peered":"2024-03-30T01:40:58.432336+0000","last_unstale":"2024-03-30T01:41:02.964874+0000","last_undegraded":"2024-03-30T01:41:02.964874+0000","last_fullsized":"2024-03-30T01:41:02.964874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:24:50.793081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869687+0000","last_change":"2024-03-30T01:41:02.799702+0000","last_active":"2024-03-30T01:41:02.869687+0000","last_peered":"2024-03-30T01:41:02.869687+0000","last_clean":"2024-03-30T01:41:02.869687+0000","last_became_active":"2024-03-30T01:41:02.799189+0000","last_became_peered":"2024-03-30T01:41:02.799189+0000","last_unstale":"2024-03-30T01:41:02.869687+0000","last_undegraded":"2024-03-30T01:41:02.869687+0000","last_fullsized":"2024-03-30T01:41:02.869687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.777920+0000","last_change":"2024-03-30T01:40:57.964577+0000","last_active":"2024-03-30T01:41:02.777920+0000","last_peered":"2024-03-30T01:41:02.777920+0000","last_clean":"2024-03-30T01:41:02.777920+0000","last_became_active":"2024-03-30T01:40:57.964277+0000","last_became_peered":"2024-03-30T01:40:57.964277+0000","last_unstale":"2024-03-30T01:41:02.777920+0000","last_undegraded":"2024-03-30T01:41:02.777920+0000","last_fullsized":"2024-03-30T01:41:02.777920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:15.934995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868392+0000","last_change":"2024-03-30T01:40:58.428923+0000","last_active":"2024-03-30T01:41:02.868392+0000","last_peered":"2024-03-30T01:41:02.868392+0000","last_clean":"2024-03-30T01:41:02.868392+0000","last_became_active":"2024-03-30T01:40:58.428204+0000","last_became_peered":"2024-03-30T01:40:58.428204+0000","last_unstale":"2024-03-30T01:41:02.868392+0000","last_undegraded":"2024-03-30T01:41:02.868392+0000","last_fullsized":"2024-03-30T01:41:02.868392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.970293+0000","last_change":"2024-03-30T01:41:02.968043+0000","last_active":"2024-03-30T01:41:02.970293+0000","last_peered":"2024-03-30T01:41:02.970293+0000","last_clean":"2024-03-30T01:41:02.970293+0000","last_became_active":"2024-03-30T01:41:02.967586+0000","last_became_peered":"2024-03-30T01:41:02.967586+0000","last_unstale":"2024-03-30T01:41:02.970293+0000","last_undegraded":"2024-03-30T01:41:02.970293+0000","last_fullsized":"2024-03-30T01:41:02.970293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:29.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782245+0000","last_change":"2024-03-30T01:40:57.962668+0000","last_active":"2024-03-30T01:41:02.782245+0000","last_peered":"2024-03-30T01:41:02.782245+0000","last_clean":"2024-03-30T01:41:02.782245+0000","last_became_active":"2024-03-30T01:40:57.962255+0000","last_became_peered":"2024-03-30T01:40:57.962255+0000","last_unstale":"2024-03-30T01:41:02.782245+0000","last_undegraded":"2024-03-30T01:41:02.782245+0000","last_fullsized":"2024-03-30T01:41:02.782245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965085+0000","last_change":"2024-03-30T01:40:58.441825+0000","last_active":"2024-03-30T01:41:02.965085+0000","last_peered":"2024-03-30T01:41:02.965085+0000","last_clean":"2024-03-30T01:41:02.965085+0000","last_became_active":"2024-03-30T01:40:58.441575+0000","last_became_peered":"2024-03-30T01:40:58.441575+0000","last_unstale":"2024-03-30T01:41:02.965085+0000","last_undegraded":"2024-03-30T01:41:02.965085+0000","last_fullsized":"2024-03-30T01:41:02.965085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:51.791864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872404+0000","last_change":"2024-03-30T01:40:58.425945+0000","last_active":"2024-03-30T01:41:02.872404+0000","last_peered":"2024-03-30T01:41:02.872404+0000","last_clean":"2024-03-30T01:41:02.872404+0000","last_became_active":"2024-03-30T01:40:58.425607+0000","last_became_peered":"2024-03-30T01:40:58.425607+0000","last_unstale":"2024-03-30T01:41:02.872404+0000","last_undegraded":"2024-03-30T01:41:02.872404+0000","last_fullsized":"2024-03-30T01:41:02.872404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868697+0000","last_change":"2024-03-30T01:40:58.427012+0000","last_active":"2024-03-30T01:41:02.868697+0000","last_peered":"2024-03-30T01:41:02.868697+0000","last_clean":"2024-03-30T01:41:02.868697+0000","last_became_active":"2024-03-30T01:40:58.426750+0000","last_became_peered":"2024-03-30T01:40:58.426750+0000","last_unstale":"2024-03-30T01:41:02.868697+0000","last_undegraded":"2024-03-30T01:41:02.868697+0000","last_fullsized":"2024-03-30T01:41:02.868697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:37.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868317+0000","last_change":"2024-03-30T01:40:58.426120+0000","last_active":"2024-03-30T01:41:02.868317+0000","last_peered":"2024-03-30T01:41:02.868317+0000","last_clean":"2024-03-30T01:41:02.868317+0000","last_became_active":"2024-03-30T01:40:58.425776+0000","last_became_peered":"2024-03-30T01:40:58.425776+0000","last_unstale":"2024-03-30T01:41:02.868317+0000","last_undegraded":"2024-03-30T01:41:02.868317+0000","last_fullsized":"2024-03-30T01:41:02.868317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869771+0000","last_change":"2024-03-30T01:40:58.436516+0000","last_active":"2024-03-30T01:41:02.869771+0000","last_peered":"2024-03-30T01:41:02.869771+0000","last_clean":"2024-03-30T01:41:02.869771+0000","last_became_active":"2024-03-30T01:40:58.436247+0000","last_became_peered":"2024-03-30T01:40:58.436247+0000","last_unstale":"2024-03-30T01:41:02.869771+0000","last_undegraded":"2024-03-30T01:41:02.869771+0000","last_fullsized":"2024-03-30T01:41:02.869771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868868+0000","last_change":"2024-03-30T01:40:58.415507+0000","last_active":"2024-03-30T01:41:02.868868+0000","last_peered":"2024-03-30T01:41:02.868868+0000","last_clean":"2024-03-30T01:41:02.868868+0000","last_became_active":"2024-03-30T01:40:58.415197+0000","last_became_peered":"2024-03-30T01:40:58.415197+0000","last_unstale":"2024-03-30T01:41:02.868868+0000","last_undegraded":"2024-03-30T01:41:02.868868+0000","last_fullsized":"2024-03-30T01:41:02.868868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868506+0000","last_change":"2024-03-30T01:40:58.425962+0000","last_active":"2024-03-30T01:41:02.868506+0000","last_peered":"2024-03-30T01:41:02.868506+0000","last_clean":"2024-03-30T01:41:02.868506+0000","last_became_active":"2024-03-30T01:40:58.425667+0000","last_became_peered":"2024-03-30T01:40:58.425667+0000","last_unstale":"2024-03-30T01:41:02.868506+0000","last_undegraded":"2024-03-30T01:41:02.868506+0000","last_fullsized":"2024-03-30T01:41:02.868506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872585+0000","last_change":"2024-03-30T01:40:58.429802+0000","last_active":"2024-03-30T01:41:02.872585+0000","last_peered":"2024-03-30T01:41:02.872585+0000","last_clean":"2024-03-30T01:41:02.872585+0000","last_became_active":"2024-03-30T01:40:58.429296+0000","last_became_peered":"2024-03-30T01:40:58.429296+0000","last_unstale":"2024-03-30T01:41:02.872585+0000","last_undegraded":"2024-03-30T01:41:02.872585+0000","last_fullsized":"2024-03-30T01:41:02.872585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869051+0000","last_change":"2024-03-30T01:40:58.420564+0000","last_active":"2024-03-30T01:41:02.869051+0000","last_peered":"2024-03-30T01:41:02.869051+0000","last_clean":"2024-03-30T01:41:02.869051+0000","last_became_active":"2024-03-30T01:40:58.420286+0000","last_became_peered":"2024-03-30T01:40:58.420286+0000","last_unstale":"2024-03-30T01:41:02.869051+0000","last_undegraded":"2024-03-30T01:41:02.869051+0000","last_fullsized":"2024-03-30T01:41:02.869051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868288+0000","last_change":"2024-03-30T01:40:58.418904+0000","last_active":"2024-03-30T01:41:02.868288+0000","last_peered":"2024-03-30T01:41:02.868288+0000","last_clean":"2024-03-30T01:41:02.868288+0000","last_became_active":"2024-03-30T01:40:58.418351+0000","last_became_peered":"2024-03-30T01:40:58.418351+0000","last_unstale":"2024-03-30T01:41:02.868288+0000","last_undegraded":"2024-03-30T01:41:02.868288+0000","last_fullsized":"2024-03-30T01:41:02.868288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868595+0000","last_change":"2024-03-30T01:40:58.416220+0000","last_active":"2024-03-30T01:41:02.868595+0000","last_peered":"2024-03-30T01:41:02.868595+0000","last_clean":"2024-03-30T01:41:02.868595+0000","last_became_active":"2024-03-30T01:40:58.415937+0000","last_became_peered":"2024-03-30T01:40:58.415937+0000","last_unstale":"2024-03-30T01:41:02.868595+0000","last_undegraded":"2024-03-30T01:41:02.868595+0000","last_fullsized":"2024-03-30T01:41:02.868595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868332+0000","last_change":"2024-03-30T01:40:58.433268+0000","last_active":"2024-03-30T01:41:02.868332+0000","last_peered":"2024-03-30T01:41:02.868332+0000","last_clean":"2024-03-30T01:41:02.868332+0000","last_became_active":"2024-03-30T01:40:58.432963+0000","last_became_peered":"2024-03-30T01:40:58.432963+0000","last_unstale":"2024-03-30T01:41:02.868332+0000","last_undegraded":"2024-03-30T01:41:02.868332+0000","last_fullsized":"2024-03-30T01:41:02.868332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869784+0000","last_change":"2024-03-30T01:40:58.440462+0000","last_active":"2024-03-30T01:41:02.869784+0000","last_peered":"2024-03-30T01:41:02.869784+0000","last_clean":"2024-03-30T01:41:02.869784+0000","last_became_active":"2024-03-30T01:40:58.440144+0000","last_became_peered":"2024-03-30T01:40:58.440144+0000","last_unstale":"2024-03-30T01:41:02.869784+0000","last_undegraded":"2024-03-30T01:41:02.869784+0000","last_fullsized":"2024-03-30T01:41:02.869784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872499+0000","last_change":"2024-03-30T01:40:58.422820+0000","last_active":"2024-03-30T01:41:02.872499+0000","last_peered":"2024-03-30T01:41:02.872499+0000","last_clean":"2024-03-30T01:41:02.872499+0000","last_became_active":"2024-03-30T01:40:58.420834+0000","last_became_peered":"2024-03-30T01:40:58.420834+0000","last_unstale":"2024-03-30T01:41:02.872499+0000","last_undegraded":"2024-03-30T01:41:02.872499+0000","last_fullsized":"2024-03-30T01:41:02.872499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4453,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":856064,"data_stored":689817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:41:18.426 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T01:41:18.426 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-30T01:41:18.427 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-30T01:41:18.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-30T01:41:18.427 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-30T01:41:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.592+0000 7f184bee1700 1 Processor -- start 2024-03-30T01:41:18.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.592+0000 7f184bee1700 1 -- start start 2024-03-30T01:41:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.593+0000 7f184bee1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1844156290 0x7f184406cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.593+0000 7f184bee1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18440a7690 0x7f184406d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.593+0000 7f184bee1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440c7210 con 0x7f18440c2060 2024-03-30T01:41:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.593+0000 7f184bee1700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18440c7a90 con 0x7f18440a7690 2024-03-30T01:41:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.593+0000 7f184bee1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1844159b50 con 0x7f1844156290 2024-03-30T01:41:18.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.593+0000 7f184947c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18440a7690 0x7f184406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.593+0000 7f184947c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18440a7690 0x7f184406d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36424/0 (socket says 172.21.15.31:36424) 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.593+0000 7f184947c700 1 -- 172.21.15.31:0/1750372857 learned_addr learned my addr 172.21.15.31:0/1750372857 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.593+0000 7f1849c7d700 1 --2- 172.21.15.31:0/1750372857 >> v2:172.21.15.59:3300/0 conn(0x7f1844156290 0x7f184406cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.594+0000 7f184947c700 1 -- 172.21.15.31:0/1750372857 >> v2:172.21.15.59:3300/0 conn(0x7f1844156290 msgr2=0x7f184406cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.594+0000 7f184947c700 1 --2- 172.21.15.31:0/1750372857 >> v2:172.21.15.59:3300/0 conn(0x7f1844156290 0x7f184406cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.594+0000 7f184947c700 1 -- 172.21.15.31:0/1750372857 >> v1:172.21.15.59:6789/0 conn(0x7f18440c2060 legacy=0x7f18440c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.594+0000 7f184947c700 1 -- 172.21.15.31:0/1750372857 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440c74d0 con 0x7f18440a7690 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.594+0000 7f184947c700 1 --2- 172.21.15.31:0/1750372857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440a7690 0x7f184406d270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f18440c5660 tx=0x7f1834009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3b31de09a257f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.594+0000 7f1848c7b700 1 -- 172.21.15.31:0/1750372857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834011830 con 0x7f18440a7690 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.594+0000 7f1848c7b700 1 -- 172.21.15.31:0/1750372857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1834011990 con 0x7f18440a7690 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.594+0000 7f1848c7b700 1 -- 172.21.15.31:0/1750372857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834011d20 con 0x7f18440a7690 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.595+0000 7f184bee1700 1 -- 172.21.15.31:0/1750372857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440a7690 msgr2=0x7f184406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.595+0000 7f184bee1700 1 --2- 172.21.15.31:0/1750372857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440a7690 0x7f184406d270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f18440c5660 tx=0x7f1834009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.595+0000 7f184bee1700 1 -- 172.21.15.31:0/1750372857 shutdown_connections 2024-03-30T01:41:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.595+0000 7f184bee1700 1 --2- 172.21.15.31:0/1750372857 >> v2:172.21.15.59:3300/0 conn(0x7f1844156290 0x7f184406cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.595+0000 7f184bee1700 1 --2- 172.21.15.31:0/1750372857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440a7690 0x7f184406d270 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.595+0000 7f184bee1700 1 -- 172.21.15.31:0/1750372857 >> 172.21.15.31:0/1750372857 conn(0x7f184406ad30 msgr2=0x7f18440a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.595+0000 7f184bee1700 1 -- 172.21.15.31:0/1750372857 shutdown_connections 2024-03-30T01:41:18.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.595+0000 7f184bee1700 1 -- 172.21.15.31:0/1750372857 wait complete. 2024-03-30T01:41:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.596+0000 7f184bee1700 1 Processor -- start 2024-03-30T01:41:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.596+0000 7f184bee1700 1 -- start start 2024-03-30T01:41:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184bee1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18440a7690 0x7f18440ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184bee1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440c2060 0x7f184415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184bee1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1844156290 0x7f184415b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184bee1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440aac20 con 0x7f18440c2060 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184bee1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184415c7a0 con 0x7f1844156290 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184bee1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184415cfa0 con 0x7f18440a7690 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184947c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440c2060 0x7f184415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184947c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440c2060 0x7f184415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36426/0 (socket says 172.21.15.31:36426) 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184947c700 1 -- 172.21.15.31:0/136827305 learned_addr learned my addr 172.21.15.31:0/136827305 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184a47e700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1844156290 0x7f184415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f1849c7d700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18440a7690 0x7f18440ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184947c700 1 -- 172.21.15.31:0/136827305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18440a7690 msgr2=0x7f18440ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184947c700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18440a7690 0x7f18440ac790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184947c700 1 -- 172.21.15.31:0/136827305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1844156290 msgr2=0x7f184415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184947c700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1844156290 0x7f184415b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184947c700 1 -- 172.21.15.31:0/136827305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834009550 con 0x7f18440c2060 2024-03-30T01:41:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f1849c7d700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18440a7690 0x7f18440ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.597+0000 7f184947c700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440c2060 0x7f184415ab40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1834000c00 tx=0x7f1834004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.598+0000 7f183a7fc700 1 -- 172.21.15.31:0/136827305 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834004570 con 0x7f18440c2060 2024-03-30T01:41:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.598+0000 7f183a7fc700 1 -- 172.21.15.31:0/136827305 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183401f070 con 0x7f18440c2060 2024-03-30T01:41:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.598+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184415a060 con 0x7f18440c2060 2024-03-30T01:41:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.598+0000 7f183a7fc700 1 -- 172.21.15.31:0/136827305 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18340038e0 con 0x7f18440c2060 2024-03-30T01:41:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.598+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184415a620 con 0x7f18440c2060 2024-03-30T01:41:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.599+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184404ea80 con 0x7f18440c2060 2024-03-30T01:41:18.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.599+0000 7f183a7fc700 1 -- 172.21.15.31:0/136827305 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1834021030 con 0x7f18440c2060 2024-03-30T01:41:18.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.600+0000 7f183a7fc700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f182806ac90 0x7f182806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:18.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.600+0000 7f1849c7d700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f182806ac90 0x7f182806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:18.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.600+0000 7f183a7fc700 1 -- 172.21.15.31:0/136827305 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f183408aa60 con 0x7f18440c2060 2024-03-30T01:41:18.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.600+0000 7f1849c7d700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f182806ac90 0x7f182806d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f184406fb40 tx=0x7f1840008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:18.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.604+0000 7f183a7fc700 1 -- 172.21.15.31:0/136827305 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1834055a80 con 0x7f18440c2060 2024-03-30T01:41:18.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.889+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f18440c29f0 con 0x7f18440c2060 2024-03-30T01:41:18.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.893+0000 7f183a7fc700 1 -- 172.21.15.31:0/136827305 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170926 (secure 0 0 0) 0x7f1834015020 con 0x7f18440c2060 2024-03-30T01:41:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f182806ac90 msgr2=0x7f182806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f182806ac90 0x7f182806d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f184406fb40 tx=0x7f1840008040 comp rx=0 tx=0).stop 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440c2060 msgr2=0x7f184415ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440c2060 0x7f184415ab40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1834000c00 tx=0x7f1834004750 comp rx=0 tx=0).stop 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 shutdown_connections 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f182806ac90 0x7f182806d150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1844156290 0x7f184415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18440a7690 0x7f18440ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 --2- 172.21.15.31:0/136827305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18440c2060 0x7f184415ab40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:18.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 >> 172.21.15.31:0/136827305 conn(0x7f184406ad30 msgr2=0x7f18440c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:18.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 shutdown_connections 2024-03-30T01:41:18.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:18.896+0000 7f184bee1700 1 -- 172.21.15.31:0/136827305 wait complete. 2024-03-30T01:41:18.916 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":47081},{"type":"v1","addr":"172.21.15.31:6833","nonce":47081}]},"active_addr":"172.21.15.31:6833/47081","active_change":"2024-03-30T01:40:41.790275+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2675543730}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1352240279}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":310271170}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1311059122}]}]} 2024-03-30T01:41:18.921 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-30T01:41:18.921 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-30T01:41:18.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:19.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.054+0000 7f6d721f0700 1 Processor -- start 2024-03-30T01:41:19.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.054+0000 7f6d721f0700 1 -- start start 2024-03-30T01:41:19.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d721f0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6d6c06a700 0x7f6d6c0c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d721f0700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6c1547d0 con 0x7f6d6c0c3180 2024-03-30T01:41:19.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d721f0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6c154a90 con 0x7f6d6c0c4710 2024-03-30T01:41:19.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d721f0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6c154d70 con 0x7f6d6c06a700 2024-03-30T01:41:19.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6d6c0c3180 0x7f6d6c06eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34452/0 (socket says 172.21.15.31:34452) 2024-03-30T01:41:19.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6b7fe700 1 -- 172.21.15.31:0/2429396386 learned_addr learned my addr 172.21.15.31:0/2429396386 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:19.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170155882 0 0) 0x7f6d6c1547d0 con 0x7f6d6c0c3180 2024-03-30T01:41:19.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6affd700 1 --2- 172.21.15.31:0/2429396386 >> v2:172.21.15.59:3300/0 conn(0x7f6d6c06a700 0x7f6d6c0c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d540035b0 con 0x7f6d6c0c3180 2024-03-30T01:41:19.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373420889 0 0) 0x7f6d540035b0 con 0x7f6d6c0c3180 2024-03-30T01:41:19.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 >> v2:172.21.15.59:3300/0 conn(0x7f6d6c06a700 msgr2=0x7f6d6c0c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 --2- 172.21.15.31:0/2429396386 >> v2:172.21.15.59:3300/0 conn(0x7f6d6c06a700 0x7f6d6c0c0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 >> v1:172.21.15.59:6789/0 conn(0x7f6d6c0c4710 legacy=0x7f6d6c0c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c155050 con 0x7f6d6c0c3180 2024-03-30T01:41:19.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f6d5c001670 con 0x7f6d6c0c3180 2024-03-30T01:41:19.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 >> v1:172.21.15.31:6789/0 conn(0x7f6d6c0c3180 legacy=0x7f6d6c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d54003a50 0x7f6d54003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.055+0000 7f6d6a7fc700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d54004850 0x7f6d54004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6a7fc700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c154d70 con 0x7f6d54005210 2024-03-30T01:41:19.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5400c110 con 0x7f6d54003a50 2024-03-30T01:41:19.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5400c2d0 con 0x7f6d54004850 2024-03-30T01:41:19.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6bfff700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6b7fe700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d54003a50 0x7f6d54003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6affd700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d54004850 0x7f6d54004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6bfff700 1 -- 172.21.15.31:0/2429396386 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d54004850 msgr2=0x7f6d54004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6bfff700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d54004850 0x7f6d54004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6bfff700 1 -- 172.21.15.31:0/2429396386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d54003a50 msgr2=0x7f6d54003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6bfff700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d54003a50 0x7f6d54003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6bfff700 1 -- 172.21.15.31:0/2429396386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d540035b0 con 0x7f6d54005210 2024-03-30T01:41:19.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6bfff700 1 -- 172.21.15.31:0/2429396386 reap_dead start 2024-03-30T01:41:19.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6bfff700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d5400ba10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400c160 tx=0x7f6d54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766dbcc46353636 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64001db0 con 0x7f6d54005210 2024-03-30T01:41:19.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400cca0 con 0x7f6d54005210 2024-03-30T01:41:19.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.056+0000 7f6d6a7fc700 1 -- 172.21.15.31:0/2429396386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64003c40 con 0x7f6d54005210 2024-03-30T01:41:19.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.057+0000 7f6d721f0700 1 -- 172.21.15.31:0/2429396386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 msgr2=0x7f6d5400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.057+0000 7f6d721f0700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d5400ba10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400c160 tx=0x7f6d54005880 comp rx=0 tx=0).stop 2024-03-30T01:41:19.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.057+0000 7f6d721f0700 1 -- 172.21.15.31:0/2429396386 shutdown_connections 2024-03-30T01:41:19.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.057+0000 7f6d721f0700 1 --2- 172.21.15.31:0/2429396386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d5400ba10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.057+0000 7f6d721f0700 1 -- 172.21.15.31:0/2429396386 >> 172.21.15.31:0/2429396386 conn(0x7f6d6c0c3a60 msgr2=0x7f6d6c0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:19.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.057+0000 7f6d721f0700 1 -- 172.21.15.31:0/2429396386 shutdown_connections 2024-03-30T01:41:19.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.057+0000 7f6d721f0700 1 -- 172.21.15.31:0/2429396386 wait complete. 2024-03-30T01:41:19.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.058+0000 7f6d721f0700 1 Processor -- start 2024-03-30T01:41:19.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.058+0000 7f6d721f0700 1 -- start start 2024-03-30T01:41:19.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d721f0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d6c158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d721f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d6c147700 0x7f6d6c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d721f0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d6c14c7a0 0x7f6d6c14cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d721f0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c158ac0 con 0x7f6d54005210 2024-03-30T01:41:19.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d721f0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c14d160 con 0x7f6d6c147700 2024-03-30T01:41:19.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d721f0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c156700 con 0x7f6d6c14c7a0 2024-03-30T01:41:19.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d6c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d6c158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36436/0 (socket says 172.21.15.31:36436) 2024-03-30T01:41:19.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d6b7fe700 1 -- 172.21.15.31:0/3112449804 learned_addr learned my addr 172.21.15.31:0/3112449804 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:19.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d6affd700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d6c147700 0x7f6d6c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.059+0000 7f6d6bfff700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d6c14c7a0 0x7f6d6c14cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d6b7fe700 1 -- 172.21.15.31:0/3112449804 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d6c14c7a0 msgr2=0x7f6d6c14cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d6b7fe700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d6c14c7a0 0x7f6d6c14cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d6b7fe700 1 -- 172.21.15.31:0/3112449804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d6c147700 msgr2=0x7f6d6c14d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d6b7fe700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d6c147700 0x7f6d6c14d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d6b7fe700 1 -- 172.21.15.31:0/3112449804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c1569e0 con 0x7f6d54005210 2024-03-30T01:41:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d6b7fe700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d6c158580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c001d70 tx=0x7f6d5c007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d53fff700 1 -- 172.21.15.31:0/3112449804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c012560 con 0x7f6d54005210 2024-03-30T01:41:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d53fff700 1 -- 172.21.15.31:0/3112449804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c0126e0 con 0x7f6d54005210 2024-03-30T01:41:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d53fff700 1 -- 172.21.15.31:0/3112449804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c012a90 con 0x7f6d54005210 2024-03-30T01:41:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c156cc0 con 0x7f6d54005210 2024-03-30T01:41:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.060+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c157280 con 0x7f6d54005210 2024-03-30T01:41:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.061+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c157410 con 0x7f6d54005210 2024-03-30T01:41:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.061+0000 7f6d53fff700 1 -- 172.21.15.31:0/3112449804 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6d5c012bf0 con 0x7f6d54005210 2024-03-30T01:41:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.062+0000 7f6d53fff700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6d5806acc0 0x7f6d5806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.062+0000 7f6d53fff700 1 -- 172.21.15.31:0/3112449804 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6d5c0894b0 con 0x7f6d54005210 2024-03-30T01:41:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.062+0000 7f6d6affd700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6d5806acc0 0x7f6d5806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.062+0000 7f6d6affd700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6d5806acc0 0x7f6d5806d180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c1ea9b0 tx=0x7f6d60009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.067+0000 7f6d53fff700 1 -- 172.21.15.31:0/3112449804 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6d5c054430 con 0x7f6d54005210 2024-03-30T01:41:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.285+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d6c04ea80 con 0x7f6d54005210 2024-03-30T01:41:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.286+0000 7f6d53fff700 1 -- 172.21.15.31:0/3112449804 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f6d5c057a50 con 0x7f6d54005210 2024-03-30T01:41:19.293 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:19.293 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.288+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6d5806acc0 msgr2=0x7f6d5806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.288+0000 7f6d721f0700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6d5806acc0 0x7f6d5806d180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c1ea9b0 tx=0x7f6d60009210 comp rx=0 tx=0).stop 2024-03-30T01:41:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.288+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 msgr2=0x7f6d6c158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.288+0000 7f6d721f0700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d6c158580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c001d70 tx=0x7f6d5c007570 comp rx=0 tx=0).stop 2024-03-30T01:41:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.288+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 shutdown_connections 2024-03-30T01:41:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.288+0000 7f6d721f0700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d54005210 0x7f6d6c158580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.288+0000 7f6d721f0700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6d5806acc0 0x7f6d5806d180 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.288+0000 7f6d721f0700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d6c14c7a0 0x7f6d6c14cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.289+0000 7f6d721f0700 1 --2- 172.21.15.31:0/3112449804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d6c147700 0x7f6d6c14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.289+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 >> 172.21.15.31:0/3112449804 conn(0x7f6d6c0c3a60 msgr2=0x7f6d6c0c4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.289+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 shutdown_connections 2024-03-30T01:41:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.289+0000 7f6d721f0700 1 -- 172.21.15.31:0/3112449804 wait complete. 2024-03-30T01:41:19.309 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-30T01:41:19.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.439+0000 7f88cdc31700 1 Processor -- start 2024-03-30T01:41:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.439+0000 7f88cdc31700 1 -- start start 2024-03-30T01:41:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.440+0000 7f88cdc31700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f88c814e5f0 0x7f88c80c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.440+0000 7f88cdc31700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c8072160 con 0x7f88c80c4100 2024-03-30T01:41:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.440+0000 7f88cdc31700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c80c2bd0 con 0x7f88c8071240 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.440+0000 7f88cdc31700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c80c2e90 con 0x7f88c814e5f0 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.440+0000 7f88c7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f88c80c4100 0x7f88c8154fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34464/0 (socket says 172.21.15.31:34464) 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.440+0000 7f88c7fff700 1 -- 172.21.15.31:0/2320036406 learned_addr learned my addr 172.21.15.31:0/2320036406 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.440+0000 7f88c77fe700 1 --2- 172.21.15.31:0/2320036406 >> v2:172.21.15.59:3300/0 conn(0x7f88c814e5f0 0x7f88c80c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.440+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025094410 0 0) 0x7f88c8072160 con 0x7f88c80c4100 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.440+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b40035b0 con 0x7f88c80c4100 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 973867121 0 0) 0x7f88b40035b0 con 0x7f88c80c4100 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 >> v2:172.21.15.59:3300/0 conn(0x7f88c814e5f0 msgr2=0x7f88c80c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 --2- 172.21.15.31:0/2320036406 >> v2:172.21.15.59:3300/0 conn(0x7f88c814e5f0 0x7f88c80c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 >> v1:172.21.15.59:6789/0 conn(0x7f88c8071240 legacy=0x7f88c8154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c80723b0 con 0x7f88c80c4100 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f88c00019f0 con 0x7f88c80c4100 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c77fe700 1 --2- 172.21.15.31:0/2320036406 >> v2:172.21.15.59:3300/0 conn(0x7f88c814e5f0 0x7f88c80c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:19.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 >> v1:172.21.15.31:6789/0 conn(0x7f88c80c4100 legacy=0x7f88c8154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88b4003a00 0x7f88b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b4004850 0x7f88b4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4005330 0x7f88b400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80c2e90 con 0x7f88b4004850 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b400c200 con 0x7f88b4003a00 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b400c3c0 con 0x7f88b4005330 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b4004850 0x7f88b4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c77fe700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88b4003a00 0x7f88b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c7fff700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4005330 0x7f88b400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c6ffd700 1 -- 172.21.15.31:0/2320036406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4005330 msgr2=0x7f88b400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4005330 0x7f88b400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c6ffd700 1 -- 172.21.15.31:0/2320036406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88b4003a00 msgr2=0x7f88b4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88b4003a00 0x7f88b4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c6ffd700 1 -- 172.21.15.31:0/2320036406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b40035b0 con 0x7f88b4004850 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c7fff700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4005330 0x7f88b400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c7fff700 1 -- 172.21.15.31:0/2320036406 reap_dead start 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.441+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b4004850 0x7f88b4004cd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88b0007fd0 tx=0x7f88b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1a49ce095f8610 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00052d0 con 0x7f88b4004850 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b0005430 con 0x7f88b4004850 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88c67fc700 1 -- 172.21.15.31:0/2320036406 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00038c0 con 0x7f88b4004850 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88cdc31700 1 -- 172.21.15.31:0/2320036406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b4004850 msgr2=0x7f88b4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88cdc31700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b4004850 0x7f88b4004cd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88b0007fd0 tx=0x7f88b4005880 comp rx=0 tx=0).stop 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88cdc31700 1 -- 172.21.15.31:0/2320036406 shutdown_connections 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88cdc31700 1 --2- 172.21.15.31:0/2320036406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b4004850 0x7f88b4004cd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88cdc31700 1 -- 172.21.15.31:0/2320036406 >> 172.21.15.31:0/2320036406 conn(0x7f88c806ba20 msgr2=0x7f88c806be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88cdc31700 1 -- 172.21.15.31:0/2320036406 shutdown_connections 2024-03-30T01:41:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.442+0000 7f88cdc31700 1 -- 172.21.15.31:0/2320036406 wait complete. 2024-03-30T01:41:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.444+0000 7f88cdc31700 1 Processor -- start 2024-03-30T01:41:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.444+0000 7f88cdc31700 1 -- start start 2024-03-30T01:41:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.444+0000 7f88cdc31700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4004850 0x7f88c8153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.444+0000 7f88cdc31700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8141620 0x7f88c81e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.444+0000 7f88cdc31700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c81e75f0 0x7f88c81428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88cdc31700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c81ed2f0 con 0x7f88c8141620 2024-03-30T01:41:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88cdc31700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c81ed5b0 con 0x7f88c81e75f0 2024-03-30T01:41:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88cdc31700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80c6680 con 0x7f88b4004850 2024-03-30T01:41:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8141620 0x7f88c81e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8141620 0x7f88c81e70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36454/0 (socket says 172.21.15.31:36454) 2024-03-30T01:41:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c6ffd700 1 -- 172.21.15.31:0/3802333840 learned_addr learned my addr 172.21.15.31:0/3802333840 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c77fe700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4004850 0x7f88c8153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c7fff700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c81e75f0 0x7f88c81428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c6ffd700 1 -- 172.21.15.31:0/3802333840 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4004850 msgr2=0x7f88c8153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4004850 0x7f88c8153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c6ffd700 1 -- 172.21.15.31:0/3802333840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c81e75f0 msgr2=0x7f88c81428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c81e75f0 0x7f88c81428c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c6ffd700 1 -- 172.21.15.31:0/3802333840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c81420c0 con 0x7f88c8141620 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c7fff700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c81e75f0 0x7f88c81428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88c6ffd700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8141620 0x7f88c81e70b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88b0015040 tx=0x7f88b0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88bffff700 1 -- 172.21.15.31:0/3802333840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0001290 con 0x7f88c8141620 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88bffff700 1 -- 172.21.15.31:0/3802333840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b001d480 con 0x7f88c8141620 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c81423a0 con 0x7f88c8141620 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88bffff700 1 -- 172.21.15.31:0/3802333840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b001d810 con 0x7f88c8141620 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.445+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c814f1a0 con 0x7f88c8141620 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.447+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c81e7a00 con 0x7f88c8141620 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.447+0000 7f88bffff700 1 -- 172.21.15.31:0/3802333840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f88b0019040 con 0x7f88c8141620 2024-03-30T01:41:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.447+0000 7f88bffff700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88a806acc0 0x7f88a806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.447+0000 7f88bffff700 1 -- 172.21.15.31:0/3802333840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f88b008a120 con 0x7f88c8141620 2024-03-30T01:41:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.447+0000 7f88c77fe700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88a806acc0 0x7f88a806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.448+0000 7f88c77fe700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88a806acc0 0x7f88a806d180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009c90 tx=0x7f88b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.452+0000 7f88bffff700 1 -- 172.21.15.31:0/3802333840 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f88b00551c0 con 0x7f88c8141620 2024-03-30T01:41:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.670+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88c804ea80 con 0x7f88c8141620 2024-03-30T01:41:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.671+0000 7f88bffff700 1 -- 172.21.15.31:0/3802333840 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f88b00587e0 con 0x7f88c8141620 2024-03-30T01:41:19.678 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:19.678 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88a806acc0 msgr2=0x7f88a806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88a806acc0 0x7f88a806d180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009c90 tx=0x7f88b8008040 comp rx=0 tx=0).stop 2024-03-30T01:41:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8141620 msgr2=0x7f88c81e70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8141620 0x7f88c81e70b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88b0015040 tx=0x7f88b0001120 comp rx=0 tx=0).stop 2024-03-30T01:41:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 shutdown_connections 2024-03-30T01:41:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88a806acc0 0x7f88a806d180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c81e75f0 0x7f88c81428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88b4004850 0x7f88c8153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 --2- 172.21.15.31:0/3802333840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8141620 0x7f88c81e70b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 >> 172.21.15.31:0/3802333840 conn(0x7f88c806ba20 msgr2=0x7f88c806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 shutdown_connections 2024-03-30T01:41:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.674+0000 7f88cdc31700 1 -- 172.21.15.31:0/3802333840 wait complete. 2024-03-30T01:41:19.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-30T01:41:19.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-30T01:41:19.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-30T01:41:19.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-30T01:41:19.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-30T01:41:19.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-30T01:41:19.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-30T01:41:19.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-30T01:41:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.931+0000 7f9ef846e700 1 Processor -- start 2024-03-30T01:41:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.931+0000 7f9ef846e700 1 -- start start 2024-03-30T01:41:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.933+0000 7f9ef846e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ef014fd30 0x7f9ef0156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.933+0000 7f9ef846e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ef00c47c0 0x7f9ef00ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.933+0000 7f9ef846e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef00c4d60 con 0x7f9ef0150110 2024-03-30T01:41:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.933+0000 7f9ef846e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef00c5040 con 0x7f9ef014fd30 2024-03-30T01:41:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.933+0000 7f9ef846e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef00c5320 con 0x7f9ef00c47c0 2024-03-30T01:41:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.933+0000 7f9ef5a09700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9ef0150110 0x7f9ef00c8380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34466/0 (socket says 172.21.15.31:34466) 2024-03-30T01:41:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.933+0000 7f9ef5a09700 1 -- 172.21.15.31:0/1032445949 learned_addr learned my addr 172.21.15.31:0/1032445949 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.933+0000 7f9ef6a0b700 1 --2- 172.21.15.31:0/1032445949 >> v2:172.21.15.59:3300/0 conn(0x7f9ef00c47c0 0x7f9ef00ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.933+0000 7f9ef620a700 1 --2- 172.21.15.31:0/1032445949 >> v2:172.21.15.31:3300/0 conn(0x7f9ef014fd30 0x7f9ef0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.934+0000 7f9ef5208700 1 -- 172.21.15.31:0/1032445949 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046448404 0 0) 0x7f9ef00c4d60 con 0x7f9ef0150110 2024-03-30T01:41:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.934+0000 7f9ef5208700 1 -- 172.21.15.31:0/1032445949 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9edc0035b0 con 0x7f9ef0150110 2024-03-30T01:41:19.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.934+0000 7f9ef620a700 1 -- 172.21.15.31:0/1032445949 >> v2:172.21.15.59:3300/0 conn(0x7f9ef00c47c0 msgr2=0x7f9ef00ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.934+0000 7f9ef620a700 1 --2- 172.21.15.31:0/1032445949 >> v2:172.21.15.59:3300/0 conn(0x7f9ef00c47c0 0x7f9ef00ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.934+0000 7f9ef620a700 1 -- 172.21.15.31:0/1032445949 >> v1:172.21.15.31:6789/0 conn(0x7f9ef0150110 legacy=0x7f9ef00c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.934+0000 7f9ef620a700 1 -- 172.21.15.31:0/1032445949 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00c5600 con 0x7f9ef014fd30 2024-03-30T01:41:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.934+0000 7f9ef620a700 1 --2- 172.21.15.31:0/1032445949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ef014fd30 0x7f9ef0156800 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00a9b10 tx=0x7f9eec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3785ed9d154ec2a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.934+0000 7f4ab3743700 1 Processor -- start 2024-03-30T01:41:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.935+0000 7fed3fe99700 1 Processor -- start 2024-03-30T01:41:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.935+0000 7f9ef5208700 1 -- 172.21.15.31:0/1032445949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011830 con 0x7f9ef014fd30 2024-03-30T01:41:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.935+0000 7f9ef5208700 1 -- 172.21.15.31:0/1032445949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec011990 con 0x7f9ef014fd30 2024-03-30T01:41:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.935+0000 7f9ef5208700 1 -- 172.21.15.31:0/1032445949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011d00 con 0x7f9ef014fd30 2024-03-30T01:41:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.935+0000 7fed3fe99700 1 -- start start 2024-03-30T01:41:19.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f9ef846e700 1 -- 172.21.15.31:0/1032445949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ef014fd30 msgr2=0x7f9ef0156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f9ef846e700 1 --2- 172.21.15.31:0/1032445949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ef014fd30 0x7f9ef0156800 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00a9b10 tx=0x7f9eec009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:19.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7fed3fe99700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed38070800 0x7fed38159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7fed3fe99700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3806caf0 con 0x7fed3806cf10 2024-03-30T01:41:19.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7fed3fe99700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed380728b0 con 0x7fed3806dc70 2024-03-30T01:41:19.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7fed3fe99700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed38072b90 con 0x7fed38070800 2024-03-30T01:41:19.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f9ef846e700 1 -- 172.21.15.31:0/1032445949 shutdown_connections 2024-03-30T01:41:19.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f9ef846e700 1 --2- 172.21.15.31:0/1032445949 >> v2:172.21.15.59:3300/0 conn(0x7f9ef00c47c0 0x7f9ef00ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f9ef846e700 1 --2- 172.21.15.31:0/1032445949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ef014fd30 0x7f9ef0156800 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f9ef846e700 1 -- 172.21.15.31:0/1032445949 >> 172.21.15.31:0/1032445949 conn(0x7f9ef006ab20 msgr2=0x7f9ef006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.935+0000 7f4ab3743700 1 -- start start 2024-03-30T01:41:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f4ab3743700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4aac150770 0x7f4aac0c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f4ab3743700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac153d60 con 0x7f4aac06c860 2024-03-30T01:41:19.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f4ab3743700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac1548e0 con 0x7f4aac0ab120 2024-03-30T01:41:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f4ab3743700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac071e70 con 0x7f4aac150770 2024-03-30T01:41:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f4ab1ce0700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4aac06c860 0x7f4aac0c0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34480/0 (socket says 172.21.15.31:34480) 2024-03-30T01:41:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.936+0000 7f4ab1ce0700 1 -- 172.21.15.31:0/3949741082 learned_addr learned my addr 172.21.15.31:0/3949741082 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7f4ab0cde700 1 --2- 172.21.15.31:0/3949741082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4aac150770 0x7f4aac0c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7fed3dc35700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed38070800 0x7fed38159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7fed3dc35700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed38070800 0x7fed38159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:59946/0 (socket says 172.21.15.31:59946) 2024-03-30T01:41:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7fed3dc35700 1 -- 172.21.15.31:0/1195885168 learned_addr learned my addr 172.21.15.31:0/1195885168 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7f9ef846e700 1 -- 172.21.15.31:0/1032445949 shutdown_connections 2024-03-30T01:41:19.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7f9ef846e700 1 -- 172.21.15.31:0/1032445949 wait complete. 2024-03-30T01:41:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.938+0000 7fed3dc35700 1 -- 172.21.15.31:0/1195885168 >> v1:172.21.15.59:6789/0 conn(0x7fed3806dc70 legacy=0x7fed3806e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.938+0000 7fed3dc35700 1 -- 172.21.15.31:0/1195885168 >> v1:172.21.15.31:6789/0 conn(0x7fed3806cf10 legacy=0x7fed38155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.938+0000 7fed3dc35700 1 -- 172.21.15.31:0/1195885168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3806d470 con 0x7fed38070800 2024-03-30T01:41:19.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.938+0000 7fed3dc35700 1 --2- 172.21.15.31:0/1195885168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed38070800 0x7fed38159600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fed28009920 tx=0x7fed28009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff675f8eb9cba474 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7efed0372700 1 Processor -- start 2024-03-30T01:41:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7f4aa3fff700 1 -- 172.21.15.31:0/3949741082 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650686275 0 0) 0x7f4aac153d60 con 0x7f4aac06c860 2024-03-30T01:41:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7f4aa3fff700 1 -- 172.21.15.31:0/3949741082 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a980057c0 con 0x7f4aac06c860 2024-03-30T01:41:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7f4aa3fff700 1 -- 172.21.15.31:0/3949741082 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028973120 0 0) 0x7f4aac1548e0 con 0x7f4aac0ab120 2024-03-30T01:41:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7f4aa3fff700 1 -- 172.21.15.31:0/3949741082 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aac153d60 con 0x7f4aac0ab120 2024-03-30T01:41:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7fed3cc33700 1 -- 172.21.15.31:0/1195885168 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28004230 con 0x7fed38070800 2024-03-30T01:41:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7fed3cc33700 1 -- 172.21.15.31:0/1195885168 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed28004390 con 0x7fed38070800 2024-03-30T01:41:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7fed3cc33700 1 -- 172.21.15.31:0/1195885168 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28009e70 con 0x7fed38070800 2024-03-30T01:41:19.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.937+0000 7efed0372700 1 -- start start 2024-03-30T01:41:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.938+0000 7efed0372700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80ab120 0x7efec8070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7efed0372700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efec8150770 0x7efec80c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7efed0372700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efec8071100 0x7efec80c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7efed0372700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec8071e70 con 0x7efec8071100 2024-03-30T01:41:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7efed0372700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec8153d60 con 0x7efec8150770 2024-03-30T01:41:19.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7efed0372700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec81548e0 con 0x7efec80ab120 2024-03-30T01:41:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7f9ef846e700 1 Processor -- start 2024-03-30T01:41:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7efece90f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efec8071100 0x7efec80c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7efece90f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efec8071100 0x7efec80c0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36476/0 (socket says 172.21.15.31:36476) 2024-03-30T01:41:19.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7efece90f700 1 -- 172.21.15.31:0/863683212 learned_addr learned my addr 172.21.15.31:0/863683212 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.940+0000 7fed3fe99700 1 -- 172.21.15.31:0/1195885168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed38070800 msgr2=0x7fed38159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.940+0000 7fed3fe99700 1 --2- 172.21.15.31:0/1195885168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed38070800 0x7fed38159600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fed28009920 tx=0x7fed28009c30 comp rx=0 tx=0).stop 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.940+0000 7efece90f700 1 -- 172.21.15.31:0/863683212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80ab120 msgr2=0x7efec8070bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.940+0000 7efece90f700 1 --2- 172.21.15.31:0/863683212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80ab120 0x7efec8070bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.940+0000 7efece90f700 1 -- 172.21.15.31:0/863683212 >> v2:172.21.15.59:3300/0 conn(0x7efec8150770 msgr2=0x7efec80c0800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.940+0000 7efece90f700 1 --2- 172.21.15.31:0/863683212 >> v2:172.21.15.59:3300/0 conn(0x7efec8150770 0x7efec80c0800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.940+0000 7efece90f700 1 -- 172.21.15.31:0/863683212 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec806c450 con 0x7efec8071100 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.940+0000 7efece90f700 1 --2- 172.21.15.31:0/863683212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec8071100 0x7efec80c0d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efec80c3450 tx=0x7efec000d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9dceb3e4668d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.940+0000 7efecd10c700 1 -- 172.21.15.31:0/863683212 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0007d30 con 0x7efec8071100 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7efecd10c700 1 -- 172.21.15.31:0/863683212 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec0007e90 con 0x7efec8071100 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7efecd10c700 1 -- 172.21.15.31:0/863683212 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0012710 con 0x7efec8071100 2024-03-30T01:41:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7fed3fe99700 1 -- 172.21.15.31:0/1195885168 shutdown_connections 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7fed3fe99700 1 --2- 172.21.15.31:0/1195885168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed38070800 0x7fed38159600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7fed3fe99700 1 -- 172.21.15.31:0/1195885168 >> 172.21.15.31:0/1195885168 conn(0x7fed380c2d60 msgr2=0x7fed380c3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.939+0000 7f9ef846e700 1 -- start start 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef846e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ef00c47c0 0x7f9ef0150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef846e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ef014fd30 0x7f9ef01f1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef846e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ef0150110 0x7f9ef0154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef846e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef01f1fb0 con 0x7f9ef00c47c0 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef846e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef01f2290 con 0x7f9ef0150110 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef846e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef01f1cb0 con 0x7f9ef014fd30 2024-03-30T01:41:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef6a0b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ef0150110 0x7f9ef0154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef620a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ef00c47c0 0x7f9ef0150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef6a0b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ef0150110 0x7f9ef0154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38742/0 (socket says 172.21.15.31:38742) 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef6a0b700 1 -- 172.21.15.31:0/1356014160 learned_addr learned my addr 172.21.15.31:0/1356014160 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef6a0b700 1 -- 172.21.15.31:0/1356014160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ef014fd30 msgr2=0x7f9ef01f1770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef6a0b700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ef014fd30 0x7f9ef01f1770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef6a0b700 1 -- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ef00c47c0 msgr2=0x7f9ef0150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.941+0000 7f9ef6a0b700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ef00c47c0 0x7f9ef0150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7f9ef6a0b700 1 -- 172.21.15.31:0/1356014160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec0094a0 con 0x7f9ef0150110 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7efed0372700 1 -- 172.21.15.31:0/863683212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec8071100 msgr2=0x7efec80c0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7efed0372700 1 --2- 172.21.15.31:0/863683212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec8071100 0x7efec80c0d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efec80c3450 tx=0x7efec000d650 comp rx=0 tx=0).stop 2024-03-30T01:41:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7f4ab0cde700 1 -- 172.21.15.31:0/3949741082 >> v1:172.21.15.59:6789/0 conn(0x7f4aac0ab120 legacy=0x7f4aac070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7f4ab0cde700 1 -- 172.21.15.31:0/3949741082 >> v1:172.21.15.31:6789/0 conn(0x7f4aac06c860 legacy=0x7f4aac0c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7f4ab0cde700 1 -- 172.21.15.31:0/3949741082 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac06c450 con 0x7f4aac150770 2024-03-30T01:41:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7fed3fe99700 1 -- 172.21.15.31:0/1195885168 shutdown_connections 2024-03-30T01:41:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7f4ab0cde700 1 --2- 172.21.15.31:0/3949741082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4aac150770 0x7f4aac0c0800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00aa10 tx=0x7f4a9c00ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e7570d606c2015 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7ff8a3cc2700 1 Processor -- start 2024-03-30T01:41:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7efed0372700 1 -- 172.21.15.31:0/863683212 shutdown_connections 2024-03-30T01:41:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7efed0372700 1 --2- 172.21.15.31:0/863683212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec8071100 0x7efec80c0d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7efed0372700 1 --2- 172.21.15.31:0/863683212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80ab120 0x7efec8070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7efed0372700 1 --2- 172.21.15.31:0/863683212 >> v2:172.21.15.59:3300/0 conn(0x7efec8150770 0x7efec80c0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7efed0372700 1 -- 172.21.15.31:0/863683212 >> 172.21.15.31:0/863683212 conn(0x7efec806aaf0 msgr2=0x7efec806af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:19.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7f4aa3fff700 1 -- 172.21.15.31:0/3949741082 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4005270 con 0x7f4aac150770 2024-03-30T01:41:19.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7f4aa3fff700 1 -- 172.21.15.31:0/3949741082 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c00ae50 con 0x7f4aac150770 2024-03-30T01:41:19.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7f4aa3fff700 1 -- 172.21.15.31:0/3949741082 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c004250 con 0x7f4aac150770 2024-03-30T01:41:19.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7efed0372700 1 -- 172.21.15.31:0/863683212 shutdown_connections 2024-03-30T01:41:19.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7ff8a3cc2700 1 -- start start 2024-03-30T01:41:19.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.943+0000 7ff8a3cc2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff89c0ab120 0x7ff89c070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.943+0000 7ff8a3cc2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c153d60 con 0x7ff89c071100 2024-03-30T01:41:19.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.943+0000 7ff8a3cc2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c1548e0 con 0x7ff89c150770 2024-03-30T01:41:19.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.943+0000 7ff8a3cc2700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c071e70 con 0x7ff89c0ab120 2024-03-30T01:41:19.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7f3d669f5700 1 Processor -- start 2024-03-30T01:41:19.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7fed3fe99700 1 -- 172.21.15.31:0/1195885168 wait complete. 2024-03-30T01:41:19.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7fed3fe99700 1 Processor -- start 2024-03-30T01:41:19.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7f9ef6a0b700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ef0150110 0x7f9ef0154330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800efc0 tx=0x7f9ee800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.943+0000 7f9ee6ffd700 1 -- 172.21.15.31:0/1356014160 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8009ea0 con 0x7f9ef0150110 2024-03-30T01:41:19.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.943+0000 7f9ee6ffd700 1 -- 172.21.15.31:0/1356014160 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee800f040 con 0x7f9ef0150110 2024-03-30T01:41:19.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7ff8a125d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff89c150770 0x7ff89c0c0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56418/0 (socket says 172.21.15.31:56418) 2024-03-30T01:41:19.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7ff8a125d700 1 -- 172.21.15.31:0/3434888081 learned_addr learned my addr 172.21.15.31:0/3434888081 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.943+0000 7f9ee6ffd700 1 -- 172.21.15.31:0/1356014160 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8014480 con 0x7f9ef0150110 2024-03-30T01:41:19.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7ff8a1a5e700 1 --2- 172.21.15.31:0/3434888081 >> v2:172.21.15.31:3300/0 conn(0x7ff89c0ab120 0x7ff89c070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.942+0000 7efed0372700 1 -- 172.21.15.31:0/863683212 wait complete. 2024-03-30T01:41:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7efed0372700 1 Processor -- start 2024-03-30T01:41:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7ff8a0a5c700 1 -- 172.21.15.31:0/3434888081 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947354679 0 0) 0x7ff89c1548e0 con 0x7ff89c150770 2024-03-30T01:41:19.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7ff8a0a5c700 1 -- 172.21.15.31:0/3434888081 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8880034d0 con 0x7ff89c150770 2024-03-30T01:41:19.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7ff8a1a5e700 1 -- 172.21.15.31:0/3434888081 >> v1:172.21.15.59:6789/0 conn(0x7ff89c150770 legacy=0x7ff89c0c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7ff8a1a5e700 1 -- 172.21.15.31:0/3434888081 >> v1:172.21.15.31:6789/0 conn(0x7ff89c071100 legacy=0x7ff89c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7ff8a1a5e700 1 -- 172.21.15.31:0/3434888081 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c06c450 con 0x7ff89c0ab120 2024-03-30T01:41:19.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7f3d669f5700 1 -- start start 2024-03-30T01:41:19.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7f3d669f5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3d6006b400 0x7f3d60151a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7f3d669f5700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d600c7fd0 con 0x7f3d600c4f60 2024-03-30T01:41:19.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7f3d669f5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d60154a20 con 0x7f3d60151f40 2024-03-30T01:41:19.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7f3d669f5700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d60154830 con 0x7f3d6006b400 2024-03-30T01:41:19.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7ff8a1a5e700 1 --2- 172.21.15.31:0/3434888081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff89c0ab120 0x7ff89c070bc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff894009d90 tx=0x7ff89400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedbc3164091d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7f3d64f92700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3d60151f40 0x7f3d6015f1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56426/0 (socket says 172.21.15.31:56426) 2024-03-30T01:41:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7f3d64f92700 1 -- 172.21.15.31:0/474241946 learned_addr learned my addr 172.21.15.31:0/474241946 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.944+0000 7fed3fe99700 1 -- start start 2024-03-30T01:41:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3fe99700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed3806cf10 0x7fed38158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3fe99700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed3806dc70 0x7fed38157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3fe99700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed38070800 0x7fed38159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3fe99700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3814b110 con 0x7fed3806dc70 2024-03-30T01:41:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3fe99700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed381579e0 con 0x7fed38070800 2024-03-30T01:41:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3fe99700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3814a350 con 0x7fed3806cf10 2024-03-30T01:41:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3dc35700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed3806cf10 0x7fed38158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3dc35700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed3806cf10 0x7fed38158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:59958/0 (socket says 172.21.15.31:59958) 2024-03-30T01:41:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3dc35700 1 -- 172.21.15.31:0/2927541841 learned_addr learned my addr 172.21.15.31:0/2927541841 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3e436700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed38070800 0x7fed38159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3d434700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed3806dc70 0x7fed38157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3dc35700 1 -- 172.21.15.31:0/2927541841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed38070800 msgr2=0x7fed38159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3dc35700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed38070800 0x7fed38159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3dc35700 1 -- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed3806dc70 msgr2=0x7fed38157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3dc35700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed3806dc70 0x7fed38157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7fed3dc35700 1 -- 172.21.15.31:0/2927541841 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28009550 con 0x7fed3806cf10 2024-03-30T01:41:19.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.945+0000 7efed0372700 1 -- start start 2024-03-30T01:41:19.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7efed0372700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efec8071100 0x7efec81fa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7efed0372700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec8150770 0x7efec80c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7efed0372700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80c5810 0x7efec80c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7efed0372700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec80bb2a0 con 0x7efec8150770 2024-03-30T01:41:19.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7efed0372700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec80bcf70 con 0x7efec8071100 2024-03-30T01:41:19.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7efed0372700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec80bb840 con 0x7efec80c5810 2024-03-30T01:41:19.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7ff8a0a5c700 1 -- 172.21.15.31:0/3434888081 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89400e070 con 0x7ff89c0ab120 2024-03-30T01:41:19.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190912244 0 0) 0x7f3d60154a20 con 0x7f3d60151f40 2024-03-30T01:41:19.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d480035b0 con 0x7f3d60151f40 2024-03-30T01:41:19.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7ff8a0a5c700 1 -- 172.21.15.31:0/3434888081 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89400bb20 con 0x7ff89c0ab120 2024-03-30T01:41:19.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.946+0000 7ff8a0a5c700 1 -- 172.21.15.31:0/3434888081 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894012430 con 0x7ff89c0ab120 2024-03-30T01:41:19.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/3434888081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff89c0ab120 msgr2=0x7ff89c070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7ff8a3cc2700 1 --2- 172.21.15.31:0/3434888081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff89c0ab120 0x7ff89c070bc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff894009d90 tx=0x7ff89400b330 comp rx=0 tx=0).stop 2024-03-30T01:41:19.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7fed3dc35700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed3806cf10 0x7fed38158620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed38073850 tx=0x7fed28004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7efece90f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80c5810 0x7efec80c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7efece90f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80c5810 0x7efec80c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:59968/0 (socket says 172.21.15.31:59968) 2024-03-30T01:41:19.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7efece90f700 1 -- 172.21.15.31:0/2360571317 learned_addr learned my addr 172.21.15.31:0/2360571317 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:19.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5ffff700 1 --2- 172.21.15.31:0/474241946 >> v2:172.21.15.31:3300/0 conn(0x7f3d6006b400 0x7f3d60151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 <== mon.1 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1613009246 0 0) 0x7f3d480035b0 con 0x7f3d60151f40 2024-03-30T01:41:19.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 >> v2:172.21.15.31:3300/0 conn(0x7f3d6006b400 msgr2=0x7f3d60151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 --2- 172.21.15.31:0/474241946 >> v2:172.21.15.31:3300/0 conn(0x7f3d6006b400 0x7f3d60151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7efece10e700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efec8071100 0x7efec81fa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/3434888081 shutdown_connections 2024-03-30T01:41:19.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7ff8a3cc2700 1 --2- 172.21.15.31:0/3434888081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff89c0ab120 0x7ff89c070bc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/3434888081 >> 172.21.15.31:0/3434888081 conn(0x7ff89c06aaf0 msgr2=0x7ff89c06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:19.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 >> v1:172.21.15.31:6789/0 conn(0x7f3d600c4f60 legacy=0x7f3d600c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600c5590 con 0x7f3d60151f40 2024-03-30T01:41:19.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 <== mon.1 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f3d580019f0 con 0x7f3d60151f40 2024-03-30T01:41:19.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 >> v1:172.21.15.59:6789/0 conn(0x7f3d60151f40 legacy=0x7f3d6015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d48003a00 0x7f3d48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d48004850 0x7f3d48004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48005410 0x7f3d4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4800c2a0 con 0x7f3d48005410 2024-03-30T01:41:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4800c410 con 0x7f3d48004850 2024-03-30T01:41:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4800c5d0 con 0x7f3d48003a00 2024-03-30T01:41:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5ffff700 1 --2- 172.21.15.31:0/474241946 >> v2:172.21.15.31:3300/0 conn(0x7f3d6006b400 0x7f3d60151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:19.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/3434888081 shutdown_connections 2024-03-30T01:41:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef01470a0 con 0x7f9ef0150110 2024-03-30T01:41:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef00a3560 con 0x7f9ef0150110 2024-03-30T01:41:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ef01f2120 con 0x7f9ef0150110 2024-03-30T01:41:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7f3d5ffff700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d48003a00 0x7f3d48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f3d5f7fe700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d48004850 0x7f3d48004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f3d64f92700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48005410 0x7f3d4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f3d5ffff700 1 -- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d48004850 msgr2=0x7f3d48004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f3d5ffff700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d48004850 0x7f3d48004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f3d5ffff700 1 -- 172.21.15.31:0/474241946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48005410 msgr2=0x7f3d4800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:19.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f3d5ffff700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d48005410 0x7f3d4800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f3d5ffff700 1 -- 172.21.15.31:0/474241946 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600c5590 con 0x7f3d48003a00 2024-03-30T01:41:19.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7efece90f700 1 -- 172.21.15.31:0/2360571317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efec8071100 msgr2=0x7efec81fa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7efece90f700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efec8071100 0x7efec81fa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7efece90f700 1 -- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec8150770 msgr2=0x7efec80c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7efece90f700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec8150770 0x7efec80c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7efece90f700 1 -- 172.21.15.31:0/2360571317 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec000d050 con 0x7efec80c5810 2024-03-30T01:41:19.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7efece90f700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80c5810 0x7efec80c4c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efec000d690 tx=0x7efec0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7efebeffd700 1 -- 172.21.15.31:0/2360571317 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00048e0 con 0x7efec80c5810 2024-03-30T01:41:19.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec80bafc0 con 0x7efec80c5810 2024-03-30T01:41:19.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec81fadf0 con 0x7efec80c5810 2024-03-30T01:41:19.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f4ab3743700 1 -- 172.21.15.31:0/3949741082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4aac150770 msgr2=0x7f4aac0c0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7f4ab3743700 1 --2- 172.21.15.31:0/3949741082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4aac150770 0x7f4aac0c0800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00aa10 tx=0x7f4a9c00ad20 comp rx=0 tx=0).stop 2024-03-30T01:41:19.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.947+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/3434888081 wait complete. 2024-03-30T01:41:19.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.949+0000 7ff8a3cc2700 1 Processor -- start 2024-03-30T01:41:19.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7fed2e7fc700 1 -- 172.21.15.31:0/2927541841 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2800bd20 con 0x7fed3806cf10 2024-03-30T01:41:19.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7fed2e7fc700 1 -- 172.21.15.31:0/2927541841 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2800bea0 con 0x7fed3806cf10 2024-03-30T01:41:19.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7fed2e7fc700 1 -- 172.21.15.31:0/2927541841 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280038e0 con 0x7fed3806cf10 2024-03-30T01:41:19.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3814a630 con 0x7fed3806cf10 2024-03-30T01:41:19.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3814abf0 con 0x7fed3806cf10 2024-03-30T01:41:19.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.949+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed38072580 con 0x7fed3806cf10 2024-03-30T01:41:19.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7efebeffd700 1 -- 172.21.15.31:0/2360571317 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec0008030 con 0x7efec80c5810 2024-03-30T01:41:19.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.948+0000 7efebeffd700 1 -- 172.21.15.31:0/2360571317 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00083a0 con 0x7efec80c5810 2024-03-30T01:41:19.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.949+0000 7f3d64f92700 1 -- 172.21.15.31:0/474241946 reap_dead start 2024-03-30T01:41:19.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7f9ee6ffd700 1 -- 172.21.15.31:0/1356014160 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9ee80145e0 con 0x7f9ef0150110 2024-03-30T01:41:19.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efeb0000fc0 con 0x7efec80c5810 2024-03-30T01:41:19.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7f9ee6ffd700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ed406ad60 0x7f9ed406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7f9ee6ffd700 1 -- 172.21.15.31:0/1356014160 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ee808a140 con 0x7f9ef0150110 2024-03-30T01:41:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7f9ee6ffd700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6816/47369,v1:172.21.15.31:6817/47369] conn(0x7f9ed40743a0 0x7f9ed40767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7f9ee6ffd700 1 -- 172.21.15.31:0/1356014160 --> [v2:172.21.15.31:6816/47369,v1:172.21.15.31:6817/47369] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ed4076eb0 con 0x7f9ed40743a0 2024-03-30T01:41:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7f9ef620a700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ed406ad60 0x7f9ed406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7f9ee6ffd700 1 -- 172.21.15.31:0/1356014160 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ee808a470 con 0x7f9ef0150110 2024-03-30T01:41:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.949+0000 7ff8a3cc2700 1 -- start start 2024-03-30T01:41:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a3cc2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff89c071100 0x7ff89c148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a3cc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff89c0ab120 0x7ff89c0c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a3cc2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff89c150770 0x7ff89c1f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a3cc2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c147a80 con 0x7ff89c071100 2024-03-30T01:41:19.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a3cc2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c1408b0 con 0x7ff89c0ab120 2024-03-30T01:41:19.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a3cc2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0c44a0 con 0x7ff89c150770 2024-03-30T01:41:19.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7f9ef5a09700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6816/47369,v1:172.21.15.31:6817/47369] conn(0x7f9ed40743a0 0x7f9ed40767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7f4ab3743700 1 -- 172.21.15.31:0/3949741082 shutdown_connections 2024-03-30T01:41:19.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7f4ab3743700 1 --2- 172.21.15.31:0/3949741082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4aac150770 0x7f4aac0c0800 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7f4ab3743700 1 -- 172.21.15.31:0/3949741082 >> 172.21.15.31:0/3949741082 conn(0x7f4aac06aaf0 msgr2=0x7f4aac06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:19.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a225f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff89c150770 0x7ff89c1f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a225f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff89c150770 0x7ff89c1f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:59974/0 (socket says 172.21.15.31:59974) 2024-03-30T01:41:19.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a225f700 1 -- 172.21.15.31:0/2368764264 learned_addr learned my addr 172.21.15.31:0/2368764264 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:19.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a125d700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff89c0ab120 0x7ff89c0c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7f4ab3743700 1 -- 172.21.15.31:0/3949741082 shutdown_connections 2024-03-30T01:41:19.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7efebeffd700 1 -- 172.21.15.31:0/2360571317 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efec0008500 con 0x7efec80c5810 2024-03-30T01:41:19.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7efebeffd700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efeac06ac90 0x7efeac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7efebeffd700 1 -- 172.21.15.31:0/2360571317 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efec008aac0 con 0x7efec80c5810 2024-03-30T01:41:19.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7efebeffd700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6824/47374,v1:172.21.15.31:6825/47374] conn(0x7efeac0742d0 0x7efeac076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:19.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7efebeffd700 1 -- 172.21.15.31:0/2360571317 --> [v2:172.21.15.31:6824/47374,v1:172.21.15.31:6825/47374] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efeac076de0 con 0x7efeac0742d0 2024-03-30T01:41:19.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7efebeffd700 1 -- 172.21.15.31:0/2360571317 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7efec008f260 con 0x7efec80c5810 2024-03-30T01:41:19.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7f4ab3743700 1 -- 172.21.15.31:0/3949741082 wait complete. 2024-03-30T01:41:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a225f700 1 -- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff89c0ab120 msgr2=0x7ff89c0c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a225f700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff89c0ab120 0x7ff89c0c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a225f700 1 -- 172.21.15.31:0/2368764264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff89c071100 msgr2=0x7ff89c148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7efece10e700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efeac06ac90 0x7efeac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a225f700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff89c071100 0x7ff89c148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7ff8a225f700 1 -- 172.21.15.31:0/2368764264 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894009a70 con 0x7ff89c150770 2024-03-30T01:41:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7efecd90d700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6824/47374,v1:172.21.15.31:6825/47374] conn(0x7efeac0742d0 0x7efeac076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:19.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7ff8a225f700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff89c150770 0x7ff89c1f1ec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c370 tx=0x7ff89800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.950+0000 7f3d5ffff700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d48003a00 0x7f3d48003e20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000ab20 tx=0x7f3d48005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49cbdad425e90f93 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5000c880 con 0x7f3d48003a00 2024-03-30T01:41:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5000c9e0 con 0x7f3d48003a00 2024-03-30T01:41:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.951+0000 7f3d5effd700 1 -- 172.21.15.31:0/474241946 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5000cd50 con 0x7f3d48003a00 2024-03-30T01:41:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7f3d669f5700 1 -- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d48003a00 msgr2=0x7f3d48003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7f3d669f5700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d48003a00 0x7f3d48003e20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000ab20 tx=0x7f3d48005880 comp rx=0 tx=0).stop 2024-03-30T01:41:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7f3d669f5700 1 -- 172.21.15.31:0/474241946 shutdown_connections 2024-03-30T01:41:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7f3d669f5700 1 --2- 172.21.15.31:0/474241946 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d48003a00 0x7f3d48003e20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:19.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7f3d669f5700 1 -- 172.21.15.31:0/474241946 >> 172.21.15.31:0/474241946 conn(0x7f3d600ca0d0 msgr2=0x7f3d600ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7ff8927fc700 1 -- 172.21.15.31:0/2368764264 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980052d0 con 0x7ff89c150770 2024-03-30T01:41:20.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7ff8927fc700 1 -- 172.21.15.31:0/2368764264 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff898005430 con 0x7ff89c150770 2024-03-30T01:41:20.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7ff8927fc700 1 -- 172.21.15.31:0/2368764264 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898006a80 con 0x7ff89c150770 2024-03-30T01:41:20.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7f3d669f5700 1 -- 172.21.15.31:0/474241946 shutdown_connections 2024-03-30T01:41:20.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.953+0000 7efecd90d700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6824/47374,v1:172.21.15.31:6825/47374] conn(0x7efeac0742d0 0x7efeac076710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89c0c4780 con 0x7ff89c150770 2024-03-30T01:41:20.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89c1f19a0 con 0x7ff89c150770 2024-03-30T01:41:20.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.953+0000 7f4ab3743700 1 Processor -- start 2024-03-30T01:41:20.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.953+0000 7efece10e700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efeac06ac90 0x7efeac06d150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006fd0 tx=0x7efeb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.953+0000 7f9ef620a700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ed406ad60 0x7f9ed406d220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009b60 tx=0x7f9eec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.954+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff89c1f1820 con 0x7ff89c150770 2024-03-30T01:41:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.953+0000 7fed2e7fc700 1 -- 172.21.15.31:0/2927541841 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fed28021070 con 0x7fed3806cf10 2024-03-30T01:41:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.953+0000 7fed2e7fc700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fed1c06ad60 0x7fed1c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.953+0000 7fed2e7fc700 1 -- 172.21.15.31:0/2927541841 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fed2808b070 con 0x7fed3806cf10 2024-03-30T01:41:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.954+0000 7fed2e7fc700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6800/47358,v1:172.21.15.31:6801/47358] conn(0x7fed1c0743a0 0x7fed1c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.954+0000 7fed2e7fc700 1 -- 172.21.15.31:0/2927541841 --> [v2:172.21.15.31:6800/47358,v1:172.21.15.31:6801/47358] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed1c076eb0 con 0x7fed1c0743a0 2024-03-30T01:41:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.954+0000 7fed2e7fc700 1 -- 172.21.15.31:0/2927541841 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed2808b420 con 0x7fed3806cf10 2024-03-30T01:41:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.954+0000 7fed3e436700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6800/47358,v1:172.21.15.31:6801/47358] conn(0x7fed1c0743a0 0x7fed1c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.954+0000 7fed3d434700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fed1c06ad60 0x7fed1c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.955+0000 7fed3e436700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6800/47358,v1:172.21.15.31:6801/47358] conn(0x7fed1c0743a0 0x7fed1c0767e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.955+0000 7fed3d434700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fed1c06ad60 0x7fed1c06d220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fed38074050 tx=0x7fed34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.955+0000 7f9ef5a09700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6816/47369,v1:172.21.15.31:6817/47369] conn(0x7f9ed40743a0 0x7f9ed40767e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.952+0000 7f3d669f5700 1 -- 172.21.15.31:0/474241946 wait complete. 2024-03-30T01:41:20.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.954+0000 7f3d669f5700 1 Processor -- start 2024-03-30T01:41:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.955+0000 7efebeffd700 1 -- 172.21.15.31:0/2360571317 <== osd.3 v2:172.21.15.31:6824/47374 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7efeac076de0 con 0x7efeac0742d0 2024-03-30T01:41:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.956+0000 7ff8927fc700 1 -- 172.21.15.31:0/2368764264 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff898003680 con 0x7ff89c150770 2024-03-30T01:41:20.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.955+0000 7f4ab3743700 1 -- start start 2024-03-30T01:41:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.956+0000 7ff8927fc700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff88006ac90 0x7ff88006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.956+0000 7ff8927fc700 1 -- 172.21.15.31:0/2368764264 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff898088f40 con 0x7ff89c150770 2024-03-30T01:41:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.956+0000 7f9ee6ffd700 1 -- 172.21.15.31:0/1356014160 <== osd.1 v2:172.21.15.31:6816/47369 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f9ed4076eb0 con 0x7f9ed40743a0 2024-03-30T01:41:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.956+0000 7ff8a1a5e700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff88006ac90 0x7ff88006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.956+0000 7ff8927fc700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:6808/47502,v1:172.21.15.59:6809/47502] conn(0x7ff880074250 0x7ff880076690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.956+0000 7ff8927fc700 1 -- 172.21.15.31:0/2368764264 --> [v2:172.21.15.59:6808/47502,v1:172.21.15.59:6809/47502] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff880076d60 con 0x7ff880074250 2024-03-30T01:41:20.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.956+0000 7ff8927fc700 1 -- 172.21.15.31:0/2368764264 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff898089390 con 0x7ff89c150770 2024-03-30T01:41:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab3743700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aac06c860 0x7f4aac06d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab3743700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4aac0ab120 0x7f4aac144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab3743700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac150770 0x7f4aac145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab3743700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac146080 con 0x7f4aac06c860 2024-03-30T01:41:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab3743700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac152b20 con 0x7f4aac150770 2024-03-30T01:41:20.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab3743700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1ee010 con 0x7f4aac0ab120 2024-03-30T01:41:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7ff8a125d700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:6808/47502,v1:172.21.15.59:6809/47502] conn(0x7ff880074250 0x7ff880076690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7ff8a1a5e700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff88006ac90 0x7ff88006d150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff894014040 tx=0x7ff894010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab14df700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aac06c860 0x7f4aac06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab1ce0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac150770 0x7f4aac145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab14df700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aac06c860 0x7f4aac06d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36566/0 (socket says 172.21.15.31:36566) 2024-03-30T01:41:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab1ce0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac150770 0x7f4aac145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38782/0 (socket says 172.21.15.31:38782) 2024-03-30T01:41:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7f4ab14df700 1 -- 172.21.15.31:0/1159449219 learned_addr learned my addr 172.21.15.31:0/1159449219 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.957+0000 7ff8a125d700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:6808/47502,v1:172.21.15.59:6809/47502] conn(0x7ff880074250 0x7ff880076690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.958+0000 7f4ab14df700 1 -- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4aac0ab120 msgr2=0x7f4aac144e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T01:41:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.958+0000 7f4ab14df700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4aac0ab120 0x7f4aac144e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.958+0000 7f4ab14df700 1 -- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac150770 msgr2=0x7f4aac145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.958+0000 7f4ab14df700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac150770 0x7f4aac145380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.958+0000 7f4ab14df700 1 -- 172.21.15.31:0/1159449219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c00a640 con 0x7f4aac06c860 2024-03-30T01:41:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.958+0000 7f4ab1ce0700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4aac150770 0x7f4aac145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.958+0000 7f4ab14df700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aac06c860 0x7f4aac06d670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800c4b0 tx=0x7f4aa800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.958+0000 7ff8927fc700 1 -- 172.21.15.31:0/2368764264 <== osd.7 v2:172.21.15.59:6808/47502 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff880076d60 con 0x7ff880074250 2024-03-30T01:41:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.959+0000 7f4aa1ffb700 1 -- 172.21.15.31:0/1159449219 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80052d0 con 0x7f4aac06c860 2024-03-30T01:41:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.959+0000 7f4aa1ffb700 1 -- 172.21.15.31:0/1159449219 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa8005430 con 0x7f4aac06c860 2024-03-30T01:41:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.959+0000 7f4aa1ffb700 1 -- 172.21.15.31:0/1159449219 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80038c0 con 0x7f4aac06c860 2024-03-30T01:41:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.959+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac152280 con 0x7f4aac06c860 2024-03-30T01:41:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.959+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac152840 con 0x7f4aac06c860 2024-03-30T01:41:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.960+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a8c000fc0 con 0x7f4aac06c860 2024-03-30T01:41:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.959+0000 7f3d669f5700 1 -- start start 2024-03-30T01:41:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.961+0000 7f3d669f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d48003a00 0x7f3d60148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.961+0000 7f3d669f5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d601f13b0 0x7f3d6014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.961+0000 7f3d669f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d6014b920 0x7f3d601efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.961+0000 7f3d669f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60155e30 con 0x7f3d6014b920 2024-03-30T01:41:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.961+0000 7f3d669f5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60149c70 con 0x7f3d48003a00 2024-03-30T01:41:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.961+0000 7f3d669f5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60149f50 con 0x7f3d601f13b0 2024-03-30T01:41:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.961+0000 7f4aa1ffb700 1 -- 172.21.15.31:0/1159449219 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4aa8003a20 con 0x7f4aac06c860 2024-03-30T01:41:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f4aa1ffb700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4a9006ace0 0x7f4a9006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f4aa1ffb700 1 -- 172.21.15.31:0/1159449219 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4aa8088930 con 0x7f4aac06c860 2024-03-30T01:41:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f4aa1ffb700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:6816/47512,v1:172.21.15.59:6817/47512] conn(0x7f4a90074320 0x7f4a90076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f4aa1ffb700 1 -- 172.21.15.31:0/1159449219 --> [v2:172.21.15.59:6816/47512,v1:172.21.15.59:6817/47512] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a90076e30 con 0x7f4a90074320 2024-03-30T01:41:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f4aa1ffb700 1 -- 172.21.15.31:0/1159449219 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4aa808d2c0 con 0x7f4aac06c860 2024-03-30T01:41:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f3d64f92700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d6014b920 0x7f3d601efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f3d64f92700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d6014b920 0x7f3d601efd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36574/0 (socket says 172.21.15.31:36574) 2024-03-30T01:41:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f3d64f92700 1 -- 172.21.15.31:0/1251683102 learned_addr learned my addr 172.21.15.31:0/1251683102 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f4ab1ce0700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:6816/47512,v1:172.21.15.59:6817/47512] conn(0x7f4a90074320 0x7f4a90076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f3d5f7fe700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d601f13b0 0x7f3d6014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.962+0000 7f3d5ffff700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d48003a00 0x7f3d60148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.963+0000 7f3d64f92700 1 -- 172.21.15.31:0/1251683102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d601f13b0 msgr2=0x7f3d6014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.963+0000 7f3d64f92700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3d601f13b0 0x7f3d6014b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.963+0000 7f3d64f92700 1 -- 172.21.15.31:0/1251683102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d48003a00 msgr2=0x7f3d60148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.963+0000 7f3d64f92700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d48003a00 0x7f3d60148230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.963+0000 7f3d64f92700 1 -- 172.21.15.31:0/1251683102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6014a230 con 0x7f3d6014b920 2024-03-30T01:41:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.958+0000 7f3e3cbd8700 1 Processor -- start 2024-03-30T01:41:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.963+0000 7f3d64f92700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d6014b920 0x7f3d601efd20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d58001330 tx=0x7f3d5800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.961+0000 7fed2e7fc700 1 -- 172.21.15.31:0/2927541841 <== osd.0 v2:172.21.15.31:6800/47358 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fed1c076eb0 con 0x7fed1c0743a0 2024-03-30T01:41:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.963+0000 7f3e3cbd8700 1 -- start start 2024-03-30T01:41:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.964+0000 7f3e3cbd8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e380722b0 0x7f3e3806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.964+0000 7f3e3cbd8700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e3806db20 0x7f3e381479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.964+0000 7f3e3cbd8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e380a88c0 con 0x7f3e38154d20 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.964+0000 7f3e3cbd8700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e380c3110 con 0x7f3e3806db20 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.964+0000 7f3e3cbd8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38155810 con 0x7f3e380722b0 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.966+0000 7f3e36d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e3806db20 0x7f3e381479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.966+0000 7f3e36d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e3806db20 0x7f3e381479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36590/0 (socket says 172.21.15.31:36590) 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.966+0000 7f3e36d9d700 1 -- 172.21.15.31:0/1883598470 learned_addr learned my addr 172.21.15.31:0/1883598470 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.967+0000 7f3e36d9d700 1 -- 172.21.15.31:0/1883598470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e380722b0 msgr2=0x7f3e3806d5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.967+0000 7f3e36d9d700 1 --2- 172.21.15.31:0/1883598470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e380722b0 0x7f3e3806d5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.967+0000 7f3e36d9d700 1 -- 172.21.15.31:0/1883598470 >> v1:172.21.15.59:6789/0 conn(0x7f3e38154d20 legacy=0x7f3e38155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.967+0000 7f3e36d9d700 1 -- 172.21.15.31:0/1883598470 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e380a85e0 con 0x7f3e3806db20 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.967+0000 7f3e36d9d700 1 --2- 172.21.15.31:0/1883598470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e3806db20 0x7f3e381479d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d150 tx=0x7f3e3000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f2c6e9e7f900ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.967+0000 7f4ab0cde700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4a9006ace0 0x7f4a9006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.968+0000 7f3d5cff9700 1 -- 172.21.15.31:0/1251683102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58013420 con 0x7f3d6014b920 2024-03-30T01:41:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.968+0000 7f3d5cff9700 1 -- 172.21.15.31:0/1251683102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d58013580 con 0x7f3d6014b920 2024-03-30T01:41:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.968+0000 7f3d5cff9700 1 -- 172.21.15.31:0/1251683102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d580138f0 con 0x7f3d6014b920 2024-03-30T01:41:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.968+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6014a510 con 0x7f3d6014b920 2024-03-30T01:41:20.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.968+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d600ab9e0 con 0x7f3d6014b920 2024-03-30T01:41:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.968+0000 7f4ab1ce0700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:6816/47512,v1:172.21.15.59:6817/47512] conn(0x7f4a90074320 0x7f4a90076760 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.969+0000 7f4ab0cde700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4a9006ace0 0x7f4a9006d1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0c23b0 tx=0x7f4a9c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.968+0000 7f3e3559a700 1 -- 172.21.15.31:0/1883598470 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30010070 con 0x7f3e3806db20 2024-03-30T01:41:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.968+0000 7f3e3559a700 1 -- 172.21.15.31:0/1883598470 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3000dd80 con 0x7f3e3806db20 2024-03-30T01:41:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.968+0000 7f3e3559a700 1 -- 172.21.15.31:0/1883598470 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000b310 con 0x7f3e3806db20 2024-03-30T01:41:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.969+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1883598470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e3806db20 msgr2=0x7f3e381479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.969+0000 7f3e3cbd8700 1 --2- 172.21.15.31:0/1883598470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e3806db20 0x7f3e381479d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d150 tx=0x7f3e3000d460 comp rx=0 tx=0).stop 2024-03-30T01:41:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.970+0000 7f4aa1ffb700 1 -- 172.21.15.31:0/1159449219 <== osd.4 v2:172.21.15.59:6816/47512 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4a90076e30 con 0x7f4a90074320 2024-03-30T01:41:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.970+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1883598470 shutdown_connections 2024-03-30T01:41:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.970+0000 7f3e3cbd8700 1 --2- 172.21.15.31:0/1883598470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e3806db20 0x7f3e381479d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.970+0000 7f3e3cbd8700 1 --2- 172.21.15.31:0/1883598470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e380722b0 0x7f3e3806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.970+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1883598470 >> 172.21.15.31:0/1883598470 conn(0x7f3e380c3a60 msgr2=0x7f3e380c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.971+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1883598470 shutdown_connections 2024-03-30T01:41:20.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.970+0000 7f3d5cff9700 1 -- 172.21.15.31:0/1251683102 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3d58013a50 con 0x7f3d6014b920 2024-03-30T01:41:20.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.971+0000 7f3d5cff9700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3d4c06ad10 0x7f3d4c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.971+0000 7f3d5cff9700 1 -- 172.21.15.31:0/1251683102 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d58089c80 con 0x7f3d6014b920 2024-03-30T01:41:20.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.971+0000 7f3d669f5700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6808/47360,v1:172.21.15.31:6809/47360] conn(0x7f3d400015e0 0x7f3d40003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.971+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 --> [v2:172.21.15.31:6808/47360,v1:172.21.15.31:6809/47360] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d40006bd0 con 0x7f3d400015e0 2024-03-30T01:41:20.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.972+0000 7f3d5f7fe700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6808/47360,v1:172.21.15.31:6809/47360] conn(0x7f3d400015e0 0x7f3d40003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.971+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1883598470 wait complete. 2024-03-30T01:41:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.973+0000 7f3e3cbd8700 1 Processor -- start 2024-03-30T01:41:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.973+0000 7f3e3cbd8700 1 -- start start 2024-03-30T01:41:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.974+0000 7f3e3cbd8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e3806db20 0x7f3e38151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.974+0000 7f3e3cbd8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e380722b0 0x7f3e381e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.974+0000 7f3e3cbd8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e38154d20 0x7f3e38144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.974+0000 7f3e3cbd8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380c6ca0 con 0x7f3e38154d20 2024-03-30T01:41:20.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.974+0000 7f3e3cbd8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3814d560 con 0x7f3e380722b0 2024-03-30T01:41:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.974+0000 7f3e3cbd8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380c6670 con 0x7f3e3806db20 2024-03-30T01:41:20.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3d5ffff700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3d4c06ad10 0x7f3d4c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e35d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e380722b0 0x7f3e381e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e35d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e380722b0 0x7f3e381e90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38810/0 (socket says 172.21.15.31:38810) 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e35d9b700 1 -- 172.21.15.31:0/1119234923 learned_addr learned my addr 172.21.15.31:0/1119234923 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e3659c700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e3806db20 0x7f3e38151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e35d9b700 1 -- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e3806db20 msgr2=0x7f3e38151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e35d9b700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e3806db20 0x7f3e38151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e35d9b700 1 -- 172.21.15.31:0/1119234923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e38154d20 msgr2=0x7f3e38144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e35d9b700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e38154d20 0x7f3e38144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e35d9b700 1 -- 172.21.15.31:0/1119234923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3000bd40 con 0x7f3e380722b0 2024-03-30T01:41:20.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3e35d9b700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e380722b0 0x7f3e381e90e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b580 tx=0x7f3e2800b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.975+0000 7f3d5ffff700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3d4c06ad10 0x7f3d4c06d1d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000ab20 tx=0x7f3d5001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.976+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1119234923 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28011830 con 0x7f3e380722b0 2024-03-30T01:41:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.976+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1119234923 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e28011990 con 0x7f3e380722b0 2024-03-30T01:41:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.976+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1119234923 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28011d40 con 0x7f3e380722b0 2024-03-30T01:41:20.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.976+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e381fa800 con 0x7f3e380722b0 2024-03-30T01:41:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.976+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e380c68c0 con 0x7f3e380722b0 2024-03-30T01:41:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.978+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1119234923 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3e2801a400 con 0x7f3e380722b0 2024-03-30T01:41:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.979+0000 7f3d5f7fe700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6808/47360,v1:172.21.15.31:6809/47360] conn(0x7f3d400015e0 0x7f3d40003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.979+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e1806ac90 0x7f3e1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.979+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1119234923 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3e28089670 con 0x7f3e380722b0 2024-03-30T01:41:20.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.981+0000 7f3e3659c700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e1806ac90 0x7f3e1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.981+0000 7f3d5cff9700 1 -- 172.21.15.31:0/1251683102 <== osd.2 v2:172.21.15.31:6808/47360 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f3d40006bd0 con 0x7f3d400015e0 2024-03-30T01:41:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.981+0000 7f3e3659c700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e1806ac90 0x7f3e1806d150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009870 tx=0x7f3e2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.986+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 --> [v2:172.21.15.31:6800/47358,v1:172.21.15.31:6801/47358] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed381564c0 con 0x7fed1c0743a0 2024-03-30T01:41:20.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.986+0000 7fed2e7fc700 1 -- 172.21.15.31:0/2927541841 <== osd.0 v2:172.21.15.31:6800/47358 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed30001b60 con 0x7fed1c0743a0 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6800/47358,v1:172.21.15.31:6801/47358] conn(0x7fed1c0743a0 msgr2=0x7fed1c0767e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3fe99700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6800/47358,v1:172.21.15.31:6801/47358] conn(0x7fed1c0743a0 0x7fed1c0767e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fed1c06ad60 msgr2=0x7fed1c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3fe99700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fed1c06ad60 0x7fed1c06d220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fed38074050 tx=0x7fed34009040 comp rx=0 tx=0).stop 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed3806cf10 msgr2=0x7fed38158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3fe99700 1 --2- 172.21.15.31:0/2927541841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed3806cf10 0x7fed38158620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed38073850 tx=0x7fed28004750 comp rx=0 tx=0).stop 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3e436700 1 -- 172.21.15.31:0/2927541841 reap_dead start 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 shutdown_connections 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 >> 172.21.15.31:0/2927541841 conn(0x7fed380c2d60 msgr2=0x7fed38154600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.987+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 shutdown_connections 2024-03-30T01:41:20.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.988+0000 7fed3fe99700 1 -- 172.21.15.31:0/2927541841 wait complete. 2024-03-30T01:41:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.991+0000 7f3e3cbd8700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:6800/47501,v1:172.21.15.59:6801/47501] conn(0x7f3e381452c0 0x7f3e38144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.991+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 --> [v2:172.21.15.59:6800/47501,v1:172.21.15.59:6801/47501] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e380c2d30 con 0x7f3e381452c0 2024-03-30T01:41:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.991+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 --> [v2:172.21.15.31:6824/47374,v1:172.21.15.31:6825/47374] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efeb0002d40 con 0x7efeac0742d0 2024-03-30T01:41:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.991+0000 7f3e36d9d700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:6800/47501,v1:172.21.15.59:6801/47501] conn(0x7f3e381452c0 0x7f3e38144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.992+0000 7f3e36d9d700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:6800/47501,v1:172.21.15.59:6801/47501] conn(0x7f3e381452c0 0x7f3e38144ab0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 --> [v2:172.21.15.59:6808/47502,v1:172.21.15.59:6809/47502] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff89c154570 con 0x7ff880074250 2024-03-30T01:41:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.991+0000 7f0959acf700 1 Processor -- start 2024-03-30T01:41:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.991+0000 7f0959acf700 1 -- start start 2024-03-30T01:41:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.992+0000 7f0959acf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0954070b80 0x7f0954164a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.992+0000 7f0959acf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f095408a970 0x7f095415e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.992+0000 7f0959acf700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09540705d0 con 0x7f09540c2f90 2024-03-30T01:41:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.992+0000 7f0959acf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095406c700 con 0x7f0954070b80 2024-03-30T01:41:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.992+0000 7f0959acf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09540710f0 con 0x7f095408a970 2024-03-30T01:41:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1119234923 <== osd.5 v2:172.21.15.59:6800/47501 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f3e3000b230 con 0x7f3e381452c0 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7f0952ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0954070b80 0x7f0954164a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7f0952ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0954070b80 0x7f0954164a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38818/0 (socket says 172.21.15.31:38818) 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7f0952ffd700 1 -- 172.21.15.31:0/1387463705 learned_addr learned my addr 172.21.15.31:0/1387463705 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7f0953fff700 1 --2- 172.21.15.31:0/1387463705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f095408a970 0x7f095415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7f0952ffd700 1 -- 172.21.15.31:0/1387463705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f095408a970 msgr2=0x7f095415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7f0952ffd700 1 --2- 172.21.15.31:0/1387463705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f095408a970 0x7f095415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7f0952ffd700 1 -- 172.21.15.31:0/1387463705 >> v1:172.21.15.31:6789/0 conn(0x7f09540c2f90 legacy=0x7f095406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.993+0000 7f0952ffd700 1 -- 172.21.15.31:0/1387463705 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09540708b0 con 0x7f0954070b80 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.994+0000 7f0952ffd700 1 --2- 172.21.15.31:0/1387463705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0954070b80 0x7f0954164a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f09480089f0 tx=0x7f094800d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c22f3be04d50d6f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.994+0000 7f09527fc700 1 -- 172.21.15.31:0/1387463705 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948010070 con 0x7f0954070b80 2024-03-30T01:41:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.994+0000 7f09527fc700 1 -- 172.21.15.31:0/1387463705 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094800d910 con 0x7f0954070b80 2024-03-30T01:41:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.994+0000 7f09527fc700 1 -- 172.21.15.31:0/1387463705 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094800dcc0 con 0x7f0954070b80 2024-03-30T01:41:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.994+0000 7efebeffd700 1 -- 172.21.15.31:0/2360571317 <== osd.3 v2:172.21.15.31:6824/47374 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efeb0002d40 con 0x7efeac0742d0 2024-03-30T01:41:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 --> [v2:172.21.15.31:6816/47369,v1:172.21.15.31:6817/47369] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ef01f1e20 con 0x7f9ed40743a0 2024-03-30T01:41:20.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 --> [v2:172.21.15.59:6816/47512,v1:172.21.15.59:6817/47512] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4a8c002d40 con 0x7f4a90074320 2024-03-30T01:41:20.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6824/47374,v1:172.21.15.31:6825/47374] conn(0x7efeac0742d0 msgr2=0x7efeac076710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7efed0372700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6824/47374,v1:172.21.15.31:6825/47374] conn(0x7efeac0742d0 0x7efeac076710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.996+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efeac06ac90 msgr2=0x7efeac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.996+0000 7efed0372700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efeac06ac90 0x7efeac06d150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006fd0 tx=0x7efeb8008040 comp rx=0 tx=0).stop 2024-03-30T01:41:20.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.996+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80c5810 msgr2=0x7efec80c4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.996+0000 7efed0372700 1 --2- 172.21.15.31:0/2360571317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efec80c5810 0x7efec80c4c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efec000d690 tx=0x7efec0004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7f0959acf700 1 -- 172.21.15.31:0/1387463705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0954070b80 msgr2=0x7f0954164a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7f0959acf700 1 --2- 172.21.15.31:0/1387463705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0954070b80 0x7f0954164a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f09480089f0 tx=0x7f094800d330 comp rx=0 tx=0).stop 2024-03-30T01:41:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7f0959acf700 1 -- 172.21.15.31:0/1387463705 shutdown_connections 2024-03-30T01:41:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7f0959acf700 1 --2- 172.21.15.31:0/1387463705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f095408a970 0x7f095415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7f0959acf700 1 --2- 172.21.15.31:0/1387463705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0954070b80 0x7f0954164a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.995+0000 7f0959acf700 1 -- 172.21.15.31:0/1387463705 >> 172.21.15.31:0/1387463705 conn(0x7f095406aaf0 msgr2=0x7f095406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.996+0000 7f0959acf700 1 -- 172.21.15.31:0/1387463705 shutdown_connections 2024-03-30T01:41:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.996+0000 7f0959acf700 1 -- 172.21.15.31:0/1387463705 wait complete. 2024-03-30T01:41:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.997+0000 7f0959acf700 1 Processor -- start 2024-03-30T01:41:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.997+0000 7f0959acf700 1 -- start start 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0959acf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0954070b80 0x7f09542046e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0959acf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095408a970 0x7f0954204c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0959acf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09540c2f90 0x7f095420a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0959acf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540a8b40 con 0x7f09540c2f90 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0959acf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095420b460 con 0x7f095408a970 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0959acf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09540a8de0 con 0x7f0954070b80 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0953fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09540c2f90 0x7f095420a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0953fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09540c2f90 0x7f095420a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36614/0 (socket says 172.21.15.31:36614) 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0953fff700 1 -- 172.21.15.31:0/107356227 learned_addr learned my addr 172.21.15.31:0/107356227 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f0952ffd700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095408a970 0x7f0954204c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.998+0000 7f09537fe700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0954070b80 0x7f09542046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.999+0000 7f0952ffd700 1 -- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0954070b80 msgr2=0x7f09542046e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.999+0000 7f0952ffd700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0954070b80 0x7f09542046e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.999+0000 7f0952ffd700 1 -- 172.21.15.31:0/107356227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09540c2f90 msgr2=0x7f095420a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.999+0000 7f0952ffd700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09540c2f90 0x7f095420a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.999+0000 7f0952ffd700 1 -- 172.21.15.31:0/107356227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480086d0 con 0x7f095408a970 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.999+0000 7f0952ffd700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095408a970 0x7f0954204c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0948019000 tx=0x7f0948004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.999+0000 7f093bfff700 1 -- 172.21.15.31:0/107356227 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948016810 con 0x7f095408a970 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.999+0000 7f093bfff700 1 -- 172.21.15.31:0/107356227 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09480169b0 con 0x7f095408a970 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:19.999+0000 7f093bfff700 1 -- 172.21.15.31:0/107356227 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948016d60 con 0x7f095408a970 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.000+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09540a90c0 con 0x7f095408a970 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.000+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09540c0280 con 0x7f095408a970 2024-03-30T01:41:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.001+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f095406ddf0 con 0x7f095408a970 2024-03-30T01:41:20.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.001+0000 7efece90f700 1 -- 172.21.15.31:0/2360571317 reap_dead start 2024-03-30T01:41:20.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.001+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 shutdown_connections 2024-03-30T01:41:20.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.001+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 >> 172.21.15.31:0/2360571317 conn(0x7efec806aaf0 msgr2=0x7efec80c0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 shutdown_connections 2024-03-30T01:41:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.001+0000 7ff8927fc700 1 -- 172.21.15.31:0/2368764264 <== osd.7 v2:172.21.15.59:6808/47502 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff88c00d640 con 0x7ff880074250 2024-03-30T01:41:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7efed0372700 1 -- 172.21.15.31:0/2360571317 wait complete. 2024-03-30T01:41:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7f4aa1ffb700 1 -- 172.21.15.31:0/1159449219 <== osd.4 v2:172.21.15.59:6816/47512 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4a8c002d40 con 0x7f4a90074320 2024-03-30T01:41:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:6808/47502,v1:172.21.15.59:6809/47502] conn(0x7ff880074250 msgr2=0x7ff880076690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7ff8a3cc2700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:6808/47502,v1:172.21.15.59:6809/47502] conn(0x7ff880074250 0x7ff880076690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff88006ac90 msgr2=0x7ff88006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7ff8a3cc2700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff88006ac90 0x7ff88006d150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff894014040 tx=0x7ff894010000 comp rx=0 tx=0).stop 2024-03-30T01:41:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff89c150770 msgr2=0x7ff89c1f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7ff8a3cc2700 1 --2- 172.21.15.31:0/2368764264 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff89c150770 0x7ff89c1f1ec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c370 tx=0x7ff89800c680 comp rx=0 tx=0).stop 2024-03-30T01:41:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.002+0000 7f093bfff700 1 -- 172.21.15.31:0/107356227 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f094802a460 con 0x7f095408a970 2024-03-30T01:41:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f093bfff700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f094006ac90 0x7f094006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:6816/47512,v1:172.21.15.59:6817/47512] conn(0x7f4a90074320 msgr2=0x7f4a90076760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f4ab3743700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.59:6816/47512,v1:172.21.15.59:6817/47512] conn(0x7f4a90074320 0x7f4a90076760 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4a9006ace0 msgr2=0x7f4a9006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f4ab3743700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4a9006ace0 0x7f4a9006d1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0c23b0 tx=0x7f4a9c010000 comp rx=0 tx=0).stop 2024-03-30T01:41:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aac06c860 msgr2=0x7f4aac06d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f4ab3743700 1 --2- 172.21.15.31:0/1159449219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4aac06c860 0x7f4aac06d670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800c4b0 tx=0x7f4aa800c7c0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f4ab1ce0700 1 -- 172.21.15.31:0/1159449219 reap_dead start 2024-03-30T01:41:20.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f093bfff700 1 -- 172.21.15.31:0/107356227 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f094808b090 con 0x7f095408a970 2024-03-30T01:41:20.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f093bfff700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:6824/47524,v1:172.21.15.59:6825/47524] conn(0x7f09400742d0 0x7f0940076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f093bfff700 1 -- 172.21.15.31:0/107356227 --> [v2:172.21.15.59:6824/47524,v1:172.21.15.59:6825/47524] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0940076de0 con 0x7f09400742d0 2024-03-30T01:41:20.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f093bfff700 1 -- 172.21.15.31:0/107356227 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f094808b440 con 0x7f095408a970 2024-03-30T01:41:20.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f09537fe700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f094006ac90 0x7f094006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7f9ee6ffd700 1 -- 172.21.15.31:0/1356014160 <== osd.1 v2:172.21.15.31:6816/47369 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9ef01f1e20 con 0x7f9ed40743a0 2024-03-30T01:41:20.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 shutdown_connections 2024-03-30T01:41:20.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.003+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 >> 172.21.15.31:0/1159449219 conn(0x7f4aac06aaf0 msgr2=0x7f4aac06aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 shutdown_connections 2024-03-30T01:41:20.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7f4ab3743700 1 -- 172.21.15.31:0/1159449219 wait complete. 2024-03-30T01:41:20.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7f09537fe700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f094006ac90 0x7f094006d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0944007fd0 tx=0x7f0944009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7ff8a225f700 1 -- 172.21.15.31:0/2368764264 reap_dead start 2024-03-30T01:41:20.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 shutdown_connections 2024-03-30T01:41:20.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 >> 172.21.15.31:0/2368764264 conn(0x7ff89c06aaf0 msgr2=0x7ff89c144070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6816/47369,v1:172.21.15.31:6817/47369] conn(0x7f9ed40743a0 msgr2=0x7f9ed40767e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7f9ef846e700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6816/47369,v1:172.21.15.31:6817/47369] conn(0x7f9ed40743a0 0x7f9ed40767e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.004+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 shutdown_connections 2024-03-30T01:41:20.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.005+0000 7ff8a3cc2700 1 -- 172.21.15.31:0/2368764264 wait complete. 2024-03-30T01:41:20.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.005+0000 7f0953fff700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:6824/47524,v1:172.21.15.59:6825/47524] conn(0x7f09400742d0 0x7f0940076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.005+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ed406ad60 msgr2=0x7f9ed406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.005+0000 7f9ef846e700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ed406ad60 0x7f9ed406d220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009b60 tx=0x7f9eec008030 comp rx=0 tx=0).stop 2024-03-30T01:41:20.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.005+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ef0150110 msgr2=0x7f9ef0154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.005+0000 7f9ef846e700 1 --2- 172.21.15.31:0/1356014160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ef0150110 0x7f9ef0154330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800efc0 tx=0x7f9ee800c3a0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.005+0000 7f9ef6a0b700 1 -- 172.21.15.31:0/1356014160 reap_dead start 2024-03-30T01:41:20.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.005+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 shutdown_connections 2024-03-30T01:41:20.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.005+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 >> 172.21.15.31:0/1356014160 conn(0x7f9ef006ab20 msgr2=0x7f9ef014f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.051 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2024-03-30T01:41:20.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-30T01:41:20.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.009+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 shutdown_connections 2024-03-30T01:41:20.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.009+0000 7f9ef846e700 1 -- 172.21.15.31:0/1356014160 wait complete. 2024-03-30T01:41:20.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.011+0000 7f0953fff700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:6824/47524,v1:172.21.15.59:6825/47524] conn(0x7f09400742d0 0x7f0940076710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.012+0000 7f093bfff700 1 -- 172.21.15.31:0/107356227 <== osd.6 v2:172.21.15.59:6824/47524 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f0940076de0 con 0x7f09400742d0 2024-03-30T01:41:20.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.020+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 --> [v2:172.21.15.31:6808/47360,v1:172.21.15.31:6809/47360] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d40005cb0 con 0x7f3d400015e0 2024-03-30T01:41:20.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.021+0000 7f3d5cff9700 1 -- 172.21.15.31:0/1251683102 <== osd.2 v2:172.21.15.31:6808/47360 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3d40005cb0 con 0x7f3d400015e0 2024-03-30T01:41:20.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.021+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6808/47360,v1:172.21.15.31:6809/47360] conn(0x7f3d400015e0 msgr2=0x7f3d40003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.021+0000 7f3d669f5700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6808/47360,v1:172.21.15.31:6809/47360] conn(0x7f3d400015e0 0x7f3d40003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.022+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3d4c06ad10 msgr2=0x7f3d4c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.022+0000 7f3d669f5700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3d4c06ad10 0x7f3d4c06d1d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000ab20 tx=0x7f3d5001c040 comp rx=0 tx=0).stop 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.022+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d6014b920 msgr2=0x7f3d601efd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.022+0000 7f3d669f5700 1 --2- 172.21.15.31:0/1251683102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d6014b920 0x7f3d601efd20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d58001330 tx=0x7f3d5800e720 comp rx=0 tx=0).stop 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.022+0000 7f3d64f92700 1 -- 172.21.15.31:0/1251683102 reap_dead start 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.025+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 shutdown_connections 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.025+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 >> 172.21.15.31:0/1251683102 conn(0x7f3d600ca0d0 msgr2=0x7f3d6014e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.025+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 shutdown_connections 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.025+0000 7f3d669f5700 1 -- 172.21.15.31:0/1251683102 wait complete. 2024-03-30T01:41:20.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.030+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 --> [v2:172.21.15.59:6800/47501,v1:172.21.15.59:6801/47501] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3e380c3530 con 0x7f3e381452c0 2024-03-30T01:41:20.056 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2024-03-30T01:41:20.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-30T01:41:20.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.031+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1119234923 <== osd.5 v2:172.21.15.59:6800/47501 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3e30003a20 con 0x7f3e381452c0 2024-03-30T01:41:20.057 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:20.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-30T01:41:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.033+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:6800/47501,v1:172.21.15.59:6801/47501] conn(0x7f3e381452c0 msgr2=0x7f3e38144ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.033+0000 7f3e3cbd8700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:6800/47501,v1:172.21.15.59:6801/47501] conn(0x7f3e381452c0 0x7f3e38144ab0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.033+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e1806ac90 msgr2=0x7f3e1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.033+0000 7f3e3cbd8700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e1806ac90 0x7f3e1806d150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009870 tx=0x7f3e2c008040 comp rx=0 tx=0).stop 2024-03-30T01:41:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.033+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e380722b0 msgr2=0x7f3e381e90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.033+0000 7f3e3cbd8700 1 --2- 172.21.15.31:0/1119234923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e380722b0 0x7f3e381e90e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b580 tx=0x7f3e2800b940 comp rx=0 tx=0).stop 2024-03-30T01:41:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.033+0000 7f3e36d9d700 1 -- 172.21.15.31:0/1119234923 reap_dead start 2024-03-30T01:41:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.034+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 shutdown_connections 2024-03-30T01:41:20.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.034+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 >> 172.21.15.31:0/1119234923 conn(0x7f3e380c3a60 msgr2=0x7f3e381534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.060 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:20.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-30T01:41:20.061 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2024-03-30T01:41:20.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-30T01:41:20.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.037+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 shutdown_connections 2024-03-30T01:41:20.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.037+0000 7f3e3cbd8700 1 -- 172.21.15.31:0/1119234923 wait complete. 2024-03-30T01:41:20.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.042+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 --> [v2:172.21.15.59:6824/47524,v1:172.21.15.59:6825/47524] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f09540a92b0 con 0x7f09400742d0 2024-03-30T01:41:20.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.043+0000 7f093bfff700 1 -- 172.21.15.31:0/107356227 <== osd.6 v2:172.21.15.59:6824/47524 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f094c00f040 con 0x7f09400742d0 2024-03-30T01:41:20.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.043+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:6824/47524,v1:172.21.15.59:6825/47524] conn(0x7f09400742d0 msgr2=0x7f0940076710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0959acf700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:6824/47524,v1:172.21.15.59:6825/47524] conn(0x7f09400742d0 0x7f0940076710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f094006ac90 msgr2=0x7f094006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0959acf700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f094006ac90 0x7f094006d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0944007fd0 tx=0x7f0944009040 comp rx=0 tx=0).stop 2024-03-30T01:41:20.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095408a970 msgr2=0x7f0954204c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0959acf700 1 --2- 172.21.15.31:0/107356227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f095408a970 0x7f0954204c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0948019000 tx=0x7f0948004750 comp rx=0 tx=0).stop 2024-03-30T01:41:20.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0953fff700 1 -- 172.21.15.31:0/107356227 reap_dead start 2024-03-30T01:41:20.065 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:20.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T01:41:20.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 shutdown_connections 2024-03-30T01:41:20.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 >> 172.21.15.31:0/107356227 conn(0x7f095406aaf0 msgr2=0x7f095406aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 shutdown_connections 2024-03-30T01:41:20.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.044+0000 7f0959acf700 1 -- 172.21.15.31:0/107356227 wait complete. 2024-03-30T01:41:20.067 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2024-03-30T01:41:20.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-30T01:41:20.069 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2024-03-30T01:41:20.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-30T01:41:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.267+0000 7f600fbac700 1 Processor -- start 2024-03-30T01:41:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.268+0000 7f600fbac700 1 -- start start 2024-03-30T01:41:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.269+0000 7f600fbac700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6008070b80 0x7f6008164a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.269+0000 7f600fbac700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f600808a970 0x7f600815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.269+0000 7f600fbac700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60080705d0 con 0x7f60080c2f90 2024-03-30T01:41:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.269+0000 7f600fbac700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600806c700 con 0x7f6008070b80 2024-03-30T01:41:20.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.269+0000 7f600fbac700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080710f0 con 0x7f600808a970 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.270+0000 7f600e149700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f600808a970 0x7f600815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.270+0000 7f600e149700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f600808a970 0x7f600815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38838/0 (socket says 172.21.15.31:38838) 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.270+0000 7f600e149700 1 -- 172.21.15.31:0/3549156401 learned_addr learned my addr 172.21.15.31:0/3549156401 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.271+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89143699 0 0) 0x7f60080705d0 con 0x7f60080c2f90 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.272+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff40057c0 con 0x7f60080c2f90 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.271+0000 7f600d147700 1 --2- 172.21.15.31:0/3549156401 >> v2:172.21.15.31:3300/0 conn(0x7f6008070b80 0x7f6008164a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 <== mon.1 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 304925971 0 0) 0x7f5ff40057c0 con 0x7f60080c2f90 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 >> v2:172.21.15.59:3300/0 conn(0x7f600808a970 msgr2=0x7f600815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 --2- 172.21.15.31:0/3549156401 >> v2:172.21.15.59:3300/0 conn(0x7f600808a970 0x7f600815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 >> v2:172.21.15.31:3300/0 conn(0x7f6008070b80 msgr2=0x7f6008164a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 --2- 172.21.15.31:0/3549156401 >> v2:172.21.15.31:3300/0 conn(0x7f6008070b80 0x7f6008164a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080708b0 con 0x7f60080c2f90 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 <== mon.1 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f6004001670 con 0x7f60080c2f90 2024-03-30T01:41:20.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 >> v1:172.21.15.59:6789/0 conn(0x7f60080c2f90 legacy=0x7f600806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff4006570 0x7f5ff4006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4006ff0 0x7f5ff40074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600806c700 con 0x7f5ff4006570 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60080710f0 con 0x7f5ff4005c10 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff401cfd0 con 0x7f5ff4006ff0 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.273+0000 7f600d948700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.274+0000 7f600e149700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4006ff0 0x7f5ff40074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.274+0000 7f600d948700 1 -- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4006ff0 msgr2=0x7f5ff40074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.274+0000 7f600d948700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4006ff0 0x7f5ff40074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.274+0000 7f600d948700 1 -- 172.21.15.31:0/3549156401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff4006570 msgr2=0x7f5ff4006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.274+0000 7f600d948700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ff4006570 0x7f5ff4006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.274+0000 7f600d948700 1 -- 172.21.15.31:0/3549156401 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080708b0 con 0x7f5ff4005c10 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.274+0000 7f600e149700 1 -- 172.21.15.31:0/3549156401 reap_dead start 2024-03-30T01:41:20.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.275+0000 7f600d948700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f600400c260 tx=0x7f5ff4007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa5bf1cb6b7e728 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.275+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004007b80 con 0x7f5ff4005c10 2024-03-30T01:41:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.275+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600400cd60 con 0x7f5ff4005c10 2024-03-30T01:41:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.275+0000 7f600c946700 1 -- 172.21.15.31:0/3549156401 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004012660 con 0x7f5ff4005c10 2024-03-30T01:41:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.276+0000 7f600fbac700 1 -- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 msgr2=0x7f5ff4006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.276+0000 7f600fbac700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f600400c260 tx=0x7f5ff4007a90 comp rx=0 tx=0).stop 2024-03-30T01:41:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.276+0000 7f600fbac700 1 -- 172.21.15.31:0/3549156401 shutdown_connections 2024-03-30T01:41:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.276+0000 7f600fbac700 1 --2- 172.21.15.31:0/3549156401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4005c10 0x7f5ff4006030 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.276+0000 7f600fbac700 1 -- 172.21.15.31:0/3549156401 >> 172.21.15.31:0/3549156401 conn(0x7f600806aaf0 msgr2=0x7f600806af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.277+0000 7f600fbac700 1 -- 172.21.15.31:0/3549156401 shutdown_connections 2024-03-30T01:41:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.277+0000 7f600fbac700 1 -- 172.21.15.31:0/3549156401 wait complete. 2024-03-30T01:41:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.279+0000 7f600fbac700 1 Processor -- start 2024-03-30T01:41:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.279+0000 7f600fbac700 1 -- start start 2024-03-30T01:41:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.280+0000 7f600fbac700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005c10 0x7f60080a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.280+0000 7f600fbac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080aaef0 0x7f60080c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.280+0000 7f600fbac700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60080bfcc0 0x7f6008158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.280+0000 7f600fbac700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008165780 con 0x7f60080bfcc0 2024-03-30T01:41:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.280+0000 7f600fbac700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60080aa1a0 con 0x7f60080aaef0 2024-03-30T01:41:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.280+0000 7f600fbac700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60081654a0 con 0x7f5ff4005c10 2024-03-30T01:41:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.281+0000 7f600d147700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080aaef0 0x7f60080c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.281+0000 7f600d147700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080aaef0 0x7f60080c6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38862/0 (socket says 172.21.15.31:38862) 2024-03-30T01:41:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.281+0000 7f600d147700 1 -- 172.21.15.31:0/1307672145 learned_addr learned my addr 172.21.15.31:0/1307672145 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.281+0000 7f600d948700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005c10 0x7f60080a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.282+0000 7f600d147700 1 -- 172.21.15.31:0/1307672145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005c10 msgr2=0x7f60080a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.282+0000 7f600d147700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005c10 0x7f60080a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.282+0000 7f600d147700 1 -- 172.21.15.31:0/1307672145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60080bfcc0 msgr2=0x7f6008158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.282+0000 7f600d147700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60080bfcc0 0x7f6008158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.282+0000 7f600d147700 1 -- 172.21.15.31:0/1307672145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008162650 con 0x7f60080aaef0 2024-03-30T01:41:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.282+0000 7f600d147700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080aaef0 0x7f60080c6070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800e930 tx=0x7f5ff800ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.284+0000 7f5ffe7fc700 1 -- 172.21.15.31:0/1307672145 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800c850 con 0x7f60080aaef0 2024-03-30T01:41:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.284+0000 7f5ffe7fc700 1 -- 172.21.15.31:0/1307672145 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff800c9b0 con 0x7f60080aaef0 2024-03-30T01:41:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.284+0000 7f5ffe7fc700 1 -- 172.21.15.31:0/1307672145 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800cd60 con 0x7f60080aaef0 2024-03-30T01:41:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.284+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6008162930 con 0x7f60080aaef0 2024-03-30T01:41:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.284+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6008165130 con 0x7f60080aaef0 2024-03-30T01:41:20.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.287+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff00052f0 con 0x7f60080aaef0 2024-03-30T01:41:20.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.289+0000 7f5ffe7fc700 1 -- 172.21.15.31:0/1307672145 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5ff8013470 con 0x7f60080aaef0 2024-03-30T01:41:20.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.289+0000 7f5ffe7fc700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5fec06ad60 0x7f5fec06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.290+0000 7f5ffe7fc700 1 -- 172.21.15.31:0/1307672145 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5ff8089540 con 0x7f60080aaef0 2024-03-30T01:41:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.292+0000 7f600d948700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5fec06ad60 0x7f5fec06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.292+0000 7f600d948700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5fec06ad60 0x7f5fec06d220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6004014040 tx=0x7f6004009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.296+0000 7f5ffe7fc700 1 -- 172.21.15.31:0/1307672145 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5ff80544a0 con 0x7f60080aaef0 2024-03-30T01:41:20.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.313+0000 7f2749b08700 1 Processor -- start 2024-03-30T01:41:20.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.313+0000 7f2749b08700 1 -- start start 2024-03-30T01:41:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.315+0000 7f2749b08700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f27440ab120 0x7f2744070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.315+0000 7f2749b08700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2744150770 0x7f27440c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.315+0000 7f2749b08700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2744153d60 con 0x7f2744071100 2024-03-30T01:41:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.315+0000 7f2749b08700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2744071e70 con 0x7f27440ab120 2024-03-30T01:41:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.315+0000 7f2749b08700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27440c1450 con 0x7f2744150770 2024-03-30T01:41:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f2743fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2744071100 0x7f27440c0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56456/0 (socket says 172.21.15.31:56456) 2024-03-30T01:41:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f2743fff700 1 -- 172.21.15.31:0/4130099065 learned_addr learned my addr 172.21.15.31:0/4130099065 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27437fe700 1 --2- 172.21.15.31:0/4130099065 >> v2:172.21.15.31:3300/0 conn(0x7f27440ab120 0x7f2744070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f2742ffd700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2744150770 0x7f27440c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115056032 0 0) 0x7f2744153d60 con 0x7f2744071100 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f272c0035b0 con 0x7f2744071100 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 <== mon.1 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2690988595 0 0) 0x7f272c0035b0 con 0x7f2744071100 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2744150770 msgr2=0x7f27440c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27427fc700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2744150770 0x7f27440c0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 >> v2:172.21.15.31:3300/0 conn(0x7f27440ab120 msgr2=0x7f2744070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27427fc700 1 --2- 172.21.15.31:0/4130099065 >> v2:172.21.15.31:3300/0 conn(0x7f27440ab120 0x7f2744070bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27441548e0 con 0x7f2744071100 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.316+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 <== mon.1 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f2734004ef0 con 0x7f2744071100 2024-03-30T01:41:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 >> v1:172.21.15.59:6789/0 conn(0x7f2744071100 legacy=0x7f27440c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f27427fc700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f272c003a00 0x7f272c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f27427fc700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f272c004360 0x7f272c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f27427fc700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c004de0 0x7f272c005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0102d0 con 0x7f272c004de0 2024-03-30T01:41:20.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c010440 con 0x7f272c003a00 2024-03-30T01:41:20.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c010600 con 0x7f272c004360 2024-03-30T01:41:20.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f2743fff700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c004de0 0x7f272c005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f27437fe700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f272c003a00 0x7f272c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f2743fff700 1 -- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f272c004360 msgr2=0x7f272c004850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:20.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f2743fff700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f272c004360 0x7f272c004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f2743fff700 1 -- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f272c003a00 msgr2=0x7f272c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f2743fff700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f272c003a00 0x7f272c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.317+0000 7f2743fff700 1 -- 172.21.15.31:0/4130099065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27441548e0 con 0x7f272c004de0 2024-03-30T01:41:20.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.318+0000 7f2743fff700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c004de0 0x7f272c005260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f273400d3d0 tx=0x7f272c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dfa83185b8ad5e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.319+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273400f400 con 0x7f272c004de0 2024-03-30T01:41:20.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.319+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273400f560 con 0x7f272c004de0 2024-03-30T01:41:20.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.319+0000 7f27427fc700 1 -- 172.21.15.31:0/4130099065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273400f8d0 con 0x7f272c004de0 2024-03-30T01:41:20.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.320+0000 7f2749b08700 1 -- 172.21.15.31:0/4130099065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c004de0 msgr2=0x7f272c005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.320+0000 7f2749b08700 1 --2- 172.21.15.31:0/4130099065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f272c004de0 0x7f272c005260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f273400d3d0 tx=0x7f272c005880 comp rx=0 tx=0).stop 2024-03-30T01:41:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.320+0000 7f2743fff700 1 -- 172.21.15.31:0/4130099065 reap_dead start 2024-03-30T01:41:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.320+0000 7f2749b08700 1 -- 172.21.15.31:0/4130099065 shutdown_connections 2024-03-30T01:41:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.320+0000 7f2749b08700 1 -- 172.21.15.31:0/4130099065 >> 172.21.15.31:0/4130099065 conn(0x7f274406aaf0 msgr2=0x7f274406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.321+0000 7f2749b08700 1 -- 172.21.15.31:0/4130099065 shutdown_connections 2024-03-30T01:41:20.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.321+0000 7f2749b08700 1 -- 172.21.15.31:0/4130099065 wait complete. 2024-03-30T01:41:20.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.323+0000 7f2749b08700 1 Processor -- start 2024-03-30T01:41:20.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.323+0000 7f2749b08700 1 -- start start 2024-03-30T01:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.324+0000 7f2749b08700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2744152450 0x7f2744147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.324+0000 7f2749b08700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27440c6000 0x7f2744147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.324+0000 7f2749b08700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274414c080 0x7f274414c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.324+0000 7f2749b08700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2744153070 con 0x7f274414c080 2024-03-30T01:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.324+0000 7f2749b08700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2744149c70 con 0x7f2744152450 2024-03-30T01:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.324+0000 7f2749b08700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274414a930 con 0x7f27440c6000 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.324+0000 7f2742ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27440c6000 0x7f2744147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.324+0000 7f2742ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27440c6000 0x7f2744147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60054/0 (socket says 172.21.15.31:60054) 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.324+0000 7f2742ffd700 1 -- 172.21.15.31:0/3092192885 learned_addr learned my addr 172.21.15.31:0/3092192885 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.325+0000 7f27437fe700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2744152450 0x7f2744147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.325+0000 7f2742ffd700 1 -- 172.21.15.31:0/3092192885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2744152450 msgr2=0x7f2744147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.325+0000 7f2742ffd700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2744152450 0x7f2744147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.325+0000 7f2742ffd700 1 -- 172.21.15.31:0/3092192885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274414c080 msgr2=0x7f274414c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.325+0000 7f2742ffd700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274414c080 0x7f274414c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.325+0000 7f2742ffd700 1 -- 172.21.15.31:0/3092192885 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744149660 con 0x7f27440c6000 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.325+0000 7f2742ffd700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27440c6000 0x7f2744147f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f273800d610 tx=0x7f273800d9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.326+0000 7f272bfff700 1 -- 172.21.15.31:0/3092192885 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738011870 con 0x7f27440c6000 2024-03-30T01:41:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.326+0000 7f272bfff700 1 -- 172.21.15.31:0/3092192885 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27380119d0 con 0x7f27440c6000 2024-03-30T01:41:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.326+0000 7f272bfff700 1 -- 172.21.15.31:0/3092192885 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738011d60 con 0x7f27440c6000 2024-03-30T01:41:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.327+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2744148490 con 0x7f27440c6000 2024-03-30T01:41:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.327+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2744149900 con 0x7f27440c6000 2024-03-30T01:41:20.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.328+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274404ea80 con 0x7f27440c6000 2024-03-30T01:41:20.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.329+0000 7f272bfff700 1 -- 172.21.15.31:0/3092192885 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2738019400 con 0x7f27440c6000 2024-03-30T01:41:20.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.330+0000 7f272bfff700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f273006ace0 0x7f273006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.330+0000 7f272bfff700 1 -- 172.21.15.31:0/3092192885 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2738089450 con 0x7f27440c6000 2024-03-30T01:41:20.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.334+0000 7f27437fe700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f273006ace0 0x7f273006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.335+0000 7f35fbeb1700 1 Processor -- start 2024-03-30T01:41:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.335+0000 7f272bfff700 1 -- 172.21.15.31:0/3092192885 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f27380544b0 con 0x7f27440c6000 2024-03-30T01:41:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.335+0000 7f27437fe700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f273006ace0 0x7f273006d1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f273c009fc0 tx=0x7f273c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.335+0000 7f35fbeb1700 1 -- start start 2024-03-30T01:41:20.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.335+0000 7fab31a00700 1 Processor -- start 2024-03-30T01:41:20.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.335+0000 7fab31a00700 1 -- start start 2024-03-30T01:41:20.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.336+0000 7f35fbeb1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35f40c4130 0x7f35f40c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.336+0000 7f35fbeb1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35f40c3620 0x7f35f40a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.336+0000 7f35fbeb1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f4152780 con 0x7f35f40a7440 2024-03-30T01:41:20.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.336+0000 7f35fbeb1700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f40c7890 con 0x7f35f40c3620 2024-03-30T01:41:20.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.336+0000 7f35fbeb1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f4152570 con 0x7f35f40c4130 2024-03-30T01:41:20.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7f35f944c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35f40c3620 0x7f35f40a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7f35fa44e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f35f40a7440 0x7f35f4152c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56468/0 (socket says 172.21.15.31:56468) 2024-03-30T01:41:20.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7f35fa44e700 1 -- 172.21.15.31:0/876145694 learned_addr learned my addr 172.21.15.31:0/876145694 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:20.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7fab31a00700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab2c072a00 0x7fab2c14e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7fab31a00700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7fab31a00700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c151cc0 con 0x7fab2c071670 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7fab31a00700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c152550 con 0x7fab2c072a00 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7fab31a00700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c151f90 con 0x7fab2c14ed90 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7fab2a7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab2c072a00 0x7fab2c14e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60070/0 (socket says 172.21.15.31:60070) 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7fab2b7fe700 1 -- 172.21.15.31:0/2740549057 learned_addr learned my addr 172.21.15.31:0/2740549057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7f35f944c700 1 -- 172.21.15.31:0/876145694 >> v2:172.21.15.59:3300/0 conn(0x7f35f40c4130 msgr2=0x7f35f40c27b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7f35f944c700 1 --2- 172.21.15.31:0/876145694 >> v2:172.21.15.59:3300/0 conn(0x7f35f40c4130 0x7f35f40c27b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7f35f944c700 1 -- 172.21.15.31:0/876145694 >> v1:172.21.15.59:6789/0 conn(0x7f35f40a7440 legacy=0x7f35f4152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.337+0000 7f35f944c700 1 -- 172.21.15.31:0/876145694 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f4070520 con 0x7f35f40c3620 2024-03-30T01:41:20.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.338+0000 7fab2b7fe700 1 -- 172.21.15.31:0/2740549057 >> v2:172.21.15.31:3300/0 conn(0x7fab2c072a00 msgr2=0x7fab2c14e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.338+0000 7fab2b7fe700 1 --2- 172.21.15.31:0/2740549057 >> v2:172.21.15.31:3300/0 conn(0x7fab2c072a00 0x7fab2c14e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.338+0000 7fab2b7fe700 1 -- 172.21.15.31:0/2740549057 >> v1:172.21.15.31:6789/0 conn(0x7fab2c071670 legacy=0x7fab2c1515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.338+0000 7fab2b7fe700 1 -- 172.21.15.31:0/2740549057 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c152270 con 0x7fab2c14ed90 2024-03-30T01:41:20.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.338+0000 7f35f944c700 1 --2- 172.21.15.31:0/876145694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c3620 0x7f35f40a6f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35f40aabc0 tx=0x7f35f0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ccd444fbd71339 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.338+0000 7fab2b7fe700 1 --2- 172.21.15.31:0/2740549057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0c5790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fab2400d150 tx=0x7fab2400d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b33653f29b1480 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.339+0000 7f7bb913d700 1 Processor -- start 2024-03-30T01:41:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.339+0000 7f160102f700 1 Processor -- start 2024-03-30T01:41:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.339+0000 7f160102f700 1 -- start start 2024-03-30T01:41:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.339+0000 7f7bb913d700 1 -- start start 2024-03-30T01:41:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.340+0000 7f7bb913d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406e0a0 0x7f7bb414db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.340+0000 7f7bb913d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb40713c0 con 0x7f7bb4070a10 2024-03-30T01:41:20.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.340+0000 7f7bb913d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb4151550 con 0x7f7bb406d450 2024-03-30T01:41:20.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.340+0000 7f7bb913d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4072350 con 0x7f7bb406e0a0 2024-03-30T01:41:20.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f160102f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15fc154cf0 0x7f15fc0c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f160102f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15fc0c46a0 0x7f15fc06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f160102f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc0a8460 con 0x7f15fc072270 2024-03-30T01:41:20.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f160102f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc0c2e40 con 0x7f15fc154cf0 2024-03-30T01:41:20.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f160102f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc155630 con 0x7f15fc0c46a0 2024-03-30T01:41:20.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f7bb259c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406e0a0 0x7f7bb414db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f7bb259c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406e0a0 0x7f7bb414db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60074/0 (socket says 172.21.15.31:60074) 2024-03-30T01:41:20.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f7bb259c700 1 -- 172.21.15.31:0/2868648300 learned_addr learned my addr 172.21.15.31:0/2868648300 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f15fb59e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f15fc072270 0x7f15fc14df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56490/0 (socket says 172.21.15.31:56490) 2024-03-30T01:41:20.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f15fb59e700 1 -- 172.21.15.31:0/1680571826 learned_addr learned my addr 172.21.15.31:0/1680571826 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:20.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f7bb259c700 1 -- 172.21.15.31:0/2868648300 >> v1:172.21.15.59:6789/0 conn(0x7f7bb406d450 legacy=0x7f7bb415b170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T01:41:20.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f7bb259c700 1 -- 172.21.15.31:0/2868648300 >> v1:172.21.15.31:6789/0 conn(0x7f7bb4070a10 legacy=0x7f7bb40c0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f7bb259c700 1 -- 172.21.15.31:0/2868648300 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb406c970 con 0x7f7bb406e0a0 2024-03-30T01:41:20.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f15fad9d700 1 --2- 172.21.15.31:0/1680571826 >> v2:172.21.15.31:3300/0 conn(0x7f15fc154cf0 0x7f15fc0c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.341+0000 7f7bb259c700 1 --2- 172.21.15.31:0/2868648300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406e0a0 0x7f7bb414db60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8008920 tx=0x7f7ba800d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ecc95e0b7e0acf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.342+0000 7f7bb1d9b700 1 -- 172.21.15.31:0/2868648300 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8010070 con 0x7f7bb406e0a0 2024-03-30T01:41:20.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.342+0000 7f7bb1d9b700 1 -- 172.21.15.31:0/2868648300 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba800da00 con 0x7f7bb406e0a0 2024-03-30T01:41:20.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.342+0000 7f7bb1d9b700 1 -- 172.21.15.31:0/2868648300 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800dd90 con 0x7f7bb406e0a0 2024-03-30T01:41:20.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.342+0000 7f15fad9d700 1 -- 172.21.15.31:0/1680571826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15fc0c46a0 msgr2=0x7f15fc06d5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:20.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.342+0000 7f15fad9d700 1 --2- 172.21.15.31:0/1680571826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15fc0c46a0 0x7f15fc06d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.342+0000 7f15fad9d700 1 -- 172.21.15.31:0/1680571826 >> v1:172.21.15.59:6789/0 conn(0x7f15fc072270 legacy=0x7f15fc14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.342+0000 7f15fad9d700 1 -- 172.21.15.31:0/1680571826 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc0a86b0 con 0x7f15fc154cf0 2024-03-30T01:41:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.342+0000 7f15fad9d700 1 --2- 172.21.15.31:0/1680571826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15fc154cf0 0x7f15fc0c32d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009840 tx=0x7f15ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f437f3f73a07f8b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.343+0000 7f7bb913d700 1 -- 172.21.15.31:0/2868648300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406e0a0 msgr2=0x7f7bb414db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.343+0000 7f7bb913d700 1 --2- 172.21.15.31:0/2868648300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406e0a0 0x7f7bb414db60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8008920 tx=0x7f7ba800d330 comp rx=0 tx=0).stop 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.343+0000 7f15f9d9b700 1 -- 172.21.15.31:0/1680571826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec004230 con 0x7f15fc154cf0 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.343+0000 7f15f9d9b700 1 -- 172.21.15.31:0/1680571826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec004390 con 0x7f15fc154cf0 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.343+0000 7f15f9d9b700 1 -- 172.21.15.31:0/1680571826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0036a0 con 0x7f15fc154cf0 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f160102f700 1 -- 172.21.15.31:0/1680571826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15fc154cf0 msgr2=0x7f15fc0c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f160102f700 1 --2- 172.21.15.31:0/1680571826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15fc154cf0 0x7f15fc0c32d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009840 tx=0x7f15ec009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f160102f700 1 -- 172.21.15.31:0/1680571826 shutdown_connections 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f160102f700 1 --2- 172.21.15.31:0/1680571826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15fc154cf0 0x7f15fc0c32d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f160102f700 1 --2- 172.21.15.31:0/1680571826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15fc0c46a0 0x7f15fc06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f160102f700 1 -- 172.21.15.31:0/1680571826 >> 172.21.15.31:0/1680571826 conn(0x7f15fc0c3a60 msgr2=0x7f15fc0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f160102f700 1 -- 172.21.15.31:0/1680571826 shutdown_connections 2024-03-30T01:41:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.339+0000 7f35f8c4b700 1 -- 172.21.15.31:0/876145694 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0004230 con 0x7f35f40c3620 2024-03-30T01:41:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.339+0000 7f35f8c4b700 1 -- 172.21.15.31:0/876145694 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f0004390 con 0x7f35f40c3620 2024-03-30T01:41:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.339+0000 7f35f8c4b700 1 -- 172.21.15.31:0/876145694 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f00036a0 con 0x7f35f40c3620 2024-03-30T01:41:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.339+0000 7f35fbeb1700 1 -- 172.21.15.31:0/876145694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c3620 msgr2=0x7f35f40a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.339+0000 7f35fbeb1700 1 --2- 172.21.15.31:0/876145694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c3620 0x7f35f40a6f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35f40aabc0 tx=0x7f35f0009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f7bb913d700 1 -- 172.21.15.31:0/2868648300 shutdown_connections 2024-03-30T01:41:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f7bb913d700 1 --2- 172.21.15.31:0/2868648300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406e0a0 0x7f7bb414db60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f7bb913d700 1 -- 172.21.15.31:0/2868648300 >> 172.21.15.31:0/2868648300 conn(0x7f7bb40c2d60 msgr2=0x7f7bb40c3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.345+0000 7f7bb913d700 1 -- 172.21.15.31:0/2868648300 shutdown_connections 2024-03-30T01:41:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7fab29ffb700 1 -- 172.21.15.31:0/2740549057 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24010070 con 0x7fab2c14ed90 2024-03-30T01:41:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.345+0000 7fab29ffb700 1 -- 172.21.15.31:0/2740549057 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2400dd80 con 0x7fab2c14ed90 2024-03-30T01:41:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.345+0000 7fab29ffb700 1 -- 172.21.15.31:0/2740549057 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2400b310 con 0x7fab2c14ed90 2024-03-30T01:41:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7fab31a00700 1 -- 172.21.15.31:0/2740549057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 msgr2=0x7fab2c0c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7fab31a00700 1 --2- 172.21.15.31:0/2740549057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0c5790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fab2400d150 tx=0x7fab2400d460 comp rx=0 tx=0).stop 2024-03-30T01:41:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7fab31a00700 1 -- 172.21.15.31:0/2740549057 shutdown_connections 2024-03-30T01:41:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7fab31a00700 1 --2- 172.21.15.31:0/2740549057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0c5790 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7fab31a00700 1 --2- 172.21.15.31:0/2740549057 >> v2:172.21.15.31:3300/0 conn(0x7fab2c072a00 0x7fab2c14e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7fab31a00700 1 -- 172.21.15.31:0/2740549057 >> 172.21.15.31:0/2740549057 conn(0x7fab2c0a82d0 msgr2=0x7fab2c0a86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.347+0000 7fab31a00700 1 -- 172.21.15.31:0/2740549057 shutdown_connections 2024-03-30T01:41:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.345+0000 7f7bb913d700 1 -- 172.21.15.31:0/2868648300 wait complete. 2024-03-30T01:41:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.344+0000 7f160102f700 1 -- 172.21.15.31:0/1680571826 wait complete. 2024-03-30T01:41:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7f160102f700 1 Processor -- start 2024-03-30T01:41:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7f160102f700 1 -- start start 2024-03-30T01:41:20.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7f35fbeb1700 1 -- 172.21.15.31:0/876145694 shutdown_connections 2024-03-30T01:41:20.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7f35fbeb1700 1 --2- 172.21.15.31:0/876145694 >> v2:172.21.15.59:3300/0 conn(0x7f35f40c4130 0x7f35f40c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7f35fbeb1700 1 --2- 172.21.15.31:0/876145694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c3620 0x7f35f40a6f00 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35f40aabc0 tx=0x7f35f0009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:20.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.346+0000 7f35fbeb1700 1 -- 172.21.15.31:0/876145694 >> 172.21.15.31:0/876145694 conn(0x7f35f40c7e30 msgr2=0x7f35f40c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.347+0000 7f7bb913d700 1 Processor -- start 2024-03-30T01:41:20.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.347+0000 7f7bb913d700 1 -- start start 2024-03-30T01:41:20.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb913d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406d450 0x7f7bb41fa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb913d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb406e0a0 0x7f7bb40c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb913d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb4070a10 0x7f7bb41db520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb913d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40ac360 con 0x7f7bb406e0a0 2024-03-30T01:41:20.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb913d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb41fa020 con 0x7f7bb4070a10 2024-03-30T01:41:20.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb913d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb41fa980 con 0x7f7bb406d450 2024-03-30T01:41:20.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.348+0000 7f160102f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15fc072270 0x7f15fc13da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.348+0000 7f160102f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15fc0c46a0 0x7f15fc140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.348+0000 7f160102f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc154cf0 0x7f15fc151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.348+0000 7f160102f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc1ef000 con 0x7f15fc0c46a0 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.348+0000 7f160102f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc15e010 con 0x7f15fc154cf0 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.348+0000 7f160102f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc150780 con 0x7f15fc072270 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.348+0000 7f15fb59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc154cf0 0x7f15fc151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.348+0000 7f15fb59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc154cf0 0x7f15fc151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38876/0 (socket says 172.21.15.31:38876) 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.348+0000 7f15fb59e700 1 -- 172.21.15.31:0/2675197366 learned_addr learned my addr 172.21.15.31:0/2675197366 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15fad9d700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15fc072270 0x7f15fc13da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15fb59e700 1 -- 172.21.15.31:0/2675197366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15fc072270 msgr2=0x7f15fc13da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15fb59e700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15fc072270 0x7f15fc13da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15fb59e700 1 -- 172.21.15.31:0/2675197366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15fc0c46a0 msgr2=0x7f15fc140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15fb59e700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15fc0c46a0 0x7f15fc140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15fb59e700 1 -- 172.21.15.31:0/2675197366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec0094a0 con 0x7f15fc154cf0 2024-03-30T01:41:20.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15fb59e700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc154cf0 0x7f15fc151820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15f4002000 tx=0x7f15f400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb4070a10 0x7f7bb41db520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb4070a10 0x7f7bb41db520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38890/0 (socket says 172.21.15.31:38890) 2024-03-30T01:41:20.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb359e700 1 -- 172.21.15.31:0/3643199745 learned_addr learned my addr 172.21.15.31:0/3643199745 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f7bb259c700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb406e0a0 0x7f7bb40c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15e37fe700 1 -- 172.21.15.31:0/2675197366 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f400ad40 con 0x7f15fc154cf0 2024-03-30T01:41:20.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15e37fe700 1 -- 172.21.15.31:0/2675197366 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f40070d0 con 0x7f15fc154cf0 2024-03-30T01:41:20.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f15e37fe700 1 -- 172.21.15.31:0/2675197366 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f400aea0 con 0x7f15fc154cf0 2024-03-30T01:41:20.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc150a60 con 0x7f15fc154cf0 2024-03-30T01:41:20.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc151020 con 0x7f15fc154cf0 2024-03-30T01:41:20.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.350+0000 7f7bb259c700 1 -- 172.21.15.31:0/3643199745 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406d450 msgr2=0x7f7bb41fa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.350+0000 7f7bb259c700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406d450 0x7f7bb41fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.350+0000 7f7bb259c700 1 -- 172.21.15.31:0/3643199745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb4070a10 msgr2=0x7f7bb41db520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.350+0000 7f7bb259c700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb4070a10 0x7f7bb41db520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.350+0000 7f7bb259c700 1 -- 172.21.15.31:0/3643199745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba80085d0 con 0x7f7bb406e0a0 2024-03-30T01:41:20.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.351+0000 7f7bb259c700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb406e0a0 0x7f7bb40c4700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8018000 tx=0x7f7ba8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15dc0052f0 con 0x7f15fc154cf0 2024-03-30T01:41:20.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7f15e37fe700 1 -- 172.21.15.31:0/2675197366 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f15f4007440 con 0x7f15fc154cf0 2024-03-30T01:41:20.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.347+0000 7fab31a00700 1 -- 172.21.15.31:0/2740549057 wait complete. 2024-03-30T01:41:20.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.349+0000 7fab31a00700 1 Processor -- start 2024-03-30T01:41:20.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.350+0000 7fab31a00700 1 -- start start 2024-03-30T01:41:20.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.351+0000 7fab31a00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab2c071670 0x7fab2c208100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.351+0000 7fab31a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab2c072a00 0x7fab2c0a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.351+0000 7fab31a00700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0bf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.351+0000 7fab31a00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c155070 con 0x7fab2c071670 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.351+0000 7fab31a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0ab110 con 0x7fab2c072a00 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.351+0000 7fab31a00700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1547d0 con 0x7fab2c14ed90 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.351+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0bf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.351+0000 7fab2b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0bf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60102/0 (socket says 172.21.15.31:60102) 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7fab2b7fe700 1 -- 172.21.15.31:0/1920110096 learned_addr learned my addr 172.21.15.31:0/1920110096 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7fab2a7fc700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab2c072a00 0x7fab2c0a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7fab2affd700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab2c071670 0x7fab2c208100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7fab2a7fc700 1 -- 172.21.15.31:0/1920110096 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 msgr2=0x7fab2c0bf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7fab2a7fc700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0bf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7fab2a7fc700 1 -- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab2c071670 msgr2=0x7fab2c208100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7fab2a7fc700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab2c071670 0x7fab2c208100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.352+0000 7fab2a7fc700 1 -- 172.21.15.31:0/1920110096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2400bd40 con 0x7fab2c072a00 2024-03-30T01:41:20.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.353+0000 7f15e37fe700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15e806ad60 0x7f15e806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.353+0000 7f35fbeb1700 1 -- 172.21.15.31:0/876145694 shutdown_connections 2024-03-30T01:41:20.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.353+0000 7f15e37fe700 1 -- 172.21.15.31:0/2675197366 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f15f40895e0 con 0x7f15fc154cf0 2024-03-30T01:41:20.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.353+0000 7f35fbeb1700 1 -- 172.21.15.31:0/876145694 wait complete. 2024-03-30T01:41:20.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.355+0000 7f35fbeb1700 1 Processor -- start 2024-03-30T01:41:20.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.355+0000 7f35fbeb1700 1 -- start start 2024-03-30T01:41:20.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.356+0000 7f35fbeb1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35f40a7440 0x7f35f40adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.356+0000 7f35fbeb1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c4130 0x7f35f41ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.356+0000 7f35fbeb1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f41f2620 0x7f35f4144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.356+0000 7f35fbeb1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f41ef620 con 0x7f35f40c4130 2024-03-30T01:41:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.356+0000 7f35fbeb1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f41f1bd0 con 0x7f35f41f2620 2024-03-30T01:41:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.356+0000 7f35fbeb1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f41f2190 con 0x7f35f40a7440 2024-03-30T01:41:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.357+0000 7f35f944c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c4130 0x7f35f41ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.357+0000 7f35f944c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c4130 0x7f35f41ef0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36734/0 (socket says 172.21.15.31:36734) 2024-03-30T01:41:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.357+0000 7f35f944c700 1 -- 172.21.15.31:0/1870520665 learned_addr learned my addr 172.21.15.31:0/1870520665 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.357+0000 7f35fa44e700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f41f2620 0x7f35f4144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.357+0000 7f35f9c4d700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35f40a7440 0x7f35f40adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f15e37fe700 1 -- 172.21.15.31:0/2675197366 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f15f4054560 con 0x7f15fc154cf0 2024-03-30T01:41:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f35fa44e700 1 -- 172.21.15.31:0/1870520665 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35f40a7440 msgr2=0x7f35f40adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f35fa44e700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35f40a7440 0x7f35f40adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f35fa44e700 1 -- 172.21.15.31:0/1870520665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c4130 msgr2=0x7f35f41ef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f35fa44e700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c4130 0x7f35f41ef0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f35fa44e700 1 -- 172.21.15.31:0/1870520665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00094a0 con 0x7f35f41f2620 2024-03-30T01:41:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f7b9b7fe700 1 -- 172.21.15.31:0/3643199745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80036a0 con 0x7f7bb406e0a0 2024-03-30T01:41:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f7b9b7fe700 1 -- 172.21.15.31:0/3643199745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8003800 con 0x7f7bb406e0a0 2024-03-30T01:41:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f7b9b7fe700 1 -- 172.21.15.31:0/3643199745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8003bb0 con 0x7f7bb406e0a0 2024-03-30T01:41:20.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.360+0000 7fab2a7fc700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab2c072a00 0x7fab2c0a6a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00e7e0 tx=0x7fab1c00eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7fab13fff700 1 -- 172.21.15.31:0/1920110096 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00c070 con 0x7fab2c072a00 2024-03-30T01:41:20.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7fab13fff700 1 -- 172.21.15.31:0/1920110096 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c013610 con 0x7fab2c072a00 2024-03-30T01:41:20.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7fab13fff700 1 -- 172.21.15.31:0/1920110096 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0139a0 con 0x7fab2c072a00 2024-03-30T01:41:20.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb40c4c40 con 0x7f7bb406e0a0 2024-03-30T01:41:20.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb40ac080 con 0x7f7bb406e0a0 2024-03-30T01:41:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.360+0000 7f15fad9d700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15e806ad60 0x7f15e806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7f15fad9d700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15e806ad60 0x7f15e806d220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009b90 tx=0x7f15ec013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.358+0000 7f35fa44e700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f41f2620 0x7f35f4144bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00cc80 tx=0x7f35ec00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7f35ea7fc700 1 -- 172.21.15.31:0/1870520665 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec00aac0 con 0x7f35f41f2620 2024-03-30T01:41:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7f35ea7fc700 1 -- 172.21.15.31:0/1870520665 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec00ac20 con 0x7f35f41f2620 2024-03-30T01:41:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7f35ea7fc700 1 -- 172.21.15.31:0/1870520665 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0070d0 con 0x7f35f41f2620 2024-03-30T01:41:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f41f2ad0 con 0x7f35f41f2620 2024-03-30T01:41:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f415d640 con 0x7f35f41f2620 2024-03-30T01:41:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c154ab0 con 0x7fab2c072a00 2024-03-30T01:41:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.361+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c0a6f90 con 0x7fab2c072a00 2024-03-30T01:41:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.362+0000 7f7b9b7fe700 1 -- 172.21.15.31:0/3643199745 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7ba8003d10 con 0x7f7bb406e0a0 2024-03-30T01:41:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.362+0000 7f7b9b7fe700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7ba006ad60 0x7f7ba006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.363+0000 7f7b9b7fe700 1 -- 172.21.15.31:0/3643199745 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7ba8089b90 con 0x7f7bb406e0a0 2024-03-30T01:41:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.364+0000 7f35ea7fc700 1 -- 172.21.15.31:0/1870520665 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f35ec007230 con 0x7f35f41f2620 2024-03-30T01:41:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.364+0000 7f35ea7fc700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35d806ace0 0x7f35d806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.365+0000 7f35ea7fc700 1 -- 172.21.15.31:0/1870520665 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f35ec089450 con 0x7f35f41f2620 2024-03-30T01:41:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.365+0000 7fab13fff700 1 -- 172.21.15.31:0/1920110096 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fab1c013b00 con 0x7fab2c072a00 2024-03-30T01:41:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.366+0000 7fab13fff700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab1806ad60 0x7fab1806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.366+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c0052f0 con 0x7fab2c072a00 2024-03-30T01:41:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.366+0000 7fab13fff700 1 -- 172.21.15.31:0/1920110096 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fab1c089260 con 0x7fab2c072a00 2024-03-30T01:41:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.367+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc0052f0 con 0x7f35f41f2620 2024-03-30T01:41:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.367+0000 7f35f9c4d700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35d806ace0 0x7f35d806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.367+0000 7fab2affd700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab1806ad60 0x7fab1806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.369+0000 7fab2affd700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab1806ad60 0x7fab1806d220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab20009960 tx=0x7fab200090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.375+0000 7fab13fff700 1 -- 172.21.15.31:0/1920110096 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fab1c054140 con 0x7fab2c072a00 2024-03-30T01:41:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.367+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b940052f0 con 0x7f7bb406e0a0 2024-03-30T01:41:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.367+0000 7f7bb2d9d700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7ba006ad60 0x7f7ba006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.369+0000 7f7bb2d9d700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7ba006ad60 0x7f7ba006d220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40096d0 tx=0x7f7ba4006c30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.369+0000 7f35f9c4d700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35d806ace0 0x7f35d806d1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35f40c2060 tx=0x7f35e40090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.377+0000 7f7b9b7fe700 1 -- 172.21.15.31:0/3643199745 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7ba8054b00 con 0x7f7bb406e0a0 2024-03-30T01:41:20.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.382+0000 7f35ea7fc700 1 -- 172.21.15.31:0/1870520665 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f35ec0543f0 con 0x7f35f41f2620 2024-03-30T01:41:20.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.411+0000 7f2edf5d7700 1 Processor -- start 2024-03-30T01:41:20.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.411+0000 7f2edf5d7700 1 -- start start 2024-03-30T01:41:20.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.413+0000 7f2edf5d7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed8150770 0x7f2ed80c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.413+0000 7f2edf5d7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ed806c860 0x7f2ed80c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.413+0000 7f2edf5d7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed8153d60 con 0x7f2ed80ab120 2024-03-30T01:41:20.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.413+0000 7f2edf5d7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed8071e70 con 0x7f2ed806c860 2024-03-30T01:41:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.413+0000 7f2edf5d7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81548e0 con 0x7f2ed8150770 2024-03-30T01:41:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.414+0000 7f2eddb74700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ed806c860 0x7f2ed80c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.414+0000 7f2eddb74700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ed806c860 0x7f2ed80c0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36742/0 (socket says 172.21.15.31:36742) 2024-03-30T01:41:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.414+0000 7f2eddb74700 1 -- 172.21.15.31:0/2833998725 learned_addr learned my addr 172.21.15.31:0/2833998725 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.414+0000 7f2eddb74700 1 -- 172.21.15.31:0/2833998725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed8150770 msgr2=0x7f2ed80c0800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T01:41:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.414+0000 7f2eddb74700 1 --2- 172.21.15.31:0/2833998725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed8150770 0x7f2ed80c0800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.414+0000 7f2eddb74700 1 -- 172.21.15.31:0/2833998725 >> v1:172.21.15.59:6789/0 conn(0x7f2ed80ab120 legacy=0x7f2ed8070bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.414+0000 7f2eddb74700 1 -- 172.21.15.31:0/2833998725 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed806c450 con 0x7f2ed806c860 2024-03-30T01:41:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.415+0000 7f2eddb74700 1 --2- 172.21.15.31:0/2833998725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed806c860 0x7f2ed80c0d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40088f0 tx=0x7f2ed400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2c1a18aa1c71f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.416+0000 7f2ecffff700 1 -- 172.21.15.31:0/2833998725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4010070 con 0x7f2ed806c860 2024-03-30T01:41:20.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.417+0000 7f2ecffff700 1 -- 172.21.15.31:0/2833998725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed400da40 con 0x7f2ed806c860 2024-03-30T01:41:20.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.417+0000 7f2ecffff700 1 -- 172.21.15.31:0/2833998725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400ddd0 con 0x7f2ed806c860 2024-03-30T01:41:20.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.418+0000 7f2edf5d7700 1 -- 172.21.15.31:0/2833998725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed806c860 msgr2=0x7f2ed80c0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.418+0000 7f2edf5d7700 1 --2- 172.21.15.31:0/2833998725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed806c860 0x7f2ed80c0d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40088f0 tx=0x7f2ed400d330 comp rx=0 tx=0).stop 2024-03-30T01:41:20.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.418+0000 7f2edf5d7700 1 -- 172.21.15.31:0/2833998725 shutdown_connections 2024-03-30T01:41:20.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.418+0000 7f2edf5d7700 1 --2- 172.21.15.31:0/2833998725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed806c860 0x7f2ed80c0d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.418+0000 7f2edf5d7700 1 --2- 172.21.15.31:0/2833998725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed8150770 0x7f2ed80c0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.418+0000 7f2edf5d7700 1 -- 172.21.15.31:0/2833998725 >> 172.21.15.31:0/2833998725 conn(0x7f2ed806aaf0 msgr2=0x7f2ed806af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.418+0000 7f2edf5d7700 1 -- 172.21.15.31:0/2833998725 shutdown_connections 2024-03-30T01:41:20.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.418+0000 7f2edf5d7700 1 -- 172.21.15.31:0/2833998725 wait complete. 2024-03-30T01:41:20.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.420+0000 7f2edf5d7700 1 Processor -- start 2024-03-30T01:41:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.421+0000 7f2edf5d7700 1 -- start start 2024-03-30T01:41:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.422+0000 7f2edf5d7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed806c860 0x7f2ed80c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.422+0000 7f2edf5d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80ab120 0x7f2ed81f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.422+0000 7f2edf5d7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed8150770 0x7f2ed81f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.422+0000 7f2edf5d7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81f8df0 con 0x7f2ed80ab120 2024-03-30T01:41:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.422+0000 7f2edf5d7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80bb810 con 0x7f2ed8150770 2024-03-30T01:41:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.422+0000 7f2edf5d7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81f7d50 con 0x7f2ed806c860 2024-03-30T01:41:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.422+0000 7f2edcb72700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80ab120 0x7f2ed81f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.422+0000 7f2edcb72700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80ab120 0x7f2ed81f8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36758/0 (socket says 172.21.15.31:36758) 2024-03-30T01:41:20.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.422+0000 7f2edcb72700 1 -- 172.21.15.31:0/413879648 learned_addr learned my addr 172.21.15.31:0/413879648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.423+0000 7f2edd373700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed806c860 0x7f2ed80c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.423+0000 7f2eddb74700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed8150770 0x7f2ed81f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.423+0000 7f2edcb72700 1 -- 172.21.15.31:0/413879648 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed806c860 msgr2=0x7f2ed80c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.423+0000 7f2edcb72700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed806c860 0x7f2ed80c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.423+0000 7f2edcb72700 1 -- 172.21.15.31:0/413879648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed8150770 msgr2=0x7f2ed81f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.423+0000 7f2edcb72700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed8150770 0x7f2ed81f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.423+0000 7f2edcb72700 1 -- 172.21.15.31:0/413879648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed40085d0 con 0x7f2ed80ab120 2024-03-30T01:41:20.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.423+0000 7f2edcb72700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80ab120 0x7f2ed81f8370 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000ba50 tx=0x7f2ed000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.431+0000 7f2ecdffb700 1 -- 172.21.15.31:0/413879648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000c930 con 0x7f2ed80ab120 2024-03-30T01:41:20.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.431+0000 7f2ecdffb700 1 -- 172.21.15.31:0/413879648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed000ca90 con 0x7f2ed80ab120 2024-03-30T01:41:20.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.431+0000 7f2ecdffb700 1 -- 172.21.15.31:0/413879648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000ce40 con 0x7f2ed80ab120 2024-03-30T01:41:20.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.432+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed81f6f90 con 0x7f2ed80ab120 2024-03-30T01:41:20.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.432+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed81f7550 con 0x7f2ed80ab120 2024-03-30T01:41:20.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.434+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed8072aa0 con 0x7f2ed80ab120 2024-03-30T01:41:20.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.436+0000 7f2ecdffb700 1 -- 172.21.15.31:0/413879648 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2ed0006d50 con 0x7f2ed80ab120 2024-03-30T01:41:20.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.437+0000 7f2ecdffb700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ebc06ad60 0x7f2ebc06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.437+0000 7f2ecdffb700 1 -- 172.21.15.31:0/413879648 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2ed0089520 con 0x7f2ed80ab120 2024-03-30T01:41:20.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.438+0000 7f2edd373700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ebc06ad60 0x7f2ebc06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.439+0000 7f2edd373700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ebc06ad60 0x7f2ebc06d220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8006fd0 tx=0x7f2ec8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.444+0000 7f2ecdffb700 1 -- 172.21.15.31:0/413879648 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2ed00543f0 con 0x7f2ed80ab120 2024-03-30T01:41:20.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.450+0000 7f0f23519700 1 Processor -- start 2024-03-30T01:41:20.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.450+0000 7f0f23519700 1 -- start start 2024-03-30T01:41:20.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.452+0000 7f0f23519700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f1c070b80 0x7f0f1c164a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.452+0000 7f0f23519700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c0705d0 con 0x7f0f1c0c2f90 2024-03-30T01:41:20.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.452+0000 7f0f23519700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c0708b0 con 0x7f0f1c08a970 2024-03-30T01:41:20.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.452+0000 7f0f23519700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c06c700 con 0x7f0f1c070b80 2024-03-30T01:41:20.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.453+0000 7f0f212b5700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0f1c0c2f90 0x7f0f1c06bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34548/0 (socket says 172.21.15.31:34548) 2024-03-30T01:41:20.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.453+0000 7f0f20ab4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f1c070b80 0x7f0f1c164a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.453+0000 7f0f212b5700 1 -- 172.21.15.31:0/1839572938 learned_addr learned my addr 172.21.15.31:0/1839572938 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:20.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.454+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833386916 0 0) 0x7f0f1c0705d0 con 0x7f0f1c0c2f90 2024-03-30T01:41:20.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.455+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f08003a00 con 0x7f0f1c0c2f90 2024-03-30T01:41:20.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.455+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366934081 0 0) 0x7f0f1c0708b0 con 0x7f0f1c08a970 2024-03-30T01:41:20.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.455+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f1c0705d0 con 0x7f0f1c08a970 2024-03-30T01:41:20.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.455+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4137621622 0 0) 0x7f0f08003a00 con 0x7f0f1c0c2f90 2024-03-30T01:41:20.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.455+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f1c070b80 msgr2=0x7f0f1c164a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.455+0000 7f0f13fff700 1 --2- 172.21.15.31:0/1839572938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f1c070b80 0x7f0f1c164a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.455+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 >> v1:172.21.15.59:6789/0 conn(0x7f0f1c08a970 legacy=0x7f0f1c15e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.455+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c06db10 con 0x7f0f1c0c2f90 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.455+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f0f18001670 con 0x7f0f1c0c2f90 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.456+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 >> v1:172.21.15.31:6789/0 conn(0x7f0f1c0c2f90 legacy=0x7f0f1c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.456+0000 7f0f13fff700 1 --2- 172.21.15.31:0/1839572938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f08005070 0x7f0f08005470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.456+0000 7f0f13fff700 1 --2- 172.21.15.31:0/1839572938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f08003c40 0x7f0f080040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.456+0000 7f0f13fff700 1 --2- 172.21.15.31:0/1839572938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f0800be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.456+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c06c700 con 0x7f0f08004600 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.456+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0800c560 con 0x7f0f08005070 2024-03-30T01:41:20.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.456+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0800c720 con 0x7f0f08003c40 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.456+0000 7f0f21ab6700 1 --2- 172.21.15.31:0/1839572938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f0800be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.457+0000 7f0f21ab6700 1 -- 172.21.15.31:0/1839572938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f08003c40 msgr2=0x7f0f080040c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.457+0000 7f0f21ab6700 1 --2- 172.21.15.31:0/1839572938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f08003c40 0x7f0f080040c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.457+0000 7f0f21ab6700 1 -- 172.21.15.31:0/1839572938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f08005070 msgr2=0x7f0f08005470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.457+0000 7f0f21ab6700 1 --2- 172.21.15.31:0/1839572938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f08005070 0x7f0f08005470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.457+0000 7f0f21ab6700 1 -- 172.21.15.31:0/1839572938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c06db10 con 0x7f0f08004600 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.457+0000 7f0f21ab6700 1 --2- 172.21.15.31:0/1839572938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f0800be60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009050 tx=0x7f0f08005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182eb39ee36ae52 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.458+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0052d0 con 0x7f0f08004600 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.458+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c005430 con 0x7f0f08004600 2024-03-30T01:41:20.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.458+0000 7f0f13fff700 1 -- 172.21.15.31:0/1839572938 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0038c0 con 0x7f0f08004600 2024-03-30T01:41:20.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.459+0000 7f0f23519700 1 -- 172.21.15.31:0/1839572938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 msgr2=0x7f0f0800be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.459+0000 7f0f23519700 1 --2- 172.21.15.31:0/1839572938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f0800be60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009050 tx=0x7f0f08005cd0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.459+0000 7f0f21ab6700 1 -- 172.21.15.31:0/1839572938 reap_dead start 2024-03-30T01:41:20.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.459+0000 7f0f23519700 1 -- 172.21.15.31:0/1839572938 shutdown_connections 2024-03-30T01:41:20.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.459+0000 7f0f23519700 1 -- 172.21.15.31:0/1839572938 >> 172.21.15.31:0/1839572938 conn(0x7f0f1c06aaf0 msgr2=0x7f0f1c06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.460+0000 7f0f23519700 1 -- 172.21.15.31:0/1839572938 shutdown_connections 2024-03-30T01:41:20.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.460+0000 7f0f23519700 1 -- 172.21.15.31:0/1839572938 wait complete. 2024-03-30T01:41:20.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.462+0000 7f0f23519700 1 Processor -- start 2024-03-30T01:41:20.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.463+0000 7f0f23519700 1 -- start start 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.464+0000 7f0f23519700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f1c0c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.464+0000 7f0f23519700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f1c154660 0x7f0f1c154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.464+0000 7f0f23519700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c155020 0x7f0f1c20ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.464+0000 7f0f23519700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c156b90 con 0x7f0f08004600 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.464+0000 7f0f23519700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c0ab470 con 0x7f0f1c155020 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.464+0000 7f0f23519700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c0ab1d0 con 0x7f0f1c154660 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.464+0000 7f0f212b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f1c0c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.464+0000 7f0f212b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f1c0c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36780/0 (socket says 172.21.15.31:36780) 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.464+0000 7f0f212b5700 1 -- 172.21.15.31:0/2688516217 learned_addr learned my addr 172.21.15.31:0/2688516217 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.465+0000 7f0f20ab4700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f1c154660 0x7f0f1c154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.465+0000 7f0f21ab6700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c155020 0x7f0f1c20ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.465+0000 7f0f212b5700 1 -- 172.21.15.31:0/2688516217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f1c154660 msgr2=0x7f0f1c154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.465+0000 7f0f212b5700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f1c154660 0x7f0f1c154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.465+0000 7f0f212b5700 1 -- 172.21.15.31:0/2688516217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c155020 msgr2=0x7f0f1c20ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.465+0000 7f0f212b5700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c155020 0x7f0f1c20ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.465+0000 7f0f212b5700 1 -- 172.21.15.31:0/2688516217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c0a9580 con 0x7f0f08004600 2024-03-30T01:41:20.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.465+0000 7f0f212b5700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f1c0c4c70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800caa0 tx=0x7f0f1800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.466+0000 7f0f11ffb700 1 -- 172.21.15.31:0/2688516217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18004a80 con 0x7f0f08004600 2024-03-30T01:41:20.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.467+0000 7f0f11ffb700 1 -- 172.21.15.31:0/2688516217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f18004be0 con 0x7f0f08004600 2024-03-30T01:41:20.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.467+0000 7f0f11ffb700 1 -- 172.21.15.31:0/2688516217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f180070d0 con 0x7f0f08004600 2024-03-30T01:41:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.467+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c0a9860 con 0x7f0f08004600 2024-03-30T01:41:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.467+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c156da0 con 0x7f0f08004600 2024-03-30T01:41:20.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.473+0000 7f0f11ffb700 1 -- 172.21.15.31:0/2688516217 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0f18007230 con 0x7f0f08004600 2024-03-30T01:41:20.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.474+0000 7f0f11ffb700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0f0006ad60 0x7f0f0006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:20.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.474+0000 7f0f11ffb700 1 -- 172.21.15.31:0/2688516217 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0f18088cd0 con 0x7f0f08004600 2024-03-30T01:41:20.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.475+0000 7f0f20ab4700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0f0006ad60 0x7f0f0006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:20.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.476+0000 7f0f20ab4700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0f0006ad60 0x7f0f0006d220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c071550 tx=0x7f0f1400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.477+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc0052f0 con 0x7f0f08004600 2024-03-30T01:41:20.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.487+0000 7f0f11ffb700 1 -- 172.21.15.31:0/2688516217 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0f18018080 con 0x7f0f08004600 2024-03-30T01:41:20.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.750+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5ff00055a0 con 0x7f60080aaef0 2024-03-30T01:41:20.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.751+0000 7f5ffe7fc700 1 -- 172.21.15.31:0/1307672145 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ff8057ac0 con 0x7f60080aaef0 2024-03-30T01:41:20.760 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-03-30T01:41:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.755+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5fec06ad60 msgr2=0x7f5fec06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.755+0000 7f600fbac700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5fec06ad60 0x7f5fec06d220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6004014040 tx=0x7f6004009030 comp rx=0 tx=0).stop 2024-03-30T01:41:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.755+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080aaef0 msgr2=0x7f60080c6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.755+0000 7f600fbac700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080aaef0 0x7f60080c6070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800e930 tx=0x7f5ff800ecf0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.755+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 shutdown_connections 2024-03-30T01:41:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.755+0000 7f600fbac700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60080bfcc0 0x7f6008158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.755+0000 7f600fbac700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5fec06ad60 0x7f5fec06d220 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6004014040 tx=0x7f6004009030 comp rx=0 tx=0).stop 2024-03-30T01:41:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.755+0000 7f600fbac700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005c10 0x7f60080a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.755+0000 7f600fbac700 1 --2- 172.21.15.31:0/1307672145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60080aaef0 0x7f60080c6070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.756+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 >> 172.21.15.31:0/1307672145 conn(0x7f600806aaf0 msgr2=0x7f600815b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.756+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 shutdown_connections 2024-03-30T01:41:20.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.757+0000 7f600fbac700 1 -- 172.21.15.31:0/1307672145 wait complete. 2024-03-30T01:41:20.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.776+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2744149db0 con 0x7f27440c6000 2024-03-30T01:41:20.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.777+0000 7f272bfff700 1 -- 172.21.15.31:0/3092192885 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2738057ad0 con 0x7f27440c6000 2024-03-30T01:41:20.785 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-03-30T01:41:20.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.0 2024-03-30T01:41:20.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.782+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f273006ace0 msgr2=0x7f273006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.782+0000 7f2749b08700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f273006ace0 0x7f273006d1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f273c009fc0 tx=0x7f273c00b040 comp rx=0 tx=0).stop 2024-03-30T01:41:20.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.782+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27440c6000 msgr2=0x7f2744147f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.782+0000 7f2749b08700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27440c6000 0x7f2744147f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f273800d610 tx=0x7f273800d9d0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.783+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 shutdown_connections 2024-03-30T01:41:20.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.783+0000 7f2749b08700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f274414c080 0x7f274414c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.783+0000 7f2749b08700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f273006ace0 0x7f273006d1a0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f273c009fc0 tx=0x7f273c00b040 comp rx=0 tx=0).stop 2024-03-30T01:41:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.783+0000 7f2749b08700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2744152450 0x7f2744147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.783+0000 7f2749b08700 1 --2- 172.21.15.31:0/3092192885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27440c6000 0x7f2744147f50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.783+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 >> 172.21.15.31:0/3092192885 conn(0x7f274406aaf0 msgr2=0x7f274406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.786+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f15dc0055a0 con 0x7f15fc154cf0 2024-03-30T01:41:20.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.788+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 shutdown_connections 2024-03-30T01:41:20.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.789+0000 7f2749b08700 1 -- 172.21.15.31:0/3092192885 wait complete. 2024-03-30T01:41:20.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.797+0000 7f15e37fe700 1 -- 172.21.15.31:0/2675197366 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15f4057b80 con 0x7f15fc154cf0 2024-03-30T01:41:20.810 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2024-03-30T01:41:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.807+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15e806ad60 msgr2=0x7f15e806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.807+0000 7f160102f700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15e806ad60 0x7f15e806d220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009b90 tx=0x7f15ec013040 comp rx=0 tx=0).stop 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc154cf0 msgr2=0x7f15fc151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc154cf0 0x7f15fc151820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f15f4002000 tx=0x7f15f400a340 comp rx=0 tx=0).stop 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 shutdown_connections 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15e806ad60 0x7f15e806d220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15fc154cf0 0x7f15fc151820 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15fc072270 0x7f15fc13da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 --2- 172.21.15.31:0/2675197366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15fc0c46a0 0x7f15fc140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 >> 172.21.15.31:0/2675197366 conn(0x7f15fc0c3a60 msgr2=0x7f15fc06a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 shutdown_connections 2024-03-30T01:41:20.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.808+0000 7f160102f700 1 -- 172.21.15.31:0/2675197366 wait complete. 2024-03-30T01:41:20.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.3 2024-03-30T01:41:20.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.822+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fab0c0055a0 con 0x7fab2c072a00 2024-03-30T01:41:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.822+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7b94005870 con 0x7f7bb406e0a0 2024-03-30T01:41:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.823+0000 7f7b9b7fe700 1 -- 172.21.15.31:0/3643199745 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ba801b030 con 0x7f7bb406e0a0 2024-03-30T01:41:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.823+0000 7fab13fff700 1 -- 172.21.15.31:0/1920110096 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab1c017020 con 0x7fab2c072a00 2024-03-30T01:41:20.830 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-03-30T01:41:20.830 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:20.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7ba006ad60 msgr2=0x7f7ba006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7ba006ad60 0x7f7ba006d220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40096d0 tx=0x7f7ba4006c30 comp rx=0 tx=0).stop 2024-03-30T01:41:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb406e0a0 msgr2=0x7f7bb40c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb406e0a0 0x7f7bb40c4700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8018000 tx=0x7f7ba8004750 comp rx=0 tx=0).stop 2024-03-30T01:41:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 shutdown_connections 2024-03-30T01:41:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7ba006ad60 0x7f7ba006d220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bb4070a10 0x7f7bb41db520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bb406d450 0x7f7bb41fa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 --2- 172.21.15.31:0/3643199745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb406e0a0 0x7f7bb40c4700 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 >> 172.21.15.31:0/3643199745 conn(0x7f7bb40c2d60 msgr2=0x7f7bb41ed290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 shutdown_connections 2024-03-30T01:41:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.827+0000 7f7bb913d700 1 -- 172.21.15.31:0/3643199745 wait complete. 2024-03-30T01:41:20.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.828+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab1806ad60 msgr2=0x7fab1806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.828+0000 7fab31a00700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab1806ad60 0x7fab1806d220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab20009960 tx=0x7fab200090e0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.828+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab2c072a00 msgr2=0x7fab2c0a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.828+0000 7fab31a00700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab2c072a00 0x7fab2c0a6a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00e7e0 tx=0x7fab1c00eaf0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.4 2024-03-30T01:41:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.829+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 shutdown_connections 2024-03-30T01:41:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.829+0000 7fab31a00700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab2c071670 0x7fab2c208100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.829+0000 7fab31a00700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab1806ad60 0x7fab1806d220 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.829+0000 7fab31a00700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab2c14ed90 0x7fab2c0bf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.829+0000 7fab31a00700 1 --2- 172.21.15.31:0/1920110096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab2c072a00 0x7fab2c0a6a50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.829+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 >> 172.21.15.31:0/1920110096 conn(0x7fab2c0a82d0 msgr2=0x7fab2c0c5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.829+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f35dc0055a0 con 0x7f35f41f2620 2024-03-30T01:41:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.829+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 shutdown_connections 2024-03-30T01:41:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.829+0000 7fab31a00700 1 -- 172.21.15.31:0/1920110096 wait complete. 2024-03-30T01:41:20.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.831+0000 7f35ea7fc700 1 -- 172.21.15.31:0/1870520665 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35ec018020 con 0x7f35f41f2620 2024-03-30T01:41:20.840 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-03-30T01:41:20.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35d806ace0 msgr2=0x7f35d806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35d806ace0 0x7f35d806d1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35f40c2060 tx=0x7f35e40090e0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f41f2620 msgr2=0x7f35f4144bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f41f2620 0x7f35f4144bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00cc80 tx=0x7f35ec00cf90 comp rx=0 tx=0).stop 2024-03-30T01:41:20.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 shutdown_connections 2024-03-30T01:41:20.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35d806ace0 0x7f35d806d1a0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35f40c2060 tx=0x7f35e40090e0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35f41f2620 0x7f35f4144bc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35f40a7440 0x7f35f40adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 --2- 172.21.15.31:0/1870520665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40c4130 0x7f35f41ef0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.837+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 >> 172.21.15.31:0/1870520665 conn(0x7f35f40c7e30 msgr2=0x7f35f41518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.840+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 shutdown_connections 2024-03-30T01:41:20.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.841+0000 7f35fbeb1700 1 -- 172.21.15.31:0/1870520665 wait complete. 2024-03-30T01:41:20.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-03-30T01:41:20.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-03-30T01:41:20.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-03-30T01:41:20.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.872+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2ed804ea80 con 0x7f2ed80ab120 2024-03-30T01:41:20.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.873+0000 7f2ecdffb700 1 -- 172.21.15.31:0/413879648 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ed0019020 con 0x7f2ed80ab120 2024-03-30T01:41:20.879 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:20.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.873+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0efc0055a0 con 0x7f0f08004600 2024-03-30T01:41:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.874+0000 7f0f11ffb700 1 -- 172.21.15.31:0/2688516217 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0f180573a0 con 0x7f0f08004600 2024-03-30T01:41:20.880 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.875+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ebc06ad60 msgr2=0x7f2ebc06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.875+0000 7f2edf5d7700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ebc06ad60 0x7f2ebc06d220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8006fd0 tx=0x7f2ec8009190 comp rx=0 tx=0).stop 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.875+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80ab120 msgr2=0x7f2ed81f8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.875+0000 7f2edf5d7700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80ab120 0x7f2ed81f8370 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000ba50 tx=0x7f2ed000bd60 comp rx=0 tx=0).stop 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 shutdown_connections 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f2edf5d7700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ebc06ad60 0x7f2ebc06d220 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f2edf5d7700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed8150770 0x7f2ed81f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f2edf5d7700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed806c860 0x7f2ed80c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f2edf5d7700 1 --2- 172.21.15.31:0/413879648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80ab120 0x7f2ed81f8370 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 >> 172.21.15.31:0/413879648 conn(0x7f2ed806aaf0 msgr2=0x7f2ed80c0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 shutdown_connections 2024-03-30T01:41:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f2edf5d7700 1 -- 172.21.15.31:0/413879648 wait complete. 2024-03-30T01:41:20.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0f0006ad60 msgr2=0x7f0f0006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0f0006ad60 0x7f0f0006d220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c071550 tx=0x7f0f1400b040 comp rx=0 tx=0).stop 2024-03-30T01:41:20.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 msgr2=0x7f0f1c0c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:20.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f1c0c4c70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800caa0 tx=0x7f0f1800ce60 comp rx=0 tx=0).stop 2024-03-30T01:41:20.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 shutdown_connections 2024-03-30T01:41:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f08004600 0x7f0f1c0c4c70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0f0006ad60 0x7f0f0006d220 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c155020 0x7f0f1c20ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 --2- 172.21.15.31:0/2688516217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f1c154660 0x7f0f1c154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 >> 172.21.15.31:0/2688516217 conn(0x7f0f1c06aaf0 msgr2=0x7f0f1c06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.876+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 shutdown_connections 2024-03-30T01:41:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:20.877+0000 7f0f23519700 1 -- 172.21.15.31:0/2688516217 wait complete. 2024-03-30T01:41:20.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-03-30T01:41:20.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-03-30T01:41:21.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-30T01:41:21.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-30T01:41:21.837 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-30T01:41:21.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-30T01:41:21.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T01:41:21.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-30T01:41:21.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-30T01:41:21.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-30T01:41:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.009+0000 7f2f3c3b0700 1 Processor -- start 2024-03-30T01:41:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.009+0000 7f2f3c3b0700 1 -- start start 2024-03-30T01:41:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.011+0000 7f2f3c3b0700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2f340ab120 0x7f2f34070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.011+0000 7f2f3c3b0700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f34153d60 con 0x7f2f34071100 2024-03-30T01:41:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.011+0000 7f2f3c3b0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f341548e0 con 0x7f2f34150770 2024-03-30T01:41:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.011+0000 7f2f3c3b0700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f34071e70 con 0x7f2f340ab120 2024-03-30T01:41:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.012+0000 7f2f3994b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2f34150770 0x7f2f340c0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56502/0 (socket says 172.21.15.31:56502) 2024-03-30T01:41:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.012+0000 7f2f3994b700 1 -- 172.21.15.31:0/1318825443 learned_addr learned my addr 172.21.15.31:0/1318825443 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.014+0000 7f2f3a14c700 1 --2- 172.21.15.31:0/1318825443 >> v2:172.21.15.31:3300/0 conn(0x7f2f340ab120 0x7f2f34070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.014+0000 7f2f3a14c700 1 -- 172.21.15.31:0/1318825443 >> v1:172.21.15.59:6789/0 conn(0x7f2f34150770 legacy=0x7f2f340c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.014+0000 7f2f3a14c700 1 -- 172.21.15.31:0/1318825443 >> v1:172.21.15.31:6789/0 conn(0x7f2f34071100 legacy=0x7f2f3415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.014+0000 7f2f3a14c700 1 -- 172.21.15.31:0/1318825443 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3406c450 con 0x7f2f340ab120 2024-03-30T01:41:22.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.015+0000 7f2f3a14c700 1 --2- 172.21.15.31:0/1318825443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f340ab120 0x7f2f34070bc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00aea0 tx=0x7f2f2c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f0919fe80fb1da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.021+0000 7f2f3914a700 1 -- 172.21.15.31:0/1318825443 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00f070 con 0x7f2f340ab120 2024-03-30T01:41:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.021+0000 7f2f3914a700 1 -- 172.21.15.31:0/1318825443 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c003a20 con 0x7f2f340ab120 2024-03-30T01:41:22.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.021+0000 7f2f3914a700 1 -- 172.21.15.31:0/1318825443 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c003db0 con 0x7f2f340ab120 2024-03-30T01:41:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.022+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/1318825443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f340ab120 msgr2=0x7f2f34070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.022+0000 7f2f3c3b0700 1 --2- 172.21.15.31:0/1318825443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f340ab120 0x7f2f34070bc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00aea0 tx=0x7f2f2c00c330 comp rx=0 tx=0).stop 2024-03-30T01:41:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.023+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/1318825443 shutdown_connections 2024-03-30T01:41:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.023+0000 7f2f3c3b0700 1 --2- 172.21.15.31:0/1318825443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f340ab120 0x7f2f34070bc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.023+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/1318825443 >> 172.21.15.31:0/1318825443 conn(0x7f2f3406aaf0 msgr2=0x7f2f3406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.023+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/1318825443 shutdown_connections 2024-03-30T01:41:22.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.023+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/1318825443 wait complete. 2024-03-30T01:41:22.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.025+0000 7f2f3c3b0700 1 Processor -- start 2024-03-30T01:41:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.025+0000 7f2f3c3b0700 1 -- start start 2024-03-30T01:41:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3c3b0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f34071100 0x7f2f3415db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3c3b0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f340ab120 0x7f2f341483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3c3b0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f34150770 0x7f2f34148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3c3b0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f341494c0 con 0x7f2f34150770 2024-03-30T01:41:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3c3b0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3415e4b0 con 0x7f2f34071100 2024-03-30T01:41:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3c3b0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f34149710 con 0x7f2f340ab120 2024-03-30T01:41:22.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3994b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f340ab120 0x7f2f341483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3994b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f340ab120 0x7f2f341483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60130/0 (socket says 172.21.15.31:60130) 2024-03-30T01:41:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3994b700 1 -- 172.21.15.31:0/3098695629 learned_addr learned my addr 172.21.15.31:0/3098695629 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.027+0000 7f2f3a14c700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f34071100 0x7f2f3415db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.028+0000 7f2f3994b700 1 -- 172.21.15.31:0/3098695629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f34071100 msgr2=0x7f2f3415db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.028+0000 7f2f3994b700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f34071100 0x7f2f3415db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.028+0000 7f2f3994b700 1 -- 172.21.15.31:0/3098695629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f34150770 msgr2=0x7f2f34148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.028+0000 7f2f3994b700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f34150770 0x7f2f34148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.028+0000 7f2f3994b700 1 -- 172.21.15.31:0/3098695629 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c00ab80 con 0x7f2f340ab120 2024-03-30T01:41:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.028+0000 7f2f3994b700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f340ab120 0x7f2f341483d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000c780 tx=0x7f2f3000ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.029+0000 7f2f2affd700 1 -- 172.21.15.31:0/3098695629 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30004db0 con 0x7f2f340ab120 2024-03-30T01:41:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.029+0000 7f2f2affd700 1 -- 172.21.15.31:0/3098695629 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30007150 con 0x7f2f340ab120 2024-03-30T01:41:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.029+0000 7f2f2affd700 1 -- 172.21.15.31:0/3098695629 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300074c0 con 0x7f2f340ab120 2024-03-30T01:41:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.029+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f341eebf0 con 0x7f2f340ab120 2024-03-30T01:41:22.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.029+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3415e080 con 0x7f2f340ab120 2024-03-30T01:41:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.032+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f180052f0 con 0x7f2f340ab120 2024-03-30T01:41:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.032+0000 7f2f2affd700 1 -- 172.21.15.31:0/3098695629 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2f30007620 con 0x7f2f340ab120 2024-03-30T01:41:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.033+0000 7f2f2affd700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f2006aef0 0x7f2f2006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.033+0000 7f2f2affd700 1 -- 172.21.15.31:0/3098695629 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2f30088a50 con 0x7f2f340ab120 2024-03-30T01:41:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.034+0000 7f2f3a14c700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f2006aef0 0x7f2f2006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.035+0000 7f2f3a14c700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f2006aef0 0x7f2f2006d3b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c015040 tx=0x7f2f2c00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.036+0000 7f1d580d5700 1 Processor -- start 2024-03-30T01:41:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.037+0000 7f1d580d5700 1 -- start start 2024-03-30T01:41:22.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.038+0000 7f2f2affd700 1 -- 172.21.15.31:0/3098695629 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2f3004e580 con 0x7f2f340ab120 2024-03-30T01:41:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.038+0000 7f1d580d5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1d50150770 0x7f1d500c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.038+0000 7f1d580d5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1d5006c860 0x7f1d500c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.038+0000 7f1d580d5700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d50153d60 con 0x7f1d500ab120 2024-03-30T01:41:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.038+0000 7f1d580d5700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d50071e70 con 0x7f1d5006c860 2024-03-30T01:41:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.038+0000 7f1d580d5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d501548e0 con 0x7f1d50150770 2024-03-30T01:41:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.039+0000 7f1d56672700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1d5006c860 0x7f1d500c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.039+0000 7f1d56672700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1d5006c860 0x7f1d500c0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36824/0 (socket says 172.21.15.31:36824) 2024-03-30T01:41:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.039+0000 7f1d56672700 1 -- 172.21.15.31:0/3444972336 learned_addr learned my addr 172.21.15.31:0/3444972336 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.039+0000 7f1d56672700 1 -- 172.21.15.31:0/3444972336 >> v2:172.21.15.59:3300/0 conn(0x7f1d50150770 msgr2=0x7f1d500c0800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T01:41:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.039+0000 7f1d56672700 1 --2- 172.21.15.31:0/3444972336 >> v2:172.21.15.59:3300/0 conn(0x7f1d50150770 0x7f1d500c0800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.039+0000 7f1d56672700 1 -- 172.21.15.31:0/3444972336 >> v1:172.21.15.31:6789/0 conn(0x7f1d500ab120 legacy=0x7f1d50070bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.039+0000 7f1d56672700 1 -- 172.21.15.31:0/3444972336 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5006c450 con 0x7f1d5006c860 2024-03-30T01:41:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.040+0000 7f1d56672700 1 --2- 172.21.15.31:0/3444972336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5006c860 0x7f1d500c0d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d500c06d0 tx=0x7f1d4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69c09e6c22f2c57 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.041+0000 7f1d54e6f700 1 -- 172.21.15.31:0/3444972336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4800e070 con 0x7f1d5006c860 2024-03-30T01:41:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.041+0000 7f1d54e6f700 1 -- 172.21.15.31:0/3444972336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d48003a20 con 0x7f1d5006c860 2024-03-30T01:41:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.041+0000 7f1d54e6f700 1 -- 172.21.15.31:0/3444972336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48003dd0 con 0x7f1d5006c860 2024-03-30T01:41:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.042+0000 7f1d580d5700 1 -- 172.21.15.31:0/3444972336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5006c860 msgr2=0x7f1d500c0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.042+0000 7f1d580d5700 1 --2- 172.21.15.31:0/3444972336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5006c860 0x7f1d500c0d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d500c06d0 tx=0x7f1d4800b330 comp rx=0 tx=0).stop 2024-03-30T01:41:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.043+0000 7f1d580d5700 1 -- 172.21.15.31:0/3444972336 shutdown_connections 2024-03-30T01:41:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.043+0000 7f1d580d5700 1 --2- 172.21.15.31:0/3444972336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d5006c860 0x7f1d500c0d40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.043+0000 7f1d580d5700 1 --2- 172.21.15.31:0/3444972336 >> v2:172.21.15.59:3300/0 conn(0x7f1d50150770 0x7f1d500c0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.043+0000 7f1d580d5700 1 -- 172.21.15.31:0/3444972336 >> 172.21.15.31:0/3444972336 conn(0x7f1d5006aaf0 msgr2=0x7f1d5006af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.043+0000 7f1d580d5700 1 -- 172.21.15.31:0/3444972336 shutdown_connections 2024-03-30T01:41:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.043+0000 7f1d580d5700 1 -- 172.21.15.31:0/3444972336 wait complete. 2024-03-30T01:41:22.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.045+0000 7f1d580d5700 1 Processor -- start 2024-03-30T01:41:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.045+0000 7f1d580d5700 1 -- start start 2024-03-30T01:41:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.046+0000 7f1d580d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d5006c860 0x7f1d500c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.046+0000 7f1d580d5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d500ab120 0x7f1d50147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.046+0000 7f1d580d5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d50150770 0x7f1d500a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.046+0000 7f1d580d5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d500c52e0 con 0x7f1d500ab120 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.046+0000 7f1d580d5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d500c3d60 con 0x7f1d5006c860 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.046+0000 7f1d580d5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d500c56f0 con 0x7f1d50150770 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.047+0000 7f1d55670700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d500ab120 0x7f1d50147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.047+0000 7f1d55670700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d500ab120 0x7f1d50147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36826/0 (socket says 172.21.15.31:36826) 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.047+0000 7f1d55670700 1 -- 172.21.15.31:0/3975880627 learned_addr learned my addr 172.21.15.31:0/3975880627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.047+0000 7f1d56672700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d50150770 0x7f1d500a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.047+0000 7f1d55670700 1 -- 172.21.15.31:0/3975880627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d50150770 msgr2=0x7f1d500a89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.047+0000 7f1d55670700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d50150770 0x7f1d500a89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.047+0000 7f1d55670700 1 -- 172.21.15.31:0/3975880627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d5006c860 msgr2=0x7f1d500c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.047+0000 7f1d55670700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d5006c860 0x7f1d500c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.047+0000 7f1d55670700 1 -- 172.21.15.31:0/3975880627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d48009b30 con 0x7f1d500ab120 2024-03-30T01:41:22.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.048+0000 7f1d55670700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d500ab120 0x7f1d50147d00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00b450 tx=0x7f1d4c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.058+0000 7f1d467fc700 1 -- 172.21.15.31:0/3975880627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c011830 con 0x7f1d500ab120 2024-03-30T01:41:22.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.058+0000 7f1d467fc700 1 -- 172.21.15.31:0/3975880627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c011990 con 0x7f1d500ab120 2024-03-30T01:41:22.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.058+0000 7f1d467fc700 1 -- 172.21.15.31:0/3975880627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c011d00 con 0x7f1d500ab120 2024-03-30T01:41:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.058+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d500c59d0 con 0x7f1d500ab120 2024-03-30T01:41:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.058+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d50148240 con 0x7f1d500ab120 2024-03-30T01:41:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.059+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d380052f0 con 0x7f1d500ab120 2024-03-30T01:41:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.063+0000 7f1d467fc700 1 -- 172.21.15.31:0/3975880627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1d4c01a400 con 0x7f1d500ab120 2024-03-30T01:41:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.064+0000 7f1d467fc700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1d3c06af00 0x7f1d3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.064+0000 7f1d467fc700 1 -- 172.21.15.31:0/3975880627 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1d4c089b70 con 0x7f1d500ab120 2024-03-30T01:41:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.065+0000 7f1d55e71700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1d3c06af00 0x7f1d3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.066+0000 7f1d55e71700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1d3c06af00 0x7f1d3c06d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009770 tx=0x7f1d40006cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.070+0000 7f1d467fc700 1 -- 172.21.15.31:0/3975880627 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1d4c054ad0 con 0x7f1d500ab120 2024-03-30T01:41:22.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.093+0000 7f02bd7c3700 1 Processor -- start 2024-03-30T01:41:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.093+0000 7f02bd7c3700 1 -- start start 2024-03-30T01:41:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.094+0000 7f02bd7c3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f02b8154760 0x7f02b8155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.094+0000 7f02bd7c3700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b80c8d10 con 0x7f02b80bd7f0 2024-03-30T01:41:22.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.094+0000 7f02bd7c3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b80c98b0 con 0x7f02b806b170 2024-03-30T01:41:22.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.094+0000 7f02bd7c3700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b806bef0 con 0x7f02b8154760 2024-03-30T01:41:22.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.095+0000 7f02ae7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f02b80bd7f0 0x7f02b815d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34562/0 (socket says 172.21.15.31:34562) 2024-03-30T01:41:22.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.095+0000 7f02ae7fc700 1 -- 172.21.15.31:0/455872022 learned_addr learned my addr 172.21.15.31:0/455872022 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:22.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.098+0000 7f02b77fe700 1 --2- 172.21.15.31:0/455872022 >> v2:172.21.15.31:3300/0 conn(0x7f02b8154760 0x7f02b8155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.098+0000 7f02b67fc700 1 -- 172.21.15.31:0/455872022 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995760415 0 0) 0x7f02b80c8d10 con 0x7f02b80bd7f0 2024-03-30T01:41:22.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.098+0000 7f02b67fc700 1 -- 172.21.15.31:0/455872022 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a40035b0 con 0x7f02b80bd7f0 2024-03-30T01:41:22.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.098+0000 7f02b67fc700 1 -- 172.21.15.31:0/455872022 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175066612 0 0) 0x7f02b80c98b0 con 0x7f02b806b170 2024-03-30T01:41:22.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.098+0000 7f02b67fc700 1 -- 172.21.15.31:0/455872022 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02b80c8d10 con 0x7f02b806b170 2024-03-30T01:41:22.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.099+0000 7f02b77fe700 1 -- 172.21.15.31:0/455872022 >> v1:172.21.15.59:6789/0 conn(0x7f02b806b170 legacy=0x7f02b80c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.099+0000 7f02b77fe700 1 -- 172.21.15.31:0/455872022 >> v1:172.21.15.31:6789/0 conn(0x7f02b80bd7f0 legacy=0x7f02b815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.099+0000 7f02b77fe700 1 -- 172.21.15.31:0/455872022 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b80c8100 con 0x7f02b8154760 2024-03-30T01:41:22.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.099+0000 7f02b77fe700 1 --2- 172.21.15.31:0/455872022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8154760 0x7f02b8155c50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009d90 tx=0x7f02b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea741a6aaf17e4e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.099+0000 7f02b67fc700 1 -- 172.21.15.31:0/455872022 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000e070 con 0x7f02b8154760 2024-03-30T01:41:22.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.100+0000 7f02b67fc700 1 -- 172.21.15.31:0/455872022 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b000bae0 con 0x7f02b8154760 2024-03-30T01:41:22.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.100+0000 7f02b67fc700 1 -- 172.21.15.31:0/455872022 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000be90 con 0x7f02b8154760 2024-03-30T01:41:22.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f2c48961700 1 Processor -- start 2024-03-30T01:41:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.102+0000 7f02bd7c3700 1 -- 172.21.15.31:0/455872022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8154760 msgr2=0x7f02b8155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.102+0000 7f02bd7c3700 1 --2- 172.21.15.31:0/455872022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8154760 0x7f02b8155c50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009d90 tx=0x7f02b000b330 comp rx=0 tx=0).stop 2024-03-30T01:41:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.102+0000 7f02bd7c3700 1 -- 172.21.15.31:0/455872022 shutdown_connections 2024-03-30T01:41:22.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.102+0000 7f02bd7c3700 1 --2- 172.21.15.31:0/455872022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8154760 0x7f02b8155c50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.102+0000 7f02bd7c3700 1 -- 172.21.15.31:0/455872022 >> 172.21.15.31:0/455872022 conn(0x7f02b80ca0d0 msgr2=0x7f02b80ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.102+0000 7f02bd7c3700 1 -- 172.21.15.31:0/455872022 shutdown_connections 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.102+0000 7f02bd7c3700 1 -- 172.21.15.31:0/455872022 wait complete. 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.104+0000 7f02bd7c3700 1 Processor -- start 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.104+0000 7f02bd7c3700 1 -- start start 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.105+0000 7f02bd7c3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02b806b170 0x7f02b80c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.105+0000 7f02bd7c3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b80bd7f0 0x7f02b80c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.105+0000 7f02bd7c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b8154760 0x7f02b8148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.105+0000 7f02bd7c3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b80c1240 con 0x7f02b80bd7f0 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.105+0000 7f02bd7c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b80c1520 con 0x7f02b8154760 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.105+0000 7f02bd7c3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b80c1800 con 0x7f02b806b170 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02ae7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b80bd7f0 0x7f02b80c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02ae7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b80bd7f0 0x7f02b80c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36854/0 (socket says 172.21.15.31:36854) 2024-03-30T01:41:22.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02ae7fc700 1 -- 172.21.15.31:0/2663184866 learned_addr learned my addr 172.21.15.31:0/2663184866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02b6ffd700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02b806b170 0x7f02b80c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02b77fe700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b8154760 0x7f02b8148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02ae7fc700 1 -- 172.21.15.31:0/2663184866 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02b806b170 msgr2=0x7f02b80c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02ae7fc700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02b806b170 0x7f02b80c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02ae7fc700 1 -- 172.21.15.31:0/2663184866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b8154760 msgr2=0x7f02b8148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02ae7fc700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b8154760 0x7f02b8148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.106+0000 7f02ae7fc700 1 -- 172.21.15.31:0/2663184866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0009a70 con 0x7f02b80bd7f0 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.107+0000 7f02ae7fc700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b80bd7f0 0x7f02b80c2d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f02a800c4f0 tx=0x7f02a800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.107+0000 7f02affff700 1 -- 172.21.15.31:0/2663184866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8005b70 con 0x7f02b80bd7f0 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.107+0000 7f02affff700 1 -- 172.21.15.31:0/2663184866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a8005cd0 con 0x7f02b80bd7f0 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.107+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b80c1de0 con 0x7f02b80bd7f0 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.107+0000 7f02affff700 1 -- 172.21.15.31:0/2663184866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a80072d0 con 0x7f02b80bd7f0 2024-03-30T01:41:22.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.107+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b80c4120 con 0x7f02b80bd7f0 2024-03-30T01:41:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.110+0000 7f194eaed700 1 Processor -- start 2024-03-30T01:41:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.110+0000 7f2c48961700 1 -- start start 2024-03-30T01:41:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.111+0000 7f2c48961700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c4406b170 0x7f2c44155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.111+0000 7f2c48961700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c440bd7f0 0x7f2c4406b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.111+0000 7f2c48961700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c440c8d10 con 0x7f2c44154760 2024-03-30T01:41:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.111+0000 7f2c48961700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4406bef0 con 0x7f2c440bd7f0 2024-03-30T01:41:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.111+0000 7f2c48961700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c440c98b0 con 0x7f2c4406b170 2024-03-30T01:41:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.112+0000 7f2c42d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c44154760 0x7f2c440c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34566/0 (socket says 172.21.15.31:34566) 2024-03-30T01:41:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.112+0000 7f2c42d9d700 1 -- 172.21.15.31:0/3759795919 learned_addr learned my addr 172.21.15.31:0/3759795919 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.111+0000 7f02affff700 1 -- 172.21.15.31:0/2663184866 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f02a8007430 con 0x7f02b80bd7f0 2024-03-30T01:41:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.112+0000 7f02affff700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f029806ac90 0x7f029806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.112+0000 7f02affff700 1 -- 172.21.15.31:0/2663184866 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f02a80cc780 con 0x7f02b80bd7f0 2024-03-30T01:41:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.113+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167029152 0 0) 0x7f2c440c8d10 con 0x7f2c44154760 2024-03-30T01:41:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.113+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c2c0035b0 con 0x7f2c44154760 2024-03-30T01:41:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.113+0000 7f02b6ffd700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f029806ac90 0x7f029806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.111+0000 7f194eaed700 1 -- start start 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.113+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 195247629 0 0) 0x7f2c2c0035b0 con 0x7f2c44154760 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.113+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 >> v2:172.21.15.59:3300/0 conn(0x7f2c4406b170 msgr2=0x7f2c44155c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.113+0000 7f2c4159a700 1 --2- 172.21.15.31:0/3759795919 >> v2:172.21.15.59:3300/0 conn(0x7f2c4406b170 0x7f2c44155c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 >> v2:172.21.15.31:3300/0 conn(0x7f2c440bd7f0 msgr2=0x7f2c4406b550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 --2- 172.21.15.31:0/3759795919 >> v2:172.21.15.31:3300/0 conn(0x7f2c440bd7f0 0x7f2c4406b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c440c8100 con 0x7f2c44154760 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f2c3c0019f0 con 0x7f2c44154760 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 >> v1:172.21.15.31:6789/0 conn(0x7f2c44154760 legacy=0x7f2c440c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 --2- 172.21.15.31:0/3759795919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c2c003a00 0x7f2c2c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 --2- 172.21.15.31:0/3759795919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2c004850 0x7f2c2c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 --2- 172.21.15.31:0/3759795919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c2c005460 0x7f2c2c00ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c440c98b0 con 0x7f2c2c005460 2024-03-30T01:41:22.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c00c370 con 0x7f2c2c004850 2024-03-30T01:41:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c00c530 con 0x7f2c2c003a00 2024-03-30T01:41:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c42d9d700 1 --2- 172.21.15.31:0/3759795919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c2c005460 0x7f2c2c00ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f194eaed700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1948150770 0x7f19480ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f194eaed700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f194806c860 0x7f19480c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f194eaed700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1948153d60 con 0x7f19480c2900 2024-03-30T01:41:22.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f194eaed700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1948071e70 con 0x7f194806c860 2024-03-30T01:41:22.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f194eaed700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19481548e0 con 0x7f1948150770 2024-03-30T01:41:22.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f02b6ffd700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f029806ac90 0x7f029806d150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f02b8156160 tx=0x7f02a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c42d9d700 1 -- 172.21.15.31:0/3759795919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c2c003a00 msgr2=0x7f2c2c003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:22.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c42d9d700 1 --2- 172.21.15.31:0/3759795919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c2c003a00 0x7f2c2c003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c42d9d700 1 -- 172.21.15.31:0/3759795919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2c004850 msgr2=0x7f2c2c004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:22.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.114+0000 7f2c42d9d700 1 --2- 172.21.15.31:0/3759795919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2c004850 0x7f2c2c004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f2c42d9d700 1 -- 172.21.15.31:0/3759795919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c440c8100 con 0x7f2c2c005460 2024-03-30T01:41:22.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f2c42d9d700 1 -- 172.21.15.31:0/3759795919 reap_dead start 2024-03-30T01:41:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f194d08a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f194806c860 0x7f19480c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f194d08a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f194806c860 0x7f19480c0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36866/0 (socket says 172.21.15.31:36866) 2024-03-30T01:41:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f194d08a700 1 -- 172.21.15.31:0/726167452 learned_addr learned my addr 172.21.15.31:0/726167452 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f2c42d9d700 1 --2- 172.21.15.31:0/3759795919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c2c005460 0x7f2c2c00ba90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00a990 tx=0x7f2c2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5f65077a46cade server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f194d08a700 1 -- 172.21.15.31:0/726167452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1948150770 msgr2=0x7f19480ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:22.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f194d08a700 1 --2- 172.21.15.31:0/726167452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1948150770 0x7f19480ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f194d08a700 1 -- 172.21.15.31:0/726167452 >> v1:172.21.15.59:6789/0 conn(0x7f19480c2900 legacy=0x7f1948070bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:22.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f194d08a700 1 -- 172.21.15.31:0/726167452 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194806c450 con 0x7f194806c860 2024-03-30T01:41:22.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c012600 con 0x7f2c2c005460 2024-03-30T01:41:22.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c0127a0 con 0x7f2c2c005460 2024-03-30T01:41:22.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f2c4159a700 1 -- 172.21.15.31:0/3759795919 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c012b50 con 0x7f2c2c005460 2024-03-30T01:41:22.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.115+0000 7f194d08a700 1 --2- 172.21.15.31:0/726167452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194806c860 0x7f19480c0800 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f19300098f0 tx=0x7f1930009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9581ff2f3d2b7722 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.116+0000 7f19477fe700 1 -- 172.21.15.31:0/726167452 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930004230 con 0x7f194806c860 2024-03-30T01:41:22.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.116+0000 7f19477fe700 1 -- 172.21.15.31:0/726167452 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1930004390 con 0x7f194806c860 2024-03-30T01:41:22.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.116+0000 7f19477fe700 1 -- 172.21.15.31:0/726167452 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19300036a0 con 0x7f194806c860 2024-03-30T01:41:22.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.116+0000 7f2c48961700 1 -- 172.21.15.31:0/3759795919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c2c005460 msgr2=0x7f2c2c00ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.116+0000 7f2c48961700 1 --2- 172.21.15.31:0/3759795919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c2c005460 0x7f2c2c00ba90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00a990 tx=0x7f2c2c005880 comp rx=0 tx=0).stop 2024-03-30T01:41:22.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f2c48961700 1 -- 172.21.15.31:0/3759795919 shutdown_connections 2024-03-30T01:41:22.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f2c48961700 1 --2- 172.21.15.31:0/3759795919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c2c005460 0x7f2c2c00ba90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f2c48961700 1 -- 172.21.15.31:0/3759795919 >> 172.21.15.31:0/3759795919 conn(0x7f2c440ca0d0 msgr2=0x7f2c440ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f194eaed700 1 -- 172.21.15.31:0/726167452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194806c860 msgr2=0x7f19480c0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f194eaed700 1 --2- 172.21.15.31:0/726167452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194806c860 0x7f19480c0800 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f19300098f0 tx=0x7f1930009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:22.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f194eaed700 1 -- 172.21.15.31:0/726167452 shutdown_connections 2024-03-30T01:41:22.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f194eaed700 1 --2- 172.21.15.31:0/726167452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194806c860 0x7f19480c0800 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f194eaed700 1 --2- 172.21.15.31:0/726167452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1948150770 0x7f19480ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f194eaed700 1 -- 172.21.15.31:0/726167452 >> 172.21.15.31:0/726167452 conn(0x7f194806aaf0 msgr2=0x7f194806af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f194eaed700 1 -- 172.21.15.31:0/726167452 shutdown_connections 2024-03-30T01:41:22.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.117+0000 7f194eaed700 1 -- 172.21.15.31:0/726167452 wait complete. 2024-03-30T01:41:22.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.118+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029c0052f0 con 0x7f02b80bd7f0 2024-03-30T01:41:22.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.119+0000 7f2c48961700 1 -- 172.21.15.31:0/3759795919 shutdown_connections 2024-03-30T01:41:22.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.119+0000 7f194eaed700 1 Processor -- start 2024-03-30T01:41:22.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.119+0000 7f194eaed700 1 -- start start 2024-03-30T01:41:22.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f194eaed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194806c860 0x7f19480c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f194eaed700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1948150770 0x7f1948151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f194eaed700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f194813dfd0 0x7f19481522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f194eaed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19480c4600 con 0x7f194806c860 2024-03-30T01:41:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f194eaed700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194813e470 con 0x7f194813dfd0 2024-03-30T01:41:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f194eaed700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19480c39f0 con 0x7f1948150770 2024-03-30T01:41:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f194d08a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f194813dfd0 0x7f19481522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f194d08a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f194813dfd0 0x7f19481522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38974/0 (socket says 172.21.15.31:38974) 2024-03-30T01:41:22.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f194d08a700 1 -- 172.21.15.31:0/53248322 learned_addr learned my addr 172.21.15.31:0/53248322 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.119+0000 7f2c48961700 1 -- 172.21.15.31:0/3759795919 wait complete. 2024-03-30T01:41:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.121+0000 7f2c48961700 1 Processor -- start 2024-03-30T01:41:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.122+0000 7f2c48961700 1 -- start start 2024-03-30T01:41:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.123+0000 7f2c48961700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c2c005460 0x7f2c440bf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.123+0000 7f2c48961700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c441efbf0 0x7f2c440adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.123+0000 7f2c48961700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c440c0a80 0x7f2c441eee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.123+0000 7f2c48961700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c441ee780 con 0x7f2c441efbf0 2024-03-30T01:41:22.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.123+0000 7f2c48961700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c440bfbb0 con 0x7f2c440c0a80 2024-03-30T01:41:22.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.123+0000 7f2c48961700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c440be8d0 con 0x7f2c2c005460 2024-03-30T01:41:22.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.122+0000 7f194d08a700 1 -- 172.21.15.31:0/53248322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1948150770 msgr2=0x7f1948151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.122+0000 7f194d08a700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1948150770 0x7f1948151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.122+0000 7f194d08a700 1 -- 172.21.15.31:0/53248322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194806c860 msgr2=0x7f19480c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.122+0000 7f194d08a700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194806c860 0x7f19480c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.122+0000 7f194d08a700 1 -- 172.21.15.31:0/53248322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1930009550 con 0x7f194813dfd0 2024-03-30T01:41:22.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.122+0000 7f194d08a700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f194813dfd0 0x7f19481522f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1930009130 tx=0x7f1930004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.123+0000 7f2c42d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c440c0a80 0x7f2c441eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.123+0000 7f2c42d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c440c0a80 0x7f2c441eee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38986/0 (socket says 172.21.15.31:38986) 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.123+0000 7f2c42d9d700 1 -- 172.21.15.31:0/493779762 learned_addr learned my addr 172.21.15.31:0/493779762 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.124+0000 7f2c4259c700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c2c005460 0x7f2c440bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.124+0000 7f2c41d9b700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c441efbf0 0x7f2c440adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.124+0000 7f2c41d9b700 1 -- 172.21.15.31:0/493779762 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c2c005460 msgr2=0x7f2c440bf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.124+0000 7f2c41d9b700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c2c005460 0x7f2c440bf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.124+0000 7f2c41d9b700 1 -- 172.21.15.31:0/493779762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c440c0a80 msgr2=0x7f2c441eee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.124+0000 7f2c41d9b700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c440c0a80 0x7f2c441eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.124+0000 7f2c41d9b700 1 -- 172.21.15.31:0/493779762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c440bdca0 con 0x7f2c441efbf0 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.125+0000 7f2c41d9b700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c441efbf0 0x7f2c440adfd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c4406c1d0 tx=0x7f2c3800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.125+0000 7f2c32ffd700 1 -- 172.21.15.31:0/493779762 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38011620 con 0x7f2c441efbf0 2024-03-30T01:41:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.125+0000 7f2c32ffd700 1 -- 172.21.15.31:0/493779762 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c380117a0 con 0x7f2c441efbf0 2024-03-30T01:41:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.125+0000 7f2c32ffd700 1 -- 172.21.15.31:0/493779762 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38011b50 con 0x7f2c441efbf0 2024-03-30T01:41:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.125+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c440bdf80 con 0x7f2c441efbf0 2024-03-30T01:41:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.125+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c440be5d0 con 0x7f2c441efbf0 2024-03-30T01:41:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.125+0000 7f02affff700 1 -- 172.21.15.31:0/2663184866 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f02a8097710 con 0x7f02b80bd7f0 2024-03-30T01:41:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.127+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4404ea80 con 0x7f2c441efbf0 2024-03-30T01:41:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.128+0000 7f2c32ffd700 1 -- 172.21.15.31:0/493779762 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2c38011cb0 con 0x7f2c441efbf0 2024-03-30T01:41:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.129+0000 7f2c32ffd700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c2406ad60 0x7f2c2406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.129+0000 7f2c32ffd700 1 -- 172.21.15.31:0/493779762 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c380892b0 con 0x7f2c441efbf0 2024-03-30T01:41:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.134+0000 7f2c32ffd700 1 -- 172.21.15.31:0/493779762 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2c38054210 con 0x7f2c441efbf0 2024-03-30T01:41:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.135+0000 7f2c4259c700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c2406ad60 0x7f2c2406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.136+0000 7f2c4259c700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c2406ad60 0x7f2c2406d220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c34006fd0 tx=0x7f2c34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.134+0000 7f19457fa700 1 -- 172.21.15.31:0/53248322 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193000b660 con 0x7f194813dfd0 2024-03-30T01:41:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.134+0000 7f19457fa700 1 -- 172.21.15.31:0/53248322 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193000b7c0 con 0x7f194813dfd0 2024-03-30T01:41:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.134+0000 7f19457fa700 1 -- 172.21.15.31:0/53248322 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193000bb70 con 0x7f194813dfd0 2024-03-30T01:41:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.134+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19480c3cd0 con 0x7f194813dfd0 2024-03-30T01:41:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.135+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19480c4290 con 0x7f194813dfd0 2024-03-30T01:41:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.138+0000 7f19457fa700 1 -- 172.21.15.31:0/53248322 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1930021070 con 0x7f194813dfd0 2024-03-30T01:41:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.138+0000 7f19457fa700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806ad60 0x7f192806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.138+0000 7f19457fa700 1 -- 172.21.15.31:0/53248322 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f193008afa0 con 0x7f194813dfd0 2024-03-30T01:41:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.143+0000 7fb1936e9700 1 Processor -- start 2024-03-30T01:41:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.143+0000 7fb1936e9700 1 -- start start 2024-03-30T01:41:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.144+0000 7fb1936e9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb18c14e6d0 0x7fb18c0c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.144+0000 7fb1936e9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb18c0c06d0 0x7fb18c1551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.144+0000 7fb1936e9700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c06cd20 con 0x7fb18c06d770 2024-03-30T01:41:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.144+0000 7fb1936e9700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c06d2e0 con 0x7fb18c0c06d0 2024-03-30T01:41:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.144+0000 7fb1936e9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c155730 con 0x7fb18c14e6d0 2024-03-30T01:41:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.144+0000 7fb191c86700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb18c06d770 0x7fb18c0bf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34576/0 (socket says 172.21.15.31:34576) 2024-03-30T01:41:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.144+0000 7fb191c86700 1 -- 172.21.15.31:0/3457265004 learned_addr learned my addr 172.21.15.31:0/3457265004 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:22.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.144+0000 7fb191485700 1 --2- 172.21.15.31:0/3457265004 >> v2:172.21.15.59:3300/0 conn(0x7fb18c14e6d0 0x7fb18c0c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.145+0000 7fb183fff700 1 -- 172.21.15.31:0/3457265004 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973166418 0 0) 0x7fb18c06cd20 con 0x7fb18c06d770 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.145+0000 7fb183fff700 1 -- 172.21.15.31:0/3457265004 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1780035b0 con 0x7fb18c06d770 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.145+0000 7fb191485700 1 -- 172.21.15.31:0/3457265004 >> v2:172.21.15.31:3300/0 conn(0x7fb18c0c06d0 msgr2=0x7fb18c1551f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.145+0000 7fb191485700 1 --2- 172.21.15.31:0/3457265004 >> v2:172.21.15.31:3300/0 conn(0x7fb18c0c06d0 0x7fb18c1551f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.145+0000 7fb191485700 1 -- 172.21.15.31:0/3457265004 >> v1:172.21.15.31:6789/0 conn(0x7fb18c06d770 legacy=0x7fb18c0bf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.145+0000 7fb191485700 1 -- 172.21.15.31:0/3457265004 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0c2720 con 0x7fb18c14e6d0 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.145+0000 7fb191485700 1 --2- 172.21.15.31:0/3457265004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c14e6d0 0x7fb18c0c0c60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb18c070180 tx=0x7fb17c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a3b5c67f58cb47 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.146+0000 7fb183fff700 1 -- 172.21.15.31:0/3457265004 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c011830 con 0x7fb18c14e6d0 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.146+0000 7fb183fff700 1 -- 172.21.15.31:0/3457265004 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c011990 con 0x7fb18c14e6d0 2024-03-30T01:41:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.146+0000 7fb183fff700 1 -- 172.21.15.31:0/3457265004 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c011d00 con 0x7fb18c14e6d0 2024-03-30T01:41:22.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.146+0000 7fb1936e9700 1 -- 172.21.15.31:0/3457265004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c14e6d0 msgr2=0x7fb18c0c0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.146+0000 7fb1936e9700 1 --2- 172.21.15.31:0/3457265004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c14e6d0 0x7fb18c0c0c60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb18c070180 tx=0x7fb17c009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:22.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.147+0000 7fb1936e9700 1 -- 172.21.15.31:0/3457265004 shutdown_connections 2024-03-30T01:41:22.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.147+0000 7fb1936e9700 1 --2- 172.21.15.31:0/3457265004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c14e6d0 0x7fb18c0c0c60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.147+0000 7fb1936e9700 1 --2- 172.21.15.31:0/3457265004 >> v2:172.21.15.31:3300/0 conn(0x7fb18c0c06d0 0x7fb18c1551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.147+0000 7fb1936e9700 1 -- 172.21.15.31:0/3457265004 >> 172.21.15.31:0/3457265004 conn(0x7fb18c0c2d60 msgr2=0x7fb18c0c3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.147+0000 7f194c889700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806ad60 0x7f192806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.147+0000 7fb1936e9700 1 -- 172.21.15.31:0/3457265004 shutdown_connections 2024-03-30T01:41:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.147+0000 7fb1936e9700 1 -- 172.21.15.31:0/3457265004 wait complete. 2024-03-30T01:41:22.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.148+0000 7f194c889700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806ad60 0x7f192806d220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f194813d180 tx=0x7f1940006cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.149+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192c0052f0 con 0x7f194813dfd0 2024-03-30T01:41:22.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.149+0000 7fb1936e9700 1 Processor -- start 2024-03-30T01:41:22.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.149+0000 7fb1936e9700 1 -- start start 2024-03-30T01:41:22.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.150+0000 7fb1936e9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb18c06d770 0x7fb18c0a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.150+0000 7fb1936e9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c14e6d0 0x7fb18c0ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.150+0000 7fb1936e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c151e00 0x7fb18c150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.150+0000 7fb1936e9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0aa0c0 con 0x7fb18c14e6d0 2024-03-30T01:41:22.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.150+0000 7fb1936e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0aa360 con 0x7fb18c151e00 2024-03-30T01:41:22.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.150+0000 7fb1936e9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0aa640 con 0x7fb18c06d770 2024-03-30T01:41:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.152+0000 7fb191485700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb18c06d770 0x7fb18c0a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.152+0000 7fb191c86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c151e00 0x7fb18c150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.152+0000 7fb191485700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb18c06d770 0x7fb18c0a9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60184/0 (socket says 172.21.15.31:60184) 2024-03-30T01:41:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb191485700 1 -- 172.21.15.31:0/4186249436 learned_addr learned my addr 172.21.15.31:0/4186249436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb191c86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c151e00 0x7fb18c150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:38994/0 (socket says 172.21.15.31:38994) 2024-03-30T01:41:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb190c84700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c14e6d0 0x7fb18c0ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb191485700 1 -- 172.21.15.31:0/4186249436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c151e00 msgr2=0x7fb18c150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb191485700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c151e00 0x7fb18c150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb191485700 1 -- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c14e6d0 msgr2=0x7fb18c0ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb191485700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c14e6d0 0x7fb18c0ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb191485700 1 -- 172.21.15.31:0/4186249436 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0094a0 con 0x7fb18c06d770 2024-03-30T01:41:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb191485700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb18c06d770 0x7fb18c0a9b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0090b0 tx=0x7fb17c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.153+0000 7fb181ffb700 1 -- 172.21.15.31:0/4186249436 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0036a0 con 0x7fb18c06d770 2024-03-30T01:41:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.154+0000 7fb181ffb700 1 -- 172.21.15.31:0/4186249436 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c01f070 con 0x7fb18c06d770 2024-03-30T01:41:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.154+0000 7fb181ffb700 1 -- 172.21.15.31:0/4186249436 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c003a00 con 0x7fb18c06d770 2024-03-30T01:41:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.154+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c0aa920 con 0x7fb18c06d770 2024-03-30T01:41:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.154+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c0aac00 con 0x7fb18c06d770 2024-03-30T01:41:22.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.155+0000 7f19457fa700 1 -- 172.21.15.31:0/53248322 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1930055f00 con 0x7f194813dfd0 2024-03-30T01:41:22.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.169+0000 7fb181ffb700 1 -- 172.21.15.31:0/4186249436 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb17c003b60 con 0x7fb18c06d770 2024-03-30T01:41:22.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.169+0000 7fb181ffb700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb17006ad60 0x7fb17006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.169+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c1561e0 con 0x7fb18c06d770 2024-03-30T01:41:22.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.169+0000 7fb181ffb700 1 -- 172.21.15.31:0/4186249436 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb17c021070 con 0x7fb18c06d770 2024-03-30T01:41:22.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.170+0000 7fb190c84700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb17006ad60 0x7fb17006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.171+0000 7f45a95bf700 1 Processor -- start 2024-03-30T01:41:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.171+0000 7fb190c84700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb17006ad60 0x7fb17006d220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb18c153930 tx=0x7fb188008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.171+0000 7f45a95bf700 1 -- start start 2024-03-30T01:41:22.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.171+0000 7f7ea5e48700 1 Processor -- start 2024-03-30T01:41:22.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.171+0000 7f7ea5e48700 1 -- start start 2024-03-30T01:41:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.172+0000 7f45a95bf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a40ab120 0x7f45a4070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.172+0000 7f45a95bf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f45a4150770 0x7f45a40c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.172+0000 7f45a95bf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a4153d60 con 0x7f45a4071100 2024-03-30T01:41:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.172+0000 7f45a95bf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a4071e70 con 0x7f45a4150770 2024-03-30T01:41:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.172+0000 7f45a95bf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c1450 con 0x7f45a40ab120 2024-03-30T01:41:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f45a2ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a40ab120 0x7f45a4070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f45a2ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a40ab120 0x7f45a4070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60192/0 (socket says 172.21.15.31:60192) 2024-03-30T01:41:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f45a2ffd700 1 -- 172.21.15.31:0/3159083594 learned_addr learned my addr 172.21.15.31:0/3159083594 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7ea5e48700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7ea00706c0 0x7f7ea0155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7ea5e48700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ea006db70 0x7f7ea01561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7ea5e48700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea00c0ff0 con 0x7f7ea00c1480 2024-03-30T01:41:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7ea5e48700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea006cfc0 con 0x7f7ea006db70 2024-03-30T01:41:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7ea5e48700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea006d2a0 con 0x7f7ea00706c0 2024-03-30T01:41:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f45a2ffd700 1 -- 172.21.15.31:0/3159083594 >> v2:172.21.15.59:3300/0 conn(0x7f45a4150770 msgr2=0x7f45a40c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f45a2ffd700 1 --2- 172.21.15.31:0/3159083594 >> v2:172.21.15.59:3300/0 conn(0x7f45a4150770 0x7f45a40c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f45a2ffd700 1 -- 172.21.15.31:0/3159083594 >> v1:172.21.15.59:6789/0 conn(0x7f45a4071100 legacy=0x7f45a40c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f45a2ffd700 1 -- 172.21.15.31:0/3159083594 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a41548e0 con 0x7f45a40ab120 2024-03-30T01:41:22.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7e9effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ea006db70 0x7f7ea01561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7e9effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ea006db70 0x7f7ea01561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36890/0 (socket says 172.21.15.31:36890) 2024-03-30T01:41:22.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7e9effd700 1 -- 172.21.15.31:0/347404077 learned_addr learned my addr 172.21.15.31:0/347404077 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7e9f7fe700 1 --2- 172.21.15.31:0/347404077 >> v2:172.21.15.59:3300/0 conn(0x7f7ea00706c0 0x7f7ea0155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.173+0000 7f7e9ffff700 1 --1- 172.21.15.31:0/347404077 >> v1:172.21.15.59:6789/0 conn(0x7f7ea00c1480 0x7f7ea00c18b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56518/0 (socket says 172.21.15.31:56518) 2024-03-30T01:41:22.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.174+0000 7f45a2ffd700 1 --2- 172.21.15.31:0/3159083594 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a40ab120 0x7f45a4070bc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f459c00d150 tx=0x7f459c00d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b390fcc79945ed8c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.174+0000 7f7e9f7fe700 1 -- 172.21.15.31:0/347404077 >> v2:172.21.15.31:3300/0 conn(0x7f7ea006db70 msgr2=0x7f7ea01561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.174+0000 7f7e9f7fe700 1 --2- 172.21.15.31:0/347404077 >> v2:172.21.15.31:3300/0 conn(0x7f7ea006db70 0x7f7ea01561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.174+0000 7f7e9f7fe700 1 -- 172.21.15.31:0/347404077 >> v1:172.21.15.59:6789/0 conn(0x7f7ea00c1480 legacy=0x7f7ea00c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.174+0000 7f7e9f7fe700 1 -- 172.21.15.31:0/347404077 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea00c09a0 con 0x7f7ea00706c0 2024-03-30T01:41:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.174+0000 7f7e9f7fe700 1 --2- 172.21.15.31:0/347404077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00706c0 0x7f7ea0155ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00c22e0 tx=0x7f7e90009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24f5407a2ee3ba79 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.174+0000 7f7e9e7fc700 1 -- 172.21.15.31:0/347404077 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90011830 con 0x7f7ea00706c0 2024-03-30T01:41:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.174+0000 7f7e9e7fc700 1 -- 172.21.15.31:0/347404077 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e90011990 con 0x7f7ea00706c0 2024-03-30T01:41:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.174+0000 7f7e9e7fc700 1 -- 172.21.15.31:0/347404077 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90011d20 con 0x7f7ea00706c0 2024-03-30T01:41:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.175+0000 7f7ea5e48700 1 -- 172.21.15.31:0/347404077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00706c0 msgr2=0x7f7ea0155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.175+0000 7f7ea5e48700 1 --2- 172.21.15.31:0/347404077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00706c0 0x7f7ea0155ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00c22e0 tx=0x7f7e90009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:22.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.175+0000 7f45a1ffb700 1 -- 172.21.15.31:0/3159083594 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c010070 con 0x7f45a40ab120 2024-03-30T01:41:22.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.175+0000 7f45a1ffb700 1 -- 172.21.15.31:0/3159083594 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c00db70 con 0x7f45a40ab120 2024-03-30T01:41:22.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.175+0000 7f45a1ffb700 1 -- 172.21.15.31:0/3159083594 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c003a20 con 0x7f45a40ab120 2024-03-30T01:41:22.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.176+0000 7f45a95bf700 1 -- 172.21.15.31:0/3159083594 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a40ab120 msgr2=0x7f45a4070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.176+0000 7f45a95bf700 1 --2- 172.21.15.31:0/3159083594 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a40ab120 0x7f45a4070bc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f459c00d150 tx=0x7f459c00d460 comp rx=0 tx=0).stop 2024-03-30T01:41:22.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.176+0000 7f7ea5e48700 1 -- 172.21.15.31:0/347404077 shutdown_connections 2024-03-30T01:41:22.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.176+0000 7f7ea5e48700 1 --2- 172.21.15.31:0/347404077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00706c0 0x7f7ea0155ca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.176+0000 7f7ea5e48700 1 --2- 172.21.15.31:0/347404077 >> v2:172.21.15.31:3300/0 conn(0x7f7ea006db70 0x7f7ea01561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.176+0000 7f7ea5e48700 1 -- 172.21.15.31:0/347404077 >> 172.21.15.31:0/347404077 conn(0x7f7ea006aaf0 msgr2=0x7f7ea006af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.176+0000 7f7ea5e48700 1 -- 172.21.15.31:0/347404077 shutdown_connections 2024-03-30T01:41:22.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.176+0000 7f7ea5e48700 1 -- 172.21.15.31:0/347404077 wait complete. 2024-03-30T01:41:22.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.177+0000 7fb181ffb700 1 -- 172.21.15.31:0/4186249436 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb18c1561e0 con 0x7fb18c06d770 2024-03-30T01:41:22.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.178+0000 7f7ea5e48700 1 Processor -- start 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.178+0000 7f7ea5e48700 1 -- start start 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.179+0000 7f7ea5e48700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea006db70 0x7f7ea00aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.179+0000 7f7ea5e48700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ea00706c0 0x7f7ea00bce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.179+0000 7f7ea5e48700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00c1480 0x7f7ea00a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.179+0000 7f7ea5e48700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01f1dd0 con 0x7f7ea006db70 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.179+0000 7f7ea5e48700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01f1220 con 0x7f7ea00c1480 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.179+0000 7f7ea5e48700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01f1430 con 0x7f7ea00706c0 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00c1480 0x7f7ea00a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea006db70 0x7f7ea00aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea006db70 0x7f7ea00aa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36900/0 (socket says 172.21.15.31:36900) 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00c1480 0x7f7ea00a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:39012/0 (socket says 172.21.15.31:39012) 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9f7fe700 1 -- 172.21.15.31:0/1232890508 learned_addr learned my addr 172.21.15.31:0/1232890508 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9ffff700 1 -- 172.21.15.31:0/1232890508 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ea00706c0 msgr2=0x7f7ea00bce40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T01:41:22.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9ffff700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ea00706c0 0x7f7ea00bce40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9ffff700 1 -- 172.21.15.31:0/1232890508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea006db70 msgr2=0x7f7ea00aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9ffff700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea006db70 0x7f7ea00aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9ffff700 1 -- 172.21.15.31:0/1232890508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900094a0 con 0x7f7ea00c1480 2024-03-30T01:41:22.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.180+0000 7f7e9ffff700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00c1480 0x7f7ea00a8460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800c9f0 tx=0x7f7e9800cd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.181+0000 7f45a95bf700 1 -- 172.21.15.31:0/3159083594 shutdown_connections 2024-03-30T01:41:22.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.181+0000 7f45a95bf700 1 --2- 172.21.15.31:0/3159083594 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a40ab120 0x7f45a4070bc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.181+0000 7f45a95bf700 1 --2- 172.21.15.31:0/3159083594 >> v2:172.21.15.59:3300/0 conn(0x7f45a4150770 0x7f45a40c0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.181+0000 7f45a95bf700 1 -- 172.21.15.31:0/3159083594 >> 172.21.15.31:0/3159083594 conn(0x7f45a406aaf0 msgr2=0x7f45a406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.181+0000 7f7e87fff700 1 -- 172.21.15.31:0/1232890508 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800a9e0 con 0x7f7ea00c1480 2024-03-30T01:41:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.181+0000 7f7e87fff700 1 -- 172.21.15.31:0/1232890508 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9800ab40 con 0x7f7ea00c1480 2024-03-30T01:41:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.181+0000 7f7e87fff700 1 -- 172.21.15.31:0/1232890508 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98007070 con 0x7f7ea00c1480 2024-03-30T01:41:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.181+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea01f1710 con 0x7f7ea00c1480 2024-03-30T01:41:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.181+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea014a840 con 0x7f7ea00c1480 2024-03-30T01:41:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.182+0000 7f45a95bf700 1 -- 172.21.15.31:0/3159083594 shutdown_connections 2024-03-30T01:41:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.183+0000 7f7e87fff700 1 -- 172.21.15.31:0/1232890508 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7e980071d0 con 0x7f7ea00c1480 2024-03-30T01:41:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.182+0000 7f45a95bf700 1 -- 172.21.15.31:0/3159083594 wait complete. 2024-03-30T01:41:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.184+0000 7f7e87fff700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7e8c06ac90 0x7f7e8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.184+0000 7f7e87fff700 1 -- 172.21.15.31:0/1232890508 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7e98089130 con 0x7f7ea00c1480 2024-03-30T01:41:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.184+0000 7f45a95bf700 1 Processor -- start 2024-03-30T01:41:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.184+0000 7f7e9f7fe700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7e8c06ac90 0x7f7e8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.185+0000 7f45a95bf700 1 -- start start 2024-03-30T01:41:22.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.185+0000 7f7e9f7fe700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7e8c06ac90 0x7f7e8c06d150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000c00 tx=0x7f7e9001d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a95bf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4071100 0x7f45a414f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a95bf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a40ab120 0x7f45a40bb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a95bf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a4150770 0x7f45a414c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a95bf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a414ca50 con 0x7f45a4071100 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a95bf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c0050 con 0x7f45a40ab120 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a95bf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c02f0 con 0x7f45a4150770 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a40ab120 0x7f45a40bb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a40ab120 0x7f45a40bb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:39016/0 (socket says 172.21.15.31:39016) 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a27fc700 1 -- 172.21.15.31:0/755445250 learned_addr learned my addr 172.21.15.31:0/755445250 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.186+0000 7f45a2ffd700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4071100 0x7f45a414f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f45a27fc700 1 -- 172.21.15.31:0/755445250 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a4150770 msgr2=0x7f45a414c510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f45a27fc700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a4150770 0x7f45a414c510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f45a27fc700 1 -- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4071100 msgr2=0x7f45a414f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f45a27fc700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4071100 0x7f45a414f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f45a27fc700 1 -- 172.21.15.31:0/755445250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c00bd40 con 0x7f45a40ab120 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f45a2ffd700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4071100 0x7f45a414f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f45a27fc700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a40ab120 0x7f45a40bb4e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f459800b4a0 tx=0x7f459800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f458b7fe700 1 -- 172.21.15.31:0/755445250 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598011830 con 0x7f45a40ab120 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f458b7fe700 1 -- 172.21.15.31:0/755445250 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4598011990 con 0x7f45a40ab120 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.187+0000 7f458b7fe700 1 -- 172.21.15.31:0/755445250 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598011d40 con 0x7f45a40ab120 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.188+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a414ed70 con 0x7f45a40ab120 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.188+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a41eb740 con 0x7f45a40ab120 2024-03-30T01:41:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.189+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a414ebd0 con 0x7f45a40ab120 2024-03-30T01:41:22.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.190+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea0159de0 con 0x7f7ea00c1480 2024-03-30T01:41:22.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.192+0000 7f458b7fe700 1 -- 172.21.15.31:0/755445250 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f459801a400 con 0x7f45a40ab120 2024-03-30T01:41:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.193+0000 7f458b7fe700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f458c06ae20 0x7f458c06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.193+0000 7f458b7fe700 1 -- 172.21.15.31:0/755445250 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4598089ac0 con 0x7f45a40ab120 2024-03-30T01:41:22.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.194+0000 7f45a2ffd700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f458c06ae20 0x7f458c06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:22.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.194+0000 7f45a2ffd700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f458c06ae20 0x7f458c06d2e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f45a40a9380 tx=0x7f459c012000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:22.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.195+0000 7f458b7fe700 1 -- 172.21.15.31:0/755445250 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f45980549a0 con 0x7f45a40ab120 2024-03-30T01:41:22.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.200+0000 7f7e87fff700 1 -- 172.21.15.31:0/1232890508 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7e98017080 con 0x7f7ea00c1480 2024-03-30T01:41:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.481+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2f180055a0 con 0x7f2f340ab120 2024-03-30T01:41:22.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.482+0000 7f2f2affd700 1 -- 172.21.15.31:0/3098695629 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f30018020 con 0x7f2f340ab120 2024-03-30T01:41:22.492 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:22.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.492+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f2006aef0 msgr2=0x7f2f2006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.492+0000 7f2f3c3b0700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f2006aef0 0x7f2f2006d3b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c015040 tx=0x7f2f2c00a000 comp rx=0 tx=0).stop 2024-03-30T01:41:22.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.492+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f340ab120 msgr2=0x7f2f341483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.492+0000 7f2f3c3b0700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f340ab120 0x7f2f341483d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000c780 tx=0x7f2f3000ca90 comp rx=0 tx=0).stop 2024-03-30T01:41:22.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.493+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 shutdown_connections 2024-03-30T01:41:22.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.493+0000 7f2f3c3b0700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f34150770 0x7f2f34148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.493+0000 7f2f3c3b0700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f2006aef0 0x7f2f2006d3b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.493+0000 7f2f3c3b0700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f34071100 0x7f2f3415db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.493+0000 7f2f3c3b0700 1 --2- 172.21.15.31:0/3098695629 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f340ab120 0x7f2f341483d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.493+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 >> 172.21.15.31:0/3098695629 conn(0x7f2f3406aaf0 msgr2=0x7f2f34151b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.493+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 shutdown_connections 2024-03-30T01:41:22.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.493+0000 7f2f3c3b0700 1 -- 172.21.15.31:0/3098695629 wait complete. 2024-03-30T01:41:22.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-03-30T01:41:22.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.521+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1d38005870 con 0x7f1d500ab120 2024-03-30T01:41:22.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.522+0000 7f1d467fc700 1 -- 172.21.15.31:0/3975880627 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1d4c011e60 con 0x7f1d500ab120 2024-03-30T01:41:22.529 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1d3c06af00 msgr2=0x7f1d3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1d3c06af00 0x7f1d3c06d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009770 tx=0x7f1d40006cd0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d500ab120 msgr2=0x7f1d50147d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d500ab120 0x7f1d50147d00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00b450 tx=0x7f1d4c00b810 comp rx=0 tx=0).stop 2024-03-30T01:41:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 shutdown_connections 2024-03-30T01:41:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1d3c06af00 0x7f1d3c06d3c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d50150770 0x7f1d500a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d5006c860 0x7f1d500c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 --2- 172.21.15.31:0/3975880627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d500ab120 0x7f1d50147d00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 >> 172.21.15.31:0/3975880627 conn(0x7f1d5006aaf0 msgr2=0x7f1d500c1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.526+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 shutdown_connections 2024-03-30T01:41:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.527+0000 7f1d580d5700 1 -- 172.21.15.31:0/3975880627 wait complete. 2024-03-30T01:41:22.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2024-03-30T01:41:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.577+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f029c0055a0 con 0x7f02b80bd7f0 2024-03-30T01:41:22.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.577+0000 7f02affff700 1 -- 172.21.15.31:0/2663184866 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02a805b070 con 0x7f02b80bd7f0 2024-03-30T01:41:22.584 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-03-30T01:41:22.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.578+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f45a414f290 con 0x7f45a40ab120 2024-03-30T01:41:22.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.579+0000 7f458b7fe700 1 -- 172.21.15.31:0/755445250 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45a414f290 con 0x7f45a40ab120 2024-03-30T01:41:22.585 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.581+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f029806ac90 msgr2=0x7f029806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.581+0000 7f02bd7c3700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f029806ac90 0x7f029806d150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f02b8156160 tx=0x7f02a0009040 comp rx=0 tx=0).stop 2024-03-30T01:41:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.581+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b80bd7f0 msgr2=0x7f02b80c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.581+0000 7f02bd7c3700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b80bd7f0 0x7f02b80c2d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f02a800c4f0 tx=0x7f02a800c800 comp rx=0 tx=0).stop 2024-03-30T01:41:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.581+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f458c06ae20 msgr2=0x7f458c06d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.581+0000 7f45a95bf700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f458c06ae20 0x7f458c06d2e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f45a40a9380 tx=0x7f459c012000 comp rx=0 tx=0).stop 2024-03-30T01:41:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.581+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a40ab120 msgr2=0x7f45a40bb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.581+0000 7f45a95bf700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a40ab120 0x7f45a40bb4e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f459800b4a0 tx=0x7f459800b860 comp rx=0 tx=0).stop 2024-03-30T01:41:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 shutdown_connections 2024-03-30T01:41:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f02bd7c3700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f029806ac90 0x7f029806d150 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f02b8156160 tx=0x7f02a0009040 comp rx=0 tx=0).stop 2024-03-30T01:41:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f02bd7c3700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02b8154760 0x7f02b8148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f02bd7c3700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02b806b170 0x7f02b80c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f02bd7c3700 1 --2- 172.21.15.31:0/2663184866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b80bd7f0 0x7f02b80c2d60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 >> 172.21.15.31:0/2663184866 conn(0x7f02b80ca0d0 msgr2=0x7f02b8157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 shutdown_connections 2024-03-30T01:41:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f45a95bf700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45a4071100 0x7f45a414f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f45a95bf700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f458c06ae20 0x7f458c06d2e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f45a95bf700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45a4150770 0x7f45a414c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f45a95bf700 1 --2- 172.21.15.31:0/755445250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45a40ab120 0x7f45a40bb4e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 >> 172.21.15.31:0/755445250 conn(0x7f45a406aaf0 msgr2=0x7f45a414d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 shutdown_connections 2024-03-30T01:41:22.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.582+0000 7f02bd7c3700 1 -- 172.21.15.31:0/2663184866 wait complete. 2024-03-30T01:41:22.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.583+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 shutdown_connections 2024-03-30T01:41:22.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.583+0000 7f45a95bf700 1 -- 172.21.15.31:0/755445250 wait complete. 2024-03-30T01:41:22.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.591+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb18c1561e0 con 0x7fb18c06d770 2024-03-30T01:41:22.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.591+0000 7fb181ffb700 1 -- 172.21.15.31:0/4186249436 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb18c1561e0 con 0x7fb18c06d770 2024-03-30T01:41:22.598 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-03-30T01:41:22.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.593+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2c44152df0 con 0x7f2c441efbf0 2024-03-30T01:41:22.600 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.594+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb17006ad60 msgr2=0x7fb17006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.594+0000 7fb1936e9700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb17006ad60 0x7fb17006d220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb18c153930 tx=0x7fb188008040 comp rx=0 tx=0).stop 2024-03-30T01:41:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.594+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb18c06d770 msgr2=0x7fb18c0a9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.594+0000 7fb1936e9700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb18c06d770 0x7fb18c0a9b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0090b0 tx=0x7fb17c004750 comp rx=0 tx=0).stop 2024-03-30T01:41:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.595+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 shutdown_connections 2024-03-30T01:41:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.595+0000 7fb1936e9700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb17006ad60 0x7fb17006d220 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.595+0000 7fb1936e9700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb18c151e00 0x7fb18c150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.595+0000 7fb1936e9700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb18c06d770 0x7fb18c0a9b80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.595+0000 7fb1936e9700 1 --2- 172.21.15.31:0/4186249436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c14e6d0 0x7fb18c0ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.595+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 >> 172.21.15.31:0/4186249436 conn(0x7fb18c0c2d60 msgr2=0x7fb18c0bf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.597+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 shutdown_connections 2024-03-30T01:41:22.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.600+0000 7fb1936e9700 1 -- 172.21.15.31:0/4186249436 wait complete. 2024-03-30T01:41:22.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-03-30T01:41:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.594+0000 7f2c32ffd700 1 -- 172.21.15.31:0/493779762 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c38057830 con 0x7f2c441efbf0 2024-03-30T01:41:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.598+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c2406ad60 msgr2=0x7f2c2406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.598+0000 7f2c48961700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c2406ad60 0x7f2c2406d220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c34006fd0 tx=0x7f2c34008040 comp rx=0 tx=0).stop 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.598+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c441efbf0 msgr2=0x7f2c440adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.598+0000 7f2c48961700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c441efbf0 0x7f2c440adfd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c4406c1d0 tx=0x7f2c3800b670 comp rx=0 tx=0).stop 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.599+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 shutdown_connections 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.599+0000 7f2c48961700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c2406ad60 0x7f2c2406d220 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.599+0000 7f2c48961700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c440c0a80 0x7f2c441eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.599+0000 7f2c48961700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c2c005460 0x7f2c440bf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.599+0000 7f2c48961700 1 --2- 172.21.15.31:0/493779762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c441efbf0 0x7f2c440adfd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.599+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 >> 172.21.15.31:0/493779762 conn(0x7f2c440ca0d0 msgr2=0x7f2c44156220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.599+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 shutdown_connections 2024-03-30T01:41:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.599+0000 7f2c48961700 1 -- 172.21.15.31:0/493779762 wait complete. 2024-03-30T01:41:22.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-03-30T01:41:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.608+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f192c0055a0 con 0x7f194813dfd0 2024-03-30T01:41:22.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.609+0000 7f19457fa700 1 -- 172.21.15.31:0/53248322 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1930059520 con 0x7f194813dfd0 2024-03-30T01:41:22.616 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2024-03-30T01:41:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806ad60 msgr2=0x7f192806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806ad60 0x7f192806d220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f194813d180 tx=0x7f1940006cd0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f194813dfd0 msgr2=0x7f19481522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f194813dfd0 0x7f19481522f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1930009130 tx=0x7f1930004750 comp rx=0 tx=0).stop 2024-03-30T01:41:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 shutdown_connections 2024-03-30T01:41:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194806c860 0x7f19480c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806ad60 0x7f192806d220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f194813dfd0 0x7f19481522f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 --2- 172.21.15.31:0/53248322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1948150770 0x7f1948151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.613+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 >> 172.21.15.31:0/53248322 conn(0x7f194806aaf0 msgr2=0x7f19480aa090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.614+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 shutdown_connections 2024-03-30T01:41:22.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.617+0000 7f194eaed700 1 -- 172.21.15.31:0/53248322 wait complete. 2024-03-30T01:41:22.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-03-30T01:41:22.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-03-30T01:41:22.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-03-30T01:41:22.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.630+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7ea004ea80 con 0x7f7ea00c1480 2024-03-30T01:41:22.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.631+0000 7f7e87fff700 1 -- 172.21.15.31:0/1232890508 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7e980540c0 con 0x7f7ea00c1480 2024-03-30T01:41:22.638 INFO:teuthology.orchestra.run.smithi031.stdout:55834574857 2024-03-30T01:41:22.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.633+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7e8c06ac90 msgr2=0x7f7e8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.633+0000 7f7ea5e48700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7e8c06ac90 0x7f7e8c06d150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000c00 tx=0x7f7e9001d040 comp rx=0 tx=0).stop 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00c1480 msgr2=0x7f7ea00a8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00c1480 0x7f7ea00a8460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800c9f0 tx=0x7f7e9800cd00 comp rx=0 tx=0).stop 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 shutdown_connections 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea006db70 0x7f7ea00aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7e8c06ac90 0x7f7e8c06d150 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea00c1480 0x7f7ea00a8460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 --2- 172.21.15.31:0/1232890508 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ea00706c0 0x7f7ea00bce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 >> 172.21.15.31:0/1232890508 conn(0x7f7ea006aaf0 msgr2=0x7f7ea00aae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 shutdown_connections 2024-03-30T01:41:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:22.634+0000 7f7ea5e48700 1 -- 172.21.15.31:0/1232890508 wait complete. 2024-03-30T01:41:22.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-03-30T01:41:23.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-30T01:41:23.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-30T01:41:23.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-30T01:41:23.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T01:41:23.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-30T01:41:23.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-30T01:41:23.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-30T01:41:23.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-30T01:41:23.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.682+0000 7f674d7e3700 1 Processor -- start 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.682+0000 7f674d7e3700 1 -- start start 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.683+0000 7f674d7e3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67480722b0 0x7f674806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.683+0000 7f674d7e3700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67480a88c0 con 0x7f674806db20 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.683+0000 7f674d7e3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67480a85e0 con 0x7f6748154d20 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.683+0000 7f674d7e3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67480c3110 con 0x7f67480722b0 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.684+0000 7f6746ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6748154d20 0x7f6748155100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56534/0 (socket says 172.21.15.31:56534) 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.684+0000 7f6746ffd700 1 -- 172.21.15.31:0/2316082872 learned_addr learned my addr 172.21.15.31:0/2316082872 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.684+0000 7f67467fc700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67480722b0 0x7f674806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.684+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008459278 0 0) 0x7f67480a85e0 con 0x7f6748154d20 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.684+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67300057c0 con 0x7f6748154d20 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610477009 0 0) 0x7f67480a88c0 con 0x7f674806db20 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67480a85e0 con 0x7f674806db20 2024-03-30T01:41:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 <== mon.1 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3630573128 0 0) 0x7f67300057c0 con 0x7f6748154d20 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67480722b0 msgr2=0x7f674806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67480722b0 0x7f674806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 >> v1:172.21.15.31:6789/0 conn(0x7f674806db20 legacy=0x7f67481479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67480c2a50 con 0x7f6748154d20 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 <== mon.1 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f67380015f0 con 0x7f6748154d20 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 >> v1:172.21.15.59:6789/0 conn(0x7f6748154d20 legacy=0x7f6748155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730005da0 0x7f6730006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6730006a60 0x7f6730006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6730007420 0x7f6730014810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67480c3110 con 0x7f6730005da0 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673000bc10 con 0x7f6730007420 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673000bdd0 con 0x7f6730006a60 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f67477fe700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6730007420 0x7f6730014810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.685+0000 7f67467fc700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6730006a60 0x7f6730006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.686+0000 7f6746ffd700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730005da0 0x7f6730006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.686+0000 7f6746ffd700 1 -- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6730006a60 msgr2=0x7f6730006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.686+0000 7f6746ffd700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6730006a60 0x7f6730006ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.686+0000 7f6746ffd700 1 -- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6730007420 msgr2=0x7f6730014810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.686+0000 7f6746ffd700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6730007420 0x7f6730014810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.686+0000 7f6746ffd700 1 -- 172.21.15.31:0/2316082872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67480c2a50 con 0x7f6730005da0 2024-03-30T01:41:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.686+0000 7f6746ffd700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730005da0 0x7f6730006180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f673800c190 tx=0x7f6730007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d688a5cff05ccaac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.687+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800cd30 con 0x7f6730005da0 2024-03-30T01:41:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.687+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673800ce90 con 0x7f6730005da0 2024-03-30T01:41:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.687+0000 7f6745ffb700 1 -- 172.21.15.31:0/2316082872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738005b80 con 0x7f6730005da0 2024-03-30T01:41:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.688+0000 7f674d7e3700 1 -- 172.21.15.31:0/2316082872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730005da0 msgr2=0x7f6730006180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.688+0000 7f674d7e3700 1 --2- 172.21.15.31:0/2316082872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6730005da0 0x7f6730006180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f673800c190 tx=0x7f6730007a90 comp rx=0 tx=0).stop 2024-03-30T01:41:23.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.689+0000 7f67477fe700 1 -- 172.21.15.31:0/2316082872 reap_dead start 2024-03-30T01:41:23.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.689+0000 7f674d7e3700 1 -- 172.21.15.31:0/2316082872 shutdown_connections 2024-03-30T01:41:23.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.689+0000 7f674d7e3700 1 -- 172.21.15.31:0/2316082872 >> 172.21.15.31:0/2316082872 conn(0x7f67480c3a60 msgr2=0x7f67480c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:23.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.689+0000 7f674d7e3700 1 -- 172.21.15.31:0/2316082872 shutdown_connections 2024-03-30T01:41:23.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.689+0000 7f674d7e3700 1 -- 172.21.15.31:0/2316082872 wait complete. 2024-03-30T01:41:23.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.691+0000 7f674d7e3700 1 Processor -- start 2024-03-30T01:41:23.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.691+0000 7f674d7e3700 1 -- start start 2024-03-30T01:41:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.692+0000 7f674d7e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674806a970 0x7f6748149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.692+0000 7f674d7e3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67481496b0 0x7f67481406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.692+0000 7f674d7e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67480c5760 0x7f6748140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.692+0000 7f674d7e3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67481e5fe0 con 0x7f67480c5760 2024-03-30T01:41:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.692+0000 7f674d7e3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67481e6510 con 0x7f674806a970 2024-03-30T01:41:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.692+0000 7f674d7e3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67480c52d0 con 0x7f67481496b0 2024-03-30T01:41:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f67467fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67481496b0 0x7f67481406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f67467fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67481496b0 0x7f67481406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60218/0 (socket says 172.21.15.31:60218) 2024-03-30T01:41:23.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f67467fc700 1 -- 172.21.15.31:0/3137778424 learned_addr learned my addr 172.21.15.31:0/3137778424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f6746ffd700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674806a970 0x7f6748149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f67477fe700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67480c5760 0x7f6748140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f67467fc700 1 -- 172.21.15.31:0/3137778424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674806a970 msgr2=0x7f6748149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f67467fc700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674806a970 0x7f6748149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f67467fc700 1 -- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67480c5760 msgr2=0x7f6748140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f67467fc700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67480c5760 0x7f6748140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.693+0000 7f67467fc700 1 -- 172.21.15.31:0/3137778424 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6748141140 con 0x7f67481496b0 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.694+0000 7f67467fc700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67481496b0 0x7f67481406c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f673c011960 tx=0x7f673c011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.694+0000 7f672f7fe700 1 -- 172.21.15.31:0/3137778424 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c011eb0 con 0x7f67481496b0 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.694+0000 7f672f7fe700 1 -- 172.21.15.31:0/3137778424 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c00f940 con 0x7f67481496b0 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.694+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674814b3f0 con 0x7f67481496b0 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.694+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67480c4c80 con 0x7f67481496b0 2024-03-30T01:41:23.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.694+0000 7f672f7fe700 1 -- 172.21.15.31:0/3137778424 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c00fcf0 con 0x7f67481496b0 2024-03-30T01:41:23.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.696+0000 7f67477fe700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67480c5760 0x7f6748140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:23.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.696+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67280052f0 con 0x7f67481496b0 2024-03-30T01:41:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.699+0000 7f672f7fe700 1 -- 172.21.15.31:0/3137778424 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f673c014440 con 0x7f67481496b0 2024-03-30T01:41:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.700+0000 7f672f7fe700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f673406ac90 0x7f673406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.700+0000 7f672f7fe700 1 -- 172.21.15.31:0/3137778424 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f673c08a200 con 0x7f67481496b0 2024-03-30T01:41:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.700+0000 7f6746ffd700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f673406ac90 0x7f673406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.701+0000 7f6746ffd700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f673406ac90 0x7f673406d150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6738000c00 tx=0x7f6738010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.703+0000 7f672f7fe700 1 -- 172.21.15.31:0/3137778424 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f673c055160 con 0x7f67481496b0 2024-03-30T01:41:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.745+0000 7f5a7efce700 1 Processor -- start 2024-03-30T01:41:23.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.745+0000 7f5a7efce700 1 -- start start 2024-03-30T01:41:23.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.747+0000 7f5a7efce700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a78150770 0x7f5a780c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.747+0000 7f5a7efce700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5a7806c860 0x7f5a780c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.747+0000 7f5a7efce700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a78153d60 con 0x7f5a780ab120 2024-03-30T01:41:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.747+0000 7f5a7efce700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a78071e70 con 0x7f5a78150770 2024-03-30T01:41:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.747+0000 7f5a7efce700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a781548e0 con 0x7f5a7806c860 2024-03-30T01:41:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.748+0000 7f5a7cd6a700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5a780ab120 0x7f5a78070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56548/0 (socket says 172.21.15.31:56548) 2024-03-30T01:41:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.748+0000 7f5a7cd6a700 1 -- 172.21.15.31:0/405247059 learned_addr learned my addr 172.21.15.31:0/405247059 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.748+0000 7f5a7d56b700 1 --2- 172.21.15.31:0/405247059 >> v2:172.21.15.59:3300/0 conn(0x7f5a7806c860 0x7f5a780c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.748+0000 7f5a77fff700 1 --2- 172.21.15.31:0/405247059 >> v2:172.21.15.31:3300/0 conn(0x7f5a78150770 0x7f5a780c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.749+0000 7f5a77fff700 1 -- 172.21.15.31:0/405247059 >> v2:172.21.15.59:3300/0 conn(0x7f5a7806c860 msgr2=0x7f5a780c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.749+0000 7f5a77fff700 1 --2- 172.21.15.31:0/405247059 >> v2:172.21.15.59:3300/0 conn(0x7f5a7806c860 0x7f5a780c0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.749+0000 7f5a77fff700 1 -- 172.21.15.31:0/405247059 >> v1:172.21.15.59:6789/0 conn(0x7f5a780ab120 legacy=0x7f5a78070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.749+0000 7f5a77fff700 1 -- 172.21.15.31:0/405247059 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7806c450 con 0x7f5a78150770 2024-03-30T01:41:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.750+0000 7f5a77fff700 1 --2- 172.21.15.31:0/405247059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a78150770 0x7f5a780c0800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a7806be50 tx=0x7f5a6800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887c624d9a446a72 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.750+0000 7f5a777fe700 1 -- 172.21.15.31:0/405247059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800dc70 con 0x7f5a78150770 2024-03-30T01:41:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.750+0000 7f5a777fe700 1 -- 172.21.15.31:0/405247059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6800ddd0 con 0x7f5a78150770 2024-03-30T01:41:23.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.750+0000 7f5a777fe700 1 -- 172.21.15.31:0/405247059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68004270 con 0x7f5a78150770 2024-03-30T01:41:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.751+0000 7f5a7efce700 1 -- 172.21.15.31:0/405247059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a78150770 msgr2=0x7f5a780c0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.751+0000 7f5a7efce700 1 --2- 172.21.15.31:0/405247059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a78150770 0x7f5a780c0800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a7806be50 tx=0x7f5a6800bdc0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.752+0000 7f5a7efce700 1 -- 172.21.15.31:0/405247059 shutdown_connections 2024-03-30T01:41:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.752+0000 7f5a7efce700 1 --2- 172.21.15.31:0/405247059 >> v2:172.21.15.59:3300/0 conn(0x7f5a7806c860 0x7f5a780c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.752+0000 7f5a7efce700 1 --2- 172.21.15.31:0/405247059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a78150770 0x7f5a780c0800 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.752+0000 7f5a7efce700 1 -- 172.21.15.31:0/405247059 >> 172.21.15.31:0/405247059 conn(0x7f5a7806aaf0 msgr2=0x7f5a7806af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.752+0000 7f5a7efce700 1 -- 172.21.15.31:0/405247059 shutdown_connections 2024-03-30T01:41:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.752+0000 7f5a7efce700 1 -- 172.21.15.31:0/405247059 wait complete. 2024-03-30T01:41:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.754+0000 7f5a7efce700 1 Processor -- start 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.754+0000 7f5a7efce700 1 -- start start 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.756+0000 7f5a7efce700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a7806c860 0x7f5a7814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.756+0000 7f5a7efce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a780ab120 0x7f5a781f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.756+0000 7f5a7efce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a78150770 0x7f5a781f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.756+0000 7f5a7efce700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78146210 con 0x7f5a780ab120 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.756+0000 7f5a7efce700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780c39f0 con 0x7f5a78150770 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.756+0000 7f5a7efce700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781464b0 con 0x7f5a7806c860 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.756+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a780ab120 0x7f5a781f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.756+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a780ab120 0x7f5a781f1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36942/0 (socket says 172.21.15.31:36942) 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.757+0000 7f5a77fff700 1 -- 172.21.15.31:0/2159960815 learned_addr learned my addr 172.21.15.31:0/2159960815 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.757+0000 7f5a7cd6a700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a7806c860 0x7f5a7814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.757+0000 7f5a77fff700 1 -- 172.21.15.31:0/2159960815 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a7806c860 msgr2=0x7f5a7814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.757+0000 7f5a77fff700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a7806c860 0x7f5a7814c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.757+0000 7f5a77fff700 1 -- 172.21.15.31:0/2159960815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a78150770 msgr2=0x7f5a781f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.757+0000 7f5a77fff700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a78150770 0x7f5a781f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.757+0000 7f5a77fff700 1 -- 172.21.15.31:0/2159960815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6800b740 con 0x7f5a780ab120 2024-03-30T01:41:23.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.758+0000 7f5a77fff700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a780ab120 0x7f5a781f1cb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800be00 tx=0x7f5a68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.759+0000 7f5a757fa700 1 -- 172.21.15.31:0/2159960815 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68009040 con 0x7f5a780ab120 2024-03-30T01:41:23.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.759+0000 7f5a757fa700 1 -- 172.21.15.31:0/2159960815 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a68003c60 con 0x7f5a780ab120 2024-03-30T01:41:23.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.759+0000 7f5a757fa700 1 -- 172.21.15.31:0/2159960815 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68041470 con 0x7f5a780ab120 2024-03-30T01:41:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.759+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a780c3cd0 con 0x7f5a780ab120 2024-03-30T01:41:23.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.759+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a781ee130 con 0x7f5a780ab120 2024-03-30T01:41:23.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.764+0000 7f5a757fa700 1 -- 172.21.15.31:0/2159960815 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5a680415d0 con 0x7f5a780ab120 2024-03-30T01:41:23.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.765+0000 7f5a757fa700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a6406ad60 0x7f5a6406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.765+0000 7f5a757fa700 1 -- 172.21.15.31:0/2159960815 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5a680acec0 con 0x7f5a780ab120 2024-03-30T01:41:23.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.766+0000 7f5a7cd6a700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a6406ad60 0x7f5a6406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.767+0000 7f5a7cd6a700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a6406ad60 0x7f5a6406d220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a70004ce0 tx=0x7f5a70004c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.767+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a580052f0 con 0x7f5a780ab120 2024-03-30T01:41:23.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.777+0000 7f5a757fa700 1 -- 172.21.15.31:0/2159960815 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5a68077f40 con 0x7f5a780ab120 2024-03-30T01:41:23.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.823+0000 7fb5a5e1c700 1 Processor -- start 2024-03-30T01:41:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.823+0000 7fb5a5e1c700 1 -- start start 2024-03-30T01:41:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.824+0000 7fb5a5e1c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5a00ab120 0x7fb5a0070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.824+0000 7fb5a5e1c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5a0071100 0x7fb5a00c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.824+0000 7fb5a5e1c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a0153d60 con 0x7fb5a0150770 2024-03-30T01:41:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.824+0000 7fb5a5e1c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a0071e70 con 0x7fb5a00ab120 2024-03-30T01:41:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.824+0000 7fb5a5e1c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a00c1450 con 0x7fb5a0071100 2024-03-30T01:41:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.825+0000 7fb59effd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb5a0150770 0x7fb5a00c0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56550/0 (socket says 172.21.15.31:56550) 2024-03-30T01:41:23.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.825+0000 7fb59effd700 1 -- 172.21.15.31:0/3789247809 learned_addr learned my addr 172.21.15.31:0/3789247809 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.825+0000 7fb59ffff700 1 --2- 172.21.15.31:0/3789247809 >> v2:172.21.15.59:3300/0 conn(0x7fb5a0071100 0x7fb5a00c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.826+0000 7fb59ffff700 1 -- 172.21.15.31:0/3789247809 >> v2:172.21.15.31:3300/0 conn(0x7fb5a00ab120 msgr2=0x7fb5a0070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.826+0000 7fb59ffff700 1 --2- 172.21.15.31:0/3789247809 >> v2:172.21.15.31:3300/0 conn(0x7fb5a00ab120 0x7fb5a0070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.826+0000 7fb59ffff700 1 -- 172.21.15.31:0/3789247809 >> v1:172.21.15.59:6789/0 conn(0x7fb5a0150770 legacy=0x7fb5a00c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.826+0000 7fb59ffff700 1 -- 172.21.15.31:0/3789247809 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a01548e0 con 0x7fb5a0071100 2024-03-30T01:41:23.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.826+0000 7fb59ffff700 1 --2- 172.21.15.31:0/3789247809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0071100 0x7fb5a00c0f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5a006be50 tx=0x7fb590009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b586bcfe6bb5a4c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.828+0000 7fb59e7fc700 1 -- 172.21.15.31:0/3789247809 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590004230 con 0x7fb5a0071100 2024-03-30T01:41:23.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.828+0000 7fb59e7fc700 1 -- 172.21.15.31:0/3789247809 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb590004390 con 0x7fb5a0071100 2024-03-30T01:41:23.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.828+0000 7fb59e7fc700 1 -- 172.21.15.31:0/3789247809 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5900036a0 con 0x7fb5a0071100 2024-03-30T01:41:23.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.829+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/3789247809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0071100 msgr2=0x7fb5a00c0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.829+0000 7fb5a5e1c700 1 --2- 172.21.15.31:0/3789247809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0071100 0x7fb5a00c0f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5a006be50 tx=0x7fb590009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:23.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.829+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/3789247809 shutdown_connections 2024-03-30T01:41:23.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.829+0000 7fb5a5e1c700 1 --2- 172.21.15.31:0/3789247809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0071100 0x7fb5a00c0f10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.829+0000 7fb5a5e1c700 1 --2- 172.21.15.31:0/3789247809 >> v2:172.21.15.31:3300/0 conn(0x7fb5a00ab120 0x7fb5a0070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.829+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/3789247809 >> 172.21.15.31:0/3789247809 conn(0x7fb5a006aaf0 msgr2=0x7fb5a006af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:23.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.830+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/3789247809 shutdown_connections 2024-03-30T01:41:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.830+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/3789247809 wait complete. 2024-03-30T01:41:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.832+0000 7fb5a5e1c700 1 Processor -- start 2024-03-30T01:41:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.833+0000 7fb5a5e1c700 1 -- start start 2024-03-30T01:41:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.834+0000 7fb5a5e1c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5a0071100 0x7fb5a014a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.834+0000 7fb5a5e1c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a00ab120 0x7fb5a01f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.834+0000 7fb5a5e1c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0150770 0x7fb5a01f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.834+0000 7fb5a5e1c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a01f8fc0 con 0x7fb5a00ab120 2024-03-30T01:41:23.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.834+0000 7fb5a5e1c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a0153b50 con 0x7fb5a0150770 2024-03-30T01:41:23.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.834+0000 7fb5a5e1c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a01532b0 con 0x7fb5a0071100 2024-03-30T01:41:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.834+0000 7fb59effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a00ab120 0x7fb5a01f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.834+0000 7fb59effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a00ab120 0x7fb5a01f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36958/0 (socket says 172.21.15.31:36958) 2024-03-30T01:41:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.834+0000 7fb59effd700 1 -- 172.21.15.31:0/134511503 learned_addr learned my addr 172.21.15.31:0/134511503 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.835+0000 7fb59ffff700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0150770 0x7fb5a01f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.835+0000 7fb59f7fe700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5a0071100 0x7fb5a014a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.835+0000 7fb59ffff700 1 -- 172.21.15.31:0/134511503 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5a0071100 msgr2=0x7fb5a014a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.835+0000 7fb59ffff700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5a0071100 0x7fb5a014a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.835+0000 7fb59ffff700 1 -- 172.21.15.31:0/134511503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a00ab120 msgr2=0x7fb5a01f9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.835+0000 7fb59ffff700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a00ab120 0x7fb5a01f9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.835+0000 7fb59ffff700 1 -- 172.21.15.31:0/134511503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5900094a0 con 0x7fb5a0150770 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.836+0000 7fb59ffff700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0150770 0x7fb5a01f9990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb590016000 tx=0x7fb590004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.838+0000 7fb587fff700 1 -- 172.21.15.31:0/134511503 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590003e60 con 0x7fb5a0150770 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.838+0000 7fb587fff700 1 -- 172.21.15.31:0/134511503 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb590030070 con 0x7fb5a0150770 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.838+0000 7fb587fff700 1 -- 172.21.15.31:0/134511503 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590010820 con 0x7fb5a0150770 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.838+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a0153590 con 0x7fb5a0150770 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.838+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a014a820 con 0x7fb5a0150770 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.843+0000 7fb587fff700 1 -- 172.21.15.31:0/134511503 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb590032030 con 0x7fb5a0150770 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.843+0000 7fb587fff700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb58c06ad60 0x7fb58c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.843+0000 7fb587fff700 1 -- 172.21.15.31:0/134511503 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb59009b6f0 con 0x7fb5a0150770 2024-03-30T01:41:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.844+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5800052f0 con 0x7fb5a0150770 2024-03-30T01:41:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.845+0000 7fb59f7fe700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb58c06ad60 0x7fb58c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.846+0000 7fb59f7fe700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb58c06ad60 0x7fb58c06d220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00c23b0 tx=0x7fb594013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.850+0000 7fb587fff700 1 -- 172.21.15.31:0/134511503 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb590066650 con 0x7fb5a0150770 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.889+0000 7faaa9de9700 1 Processor -- start 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.889+0000 7faaa9de9700 1 -- start start 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.890+0000 7faaa9de9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa4155100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.890+0000 7faaa9de9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faaa40c3530 0x7faaa41479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.890+0000 7faaa9de9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa40a88c0 con 0x7faaa40722b0 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.890+0000 7faaa9de9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa40c3110 con 0x7faaa40c3530 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.890+0000 7faaa9de9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40a85e0 con 0x7faaa4154d20 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.891+0000 7faaa37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa4155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.891+0000 7faaa3fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faaa40c3530 0x7faaa41479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.891+0000 7faaa37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa4155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60244/0 (socket says 172.21.15.31:60244) 2024-03-30T01:41:23.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.891+0000 7faaa37fe700 1 -- 172.21.15.31:0/1400149281 learned_addr learned my addr 172.21.15.31:0/1400149281 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.891+0000 7faaa3fff700 1 -- 172.21.15.31:0/1400149281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 msgr2=0x7faaa4155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.891+0000 7faaa3fff700 1 --2- 172.21.15.31:0/1400149281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa4155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.891+0000 7faaa3fff700 1 -- 172.21.15.31:0/1400149281 >> v1:172.21.15.59:6789/0 conn(0x7faaa40722b0 legacy=0x7faaa406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.891+0000 7faaa3fff700 1 -- 172.21.15.31:0/1400149281 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa40c2a50 con 0x7faaa40c3530 2024-03-30T01:41:23.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.892+0000 7faaa3fff700 1 --2- 172.21.15.31:0/1400149281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa40c3530 0x7faaa41479d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009d90 tx=0x7faa9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19daa1efda10fb8c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.893+0000 7f7a3fdf6700 1 Processor -- start 2024-03-30T01:41:23.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.893+0000 7f7a3fdf6700 1 -- start start 2024-03-30T01:41:23.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.894+0000 7f6c08fd7700 1 Processor -- start 2024-03-30T01:41:23.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.894+0000 7f6c08fd7700 1 -- start start 2024-03-30T01:41:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.895+0000 7f6c08fd7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6c0414e2c0 0x7f6c0406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.895+0000 7f6c08fd7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c040c4420 0x7f6c04150b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.895+0000 7f6c08fd7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c040c1660 con 0x7f6c0406e000 2024-03-30T01:41:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.895+0000 7f6c08fd7700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c040c0cd0 con 0x7f6c0414e2c0 2024-03-30T01:41:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.895+0000 7f6c08fd7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c04151750 con 0x7f6c040c4420 2024-03-30T01:41:23.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.894+0000 7f7a3fdf6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7a380c4f60 0x7f7a380c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.894+0000 7f7a3fdf6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a380c62b0 0x7f7a3815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.894+0000 7f7a3fdf6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a380c7fd0 con 0x7f7a3806b400 2024-03-30T01:41:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.894+0000 7f7a3fdf6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a38154830 con 0x7f7a380c4f60 2024-03-30T01:41:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.894+0000 7f7a3fdf6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380c5380 con 0x7f7a380c62b0 2024-03-30T01:41:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.894+0000 7f7a3db92700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7a3806b400 0x7f7a38151a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56574/0 (socket says 172.21.15.31:56574) 2024-03-30T01:41:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.894+0000 7f7a3db92700 1 -- 172.21.15.31:0/8444043 learned_addr learned my addr 172.21.15.31:0/8444043 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.895+0000 7f7a3e393700 1 --2- 172.21.15.31:0/8444043 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a380c62b0 0x7f7a3815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.895+0000 7f7a3cb90700 1 -- 172.21.15.31:0/8444043 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637794734 0 0) 0x7f7a380c7fd0 con 0x7f7a3806b400 2024-03-30T01:41:23.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f7a3cb90700 1 -- 172.21.15.31:0/8444043 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a240035b0 con 0x7f7a3806b400 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c01d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c040c4420 0x7f6c04150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c0259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6c0414e2c0 0x7f6c0406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c02d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6c0406e000 0x7f6c04151040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56576/0 (socket says 172.21.15.31:56576) 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c02d9d700 1 -- 172.21.15.31:0/1309095567 learned_addr learned my addr 172.21.15.31:0/1309095567 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c0259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6c0414e2c0 0x7f6c0406dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:39092/0 (socket says 172.21.15.31:39092) 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c01d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c040c4420 0x7f6c04150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60262/0 (socket says 172.21.15.31:60262) 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c0259c700 1 -- 172.21.15.31:0/1309095567 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c040c4420 msgr2=0x7f6c04150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c0259c700 1 --2- 172.21.15.31:0/1309095567 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c040c4420 0x7f6c04150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c0259c700 1 -- 172.21.15.31:0/1309095567 >> v1:172.21.15.59:6789/0 conn(0x7f6c0406e000 legacy=0x7f6c04151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c0259c700 1 -- 172.21.15.31:0/1309095567 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c040c6330 con 0x7f6c0414e2c0 2024-03-30T01:41:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f7a3cb90700 1 -- 172.21.15.31:0/8444043 <== mon.1 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23950202 0 0) 0x7f7a240035b0 con 0x7f7a3806b400 2024-03-30T01:41:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f7a3e393700 1 -- 172.21.15.31:0/8444043 >> v2:172.21.15.59:3300/0 conn(0x7f7a380c4f60 msgr2=0x7f7a380c5d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T01:41:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f7a3e393700 1 --2- 172.21.15.31:0/8444043 >> v2:172.21.15.59:3300/0 conn(0x7f7a380c4f60 0x7f7a380c5d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f7a3e393700 1 -- 172.21.15.31:0/8444043 >> v1:172.21.15.59:6789/0 conn(0x7f7a3806b400 legacy=0x7f7a38151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f7a3e393700 1 -- 172.21.15.31:0/8444043 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38154a20 con 0x7f7a380c62b0 2024-03-30T01:41:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f7a3e393700 1 --2- 172.21.15.31:0/8444043 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a380c62b0 0x7f7a3815f1b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009d90 tx=0x7f7a3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9622e1b93f9991ab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.896+0000 7f6c0259c700 1 --2- 172.21.15.31:0/1309095567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 0x7f6c0406dac0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009840 tx=0x7f6bf4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d87a4c7f2882c9e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.897+0000 7faaa27fc700 1 -- 172.21.15.31:0/1400149281 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00e070 con 0x7faaa40c3530 2024-03-30T01:41:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.897+0000 7faaa27fc700 1 -- 172.21.15.31:0/1400149281 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c00bb20 con 0x7faaa40c3530 2024-03-30T01:41:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.897+0000 7faaa27fc700 1 -- 172.21.15.31:0/1400149281 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c012430 con 0x7faaa40c3530 2024-03-30T01:41:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.898+0000 7faaa9de9700 1 -- 172.21.15.31:0/1400149281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa40c3530 msgr2=0x7faaa41479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.898+0000 7faaa9de9700 1 --2- 172.21.15.31:0/1400149281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa40c3530 0x7faaa41479d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009d90 tx=0x7faa9c00b330 comp rx=0 tx=0).stop 2024-03-30T01:41:23.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.898+0000 7fac36087700 1 Processor -- start 2024-03-30T01:41:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.898+0000 7faaa9de9700 1 -- 172.21.15.31:0/1400149281 shutdown_connections 2024-03-30T01:41:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.898+0000 7faaa9de9700 1 --2- 172.21.15.31:0/1400149281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa40c3530 0x7faaa41479d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.898+0000 7faaa9de9700 1 --2- 172.21.15.31:0/1400149281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa4155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.898+0000 7faaa9de9700 1 -- 172.21.15.31:0/1400149281 >> 172.21.15.31:0/1400149281 conn(0x7faaa40c3a60 msgr2=0x7faaa40c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:23.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.899+0000 7fac36087700 1 -- start start 2024-03-30T01:41:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7fac36087700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fac300a5920 0x7fac300c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7fac36087700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac300c0f20 0x7fac300a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.899+0000 7f6c0159a700 1 -- 172.21.15.31:0/1309095567 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4004230 con 0x7f6c0414e2c0 2024-03-30T01:41:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.899+0000 7f6c0159a700 1 -- 172.21.15.31:0/1309095567 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf4004390 con 0x7f6c0414e2c0 2024-03-30T01:41:23.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.899+0000 7f6c0159a700 1 -- 172.21.15.31:0/1309095567 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40036a0 con 0x7f6c0414e2c0 2024-03-30T01:41:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.899+0000 7f7a3cb90700 1 -- 172.21.15.31:0/8444043 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000e070 con 0x7f7a380c62b0 2024-03-30T01:41:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.899+0000 7f7a3cb90700 1 -- 172.21.15.31:0/8444043 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a30003a20 con 0x7f7a380c62b0 2024-03-30T01:41:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.899+0000 7f7a3cb90700 1 -- 172.21.15.31:0/8444043 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28001670 con 0x7f7a380c62b0 2024-03-30T01:41:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/8444043 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a380c62b0 msgr2=0x7f7a3815f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7f7a3fdf6700 1 --2- 172.21.15.31:0/8444043 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a380c62b0 0x7f7a3815f1b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009d90 tx=0x7f7a3000b330 comp rx=0 tx=0).stop 2024-03-30T01:41:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/8444043 shutdown_connections 2024-03-30T01:41:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7f7a3fdf6700 1 --2- 172.21.15.31:0/8444043 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a380c62b0 0x7f7a3815f1b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7f7a3fdf6700 1 --2- 172.21.15.31:0/8444043 >> v2:172.21.15.59:3300/0 conn(0x7f7a380c4f60 0x7f7a380c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/8444043 >> 172.21.15.31:0/8444043 conn(0x7f7a380ca0d0 msgr2=0x7f7a380ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:23.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7fac36087700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac3006dac0 0x7fac3006df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7fac36087700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac301569b0 con 0x7fac300c0f20 2024-03-30T01:41:23.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7fac36087700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac30158cd0 con 0x7fac300a5920 2024-03-30T01:41:23.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.900+0000 7fac36087700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac30158f90 con 0x7fac3006dac0 2024-03-30T01:41:23.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fac300a5920 0x7fac300c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fac300a5920 0x7fac300c2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:39106/0 (socket says 172.21.15.31:39106) 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2f7fe700 1 -- 172.21.15.31:0/2110975258 learned_addr learned my addr 172.21.15.31:0/2110975258 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2effd700 1 --2- 172.21.15.31:0/2110975258 >> v2:172.21.15.31:3300/0 conn(0x7fac300c0f20 0x7fac300a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2ffff700 1 --2- 172.21.15.31:0/2110975258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac3006dac0 0x7fac3006df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2f7fe700 1 -- 172.21.15.31:0/2110975258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac3006dac0 msgr2=0x7fac3006df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/2110975258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac3006dac0 0x7fac3006df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2f7fe700 1 -- 172.21.15.31:0/2110975258 >> v2:172.21.15.31:3300/0 conn(0x7fac300c0f20 msgr2=0x7fac300a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/2110975258 >> v2:172.21.15.31:3300/0 conn(0x7fac300c0f20 0x7fac300a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2f7fe700 1 -- 172.21.15.31:0/2110975258 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac30159270 con 0x7fac300a5920 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2ffff700 1 --2- 172.21.15.31:0/2110975258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac3006dac0 0x7fac3006df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7f6c08fd7700 1 -- 172.21.15.31:0/1309095567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 msgr2=0x7f6c0406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7f6c08fd7700 1 --2- 172.21.15.31:0/1309095567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 0x7f6c0406dac0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009840 tx=0x7f6bf4009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:23.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/2110975258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 0x7fac300c2750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fac300a90f0 tx=0x7fac20009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac814b5506ef03b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/8444043 shutdown_connections 2024-03-30T01:41:23.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.901+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/8444043 wait complete. 2024-03-30T01:41:23.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.903+0000 7f7a3fdf6700 1 Processor -- start 2024-03-30T01:41:23.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.903+0000 7f6c08fd7700 1 -- 172.21.15.31:0/1309095567 shutdown_connections 2024-03-30T01:41:23.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.903+0000 7f6c08fd7700 1 --2- 172.21.15.31:0/1309095567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 0x7f6c0406dac0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.903+0000 7f6c08fd7700 1 --2- 172.21.15.31:0/1309095567 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c040c4420 0x7f6c04150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.903+0000 7f6c08fd7700 1 -- 172.21.15.31:0/1309095567 >> 172.21.15.31:0/1309095567 conn(0x7f6c040a82d0 msgr2=0x7f6c040a86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.904+0000 7f6c08fd7700 1 -- 172.21.15.31:0/1309095567 shutdown_connections 2024-03-30T01:41:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.904+0000 7f6c08fd7700 1 -- 172.21.15.31:0/1309095567 wait complete. 2024-03-30T01:41:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.906+0000 7f6c08fd7700 1 Processor -- start 2024-03-30T01:41:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.906+0000 7f7a3fdf6700 1 -- start start 2024-03-30T01:41:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7fac2e7fc700 1 -- 172.21.15.31:0/2110975258 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20011830 con 0x7fac300a5920 2024-03-30T01:41:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7fac2e7fc700 1 -- 172.21.15.31:0/2110975258 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac20011990 con 0x7fac300a5920 2024-03-30T01:41:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7fac2e7fc700 1 -- 172.21.15.31:0/2110975258 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20011d20 con 0x7fac300a5920 2024-03-30T01:41:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7f7a3fdf6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a3806b400 0x7f7a380c0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7f7a3fdf6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a380c4f60 0x7f7a380aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7f7a3fdf6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a380c62b0 0x7f7a3814aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7f7a3fdf6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a381e9950 con 0x7f7a380c4f60 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7f7a3fdf6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38158810 con 0x7f7a380c62b0 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7f7a3fdf6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380acd50 con 0x7f7a3806b400 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3d391700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a380c4f60 0x7f7a380aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3d391700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a380c4f60 0x7f7a380aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36968/0 (socket says 172.21.15.31:36968) 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3d391700 1 -- 172.21.15.31:0/115141860 learned_addr learned my addr 172.21.15.31:0/115141860 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3db92700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a3806b400 0x7f7a380c0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3e393700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a380c62b0 0x7f7a3814aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3db92700 1 -- 172.21.15.31:0/115141860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a380c62b0 msgr2=0x7f7a3814aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3db92700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a380c62b0 0x7f7a3814aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3db92700 1 -- 172.21.15.31:0/115141860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a380c4f60 msgr2=0x7f7a380aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3db92700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a380c4f60 0x7f7a380aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.908+0000 7f7a3db92700 1 -- 172.21.15.31:0/115141860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a30009a70 con 0x7f7a3806b400 2024-03-30T01:41:23.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7faaa9de9700 1 -- 172.21.15.31:0/1400149281 shutdown_connections 2024-03-30T01:41:23.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.907+0000 7faaa9de9700 1 -- 172.21.15.31:0/1400149281 wait complete. 2024-03-30T01:41:23.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.909+0000 7faaa9de9700 1 Processor -- start 2024-03-30T01:41:23.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.909+0000 7f6c08fd7700 1 -- start start 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c08fd7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c0406e000 0x7f6c0414bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c08fd7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c040c4420 0x7f6c041f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c08fd7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 0x7f6c0414d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c08fd7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c041fa760 con 0x7f6c040c4420 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c08fd7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040a58c0 con 0x7f6c0414e2c0 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c08fd7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c04071710 con 0x7f6c0406e000 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c02d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 0x7f6c0414d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c02d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 0x7f6c0414d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:39122/0 (socket says 172.21.15.31:39122) 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c02d9d700 1 -- 172.21.15.31:0/2380808182 learned_addr learned my addr 172.21.15.31:0/2380808182 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c01d9b700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c040c4420 0x7f6c041f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c02d9d700 1 -- 172.21.15.31:0/2380808182 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c0406e000 msgr2=0x7f6c0414bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c02d9d700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c0406e000 0x7f6c0414bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c02d9d700 1 -- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c040c4420 msgr2=0x7f6c041f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c02d9d700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c040c4420 0x7f6c041f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7f6c02d9d700 1 -- 172.21.15.31:0/2380808182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf40094a0 con 0x7f6c0414e2c0 2024-03-30T01:41:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.909+0000 7faaa9de9700 1 -- start start 2024-03-30T01:41:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7faaa9de9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa40722b0 0x7faaa4144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7faaa9de9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaa40c3530 0x7faaa406a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7faaa9de9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa41e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7faaa9de9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40c4fd0 con 0x7faaa40c3530 2024-03-30T01:41:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7faaa9de9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa41fa2c0 con 0x7faaa40722b0 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.910+0000 7faaa9de9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40c4cf0 con 0x7faaa4154d20 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa40722b0 0x7faaa4144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaa40c3530 0x7faaa406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaa40c3530 0x7faaa406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36994/0 (socket says 172.21.15.31:36994) 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa2ffd700 1 -- 172.21.15.31:0/2271000860 learned_addr learned my addr 172.21.15.31:0/2271000860 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa3fff700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa41e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa3fff700 1 -- 172.21.15.31:0/2271000860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa40722b0 msgr2=0x7faaa4144850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa3fff700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa40722b0 0x7faaa4144850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa3fff700 1 -- 172.21.15.31:0/2271000860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaa40c3530 msgr2=0x7faaa406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa3fff700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaa40c3530 0x7faaa406a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa3fff700 1 -- 172.21.15.31:0/2271000860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c009a70 con 0x7faaa4154d20 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.911+0000 7faaa3fff700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa41e6fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faaa406c7a0 tx=0x7faa9c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7faa8bfff700 1 -- 172.21.15.31:0/2271000860 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c004590 con 0x7faaa4154d20 2024-03-30T01:41:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7faa8bfff700 1 -- 172.21.15.31:0/2271000860 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c020070 con 0x7faaa4154d20 2024-03-30T01:41:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7fac36087700 1 -- 172.21.15.31:0/2110975258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 msgr2=0x7fac300c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7fac36087700 1 --2- 172.21.15.31:0/2110975258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 0x7fac300c2750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fac300a90f0 tx=0x7fac20009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7fac36087700 1 -- 172.21.15.31:0/2110975258 shutdown_connections 2024-03-30T01:41:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7fac36087700 1 --2- 172.21.15.31:0/2110975258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac3006dac0 0x7fac3006df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7fac36087700 1 --2- 172.21.15.31:0/2110975258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 0x7fac300c2750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7fac36087700 1 --2- 172.21.15.31:0/2110975258 >> v2:172.21.15.31:3300/0 conn(0x7fac300c0f20 0x7fac300a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7fac36087700 1 -- 172.21.15.31:0/2110975258 >> 172.21.15.31:0/2110975258 conn(0x7fac300a82d0 msgr2=0x7fac300a86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7fac36087700 1 -- 172.21.15.31:0/2110975258 shutdown_connections 2024-03-30T01:41:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7fac36087700 1 -- 172.21.15.31:0/2110975258 wait complete. 2024-03-30T01:41:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.914+0000 7fac36087700 1 Processor -- start 2024-03-30T01:41:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa41f9c70 con 0x7faaa4154d20 2024-03-30T01:41:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7faa8bfff700 1 -- 172.21.15.31:0/2271000860 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c004950 con 0x7faaa4154d20 2024-03-30T01:41:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.912+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa4144d90 con 0x7faaa4154d20 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.913+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa41f9ad0 con 0x7faaa4154d20 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.915+0000 7faa8bfff700 1 -- 172.21.15.31:0/2271000860 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faa9c031480 con 0x7faaa4154d20 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.915+0000 7fac36087700 1 -- start start 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7fac36087700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac3006dac0 0x7fac301f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7fac36087700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 0x7fac301f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7fac36087700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac300c0f20 0x7fac301f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7fac36087700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300c5240 con 0x7fac3006dac0 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7fac36087700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300aa1c0 con 0x7fac300a5920 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7fac36087700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac301fa5a0 con 0x7fac300c0f20 2024-03-30T01:41:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7f6c02d9d700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 0x7f6c0414d060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8005930 tx=0x7f6bf800cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7f6bf2ffd700 1 -- 172.21.15.31:0/2380808182 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8005d90 con 0x7f6c0414e2c0 2024-03-30T01:41:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7f6bf2ffd700 1 -- 172.21.15.31:0/2380808182 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf8007070 con 0x7f6c0414e2c0 2024-03-30T01:41:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7f6bf2ffd700 1 -- 172.21.15.31:0/2380808182 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8007420 con 0x7f6c0414e2c0 2024-03-30T01:41:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.918+0000 7f7a3db92700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a3806b400 0x7f7a380c0300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c750 tx=0x7f7a2800ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7fac2effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 0x7fac301f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7fac2effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 0x7fac301f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:39152/0 (socket says 172.21.15.31:39152) 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7fac2effd700 1 -- 172.21.15.31:0/3029627354 learned_addr learned my addr 172.21.15.31:0/3029627354 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac2ffff700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac300c0f20 0x7fac301f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac2effd700 1 -- 172.21.15.31:0/3029627354 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac300c0f20 msgr2=0x7fac301f38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac2effd700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac300c0f20 0x7fac301f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac2effd700 1 -- 172.21.15.31:0/3029627354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac3006dac0 msgr2=0x7fac301f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac2effd700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac3006dac0 0x7fac301f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac2effd700 1 -- 172.21.15.31:0/3029627354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20009550 con 0x7fac300a5920 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac2effd700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 0x7fac301f3380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac2400e980 tx=0x7fac2400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac17fff700 1 -- 172.21.15.31:0/3029627354 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2400c850 con 0x7fac300a5920 2024-03-30T01:41:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac17fff700 1 -- 172.21.15.31:0/3029627354 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2400c9b0 con 0x7fac300a5920 2024-03-30T01:41:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7fac17fff700 1 -- 172.21.15.31:0/3029627354 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2400cd40 con 0x7fac300a5920 2024-03-30T01:41:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c041f6940 con 0x7f6c0414e2c0 2024-03-30T01:41:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.917+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c041f6be0 con 0x7f6c0414e2c0 2024-03-30T01:41:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.918+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac301f7070 con 0x7fac300a5920 2024-03-30T01:41:23.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.919+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac300a7e10 con 0x7fac300a5920 2024-03-30T01:41:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.919+0000 7f6bf2ffd700 1 -- 172.21.15.31:0/2380808182 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6bf8007580 con 0x7f6c0414e2c0 2024-03-30T01:41:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.919+0000 7f6bf2ffd700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6be406ad60 0x7f6be406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.920+0000 7f6bf2ffd700 1 -- 172.21.15.31:0/2380808182 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6bf80890f0 con 0x7f6c0414e2c0 2024-03-30T01:41:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.921+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c040c1ad0 con 0x7f6c0414e2c0 2024-03-30T01:41:23.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.920+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac300a9fb0 con 0x7fac300a5920 2024-03-30T01:41:23.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.922+0000 7f6c0259c700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6be406ad60 0x7f6be406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.923+0000 7f6c0259c700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6be406ad60 0x7f6be406d220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c040c07b0 tx=0x7f6bf4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.919+0000 7f7a2e7fc700 1 -- 172.21.15.31:0/115141860 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28004c80 con 0x7f7a3806b400 2024-03-30T01:41:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.919+0000 7f7a2e7fc700 1 -- 172.21.15.31:0/115141860 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a28004de0 con 0x7f7a3806b400 2024-03-30T01:41:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.919+0000 7f7a2e7fc700 1 -- 172.21.15.31:0/115141860 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280072d0 con 0x7f7a3806b400 2024-03-30T01:41:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.920+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3814bbd0 con 0x7f7a3806b400 2024-03-30T01:41:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.920+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a380ad2d0 con 0x7f7a3806b400 2024-03-30T01:41:23.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.922+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3804ea80 con 0x7f7a3806b400 2024-03-30T01:41:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7faa8bfff700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faa8c06ae20 0x7faa8c06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.916+0000 7faa8bfff700 1 -- 172.21.15.31:0/2271000860 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faa9c022070 con 0x7faaa4154d20 2024-03-30T01:41:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.921+0000 7faa8bfff700 1 -- 172.21.15.31:0/2271000860 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7faa9c056300 con 0x7faaa4154d20 2024-03-30T01:41:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.922+0000 7faaa37fe700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faa8c06ae20 0x7faa8c06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.923+0000 7faaa37fe700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faa8c06ae20 0x7faa8c06d2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faa9400b240 tx=0x7faa9400b4d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.927+0000 7f7a2e7fc700 1 -- 172.21.15.31:0/115141860 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a28007430 con 0x7f7a3806b400 2024-03-30T01:41:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.927+0000 7f6bf2ffd700 1 -- 172.21.15.31:0/2380808182 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6c040c1ad0 con 0x7f6c0414e2c0 2024-03-30T01:41:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.925+0000 7fac17fff700 1 -- 172.21.15.31:0/3029627354 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fac24013470 con 0x7fac300a5920 2024-03-30T01:41:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.926+0000 7fac17fff700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fac1c06ac90 0x7fac1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.926+0000 7fac17fff700 1 -- 172.21.15.31:0/3029627354 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fac24089680 con 0x7fac300a5920 2024-03-30T01:41:23.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.928+0000 7f7a2e7fc700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7a1c06ace0 0x7f7a1c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.928+0000 7f7a2e7fc700 1 -- 172.21.15.31:0/115141860 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7a28089080 con 0x7f7a3806b400 2024-03-30T01:41:23.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.928+0000 7f7a3d391700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7a1c06ace0 0x7f7a1c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.929+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fac1c06ac90 0x7fac1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.929+0000 7f7a3d391700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7a1c06ace0 0x7f7a1c06d1a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a381e7790 tx=0x7f7a34009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.929+0000 7fac2f7fe700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fac1c06ac90 0x7fac1c06d150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac20009fd0 tx=0x7fac20010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.931+0000 7fac17fff700 1 -- 172.21.15.31:0/3029627354 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fac300a9fb0 con 0x7fac300a5920 2024-03-30T01:41:23.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.932+0000 7f7a2e7fc700 1 -- 172.21.15.31:0/115141860 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7a280bc580 con 0x7f7a3806b400 2024-03-30T01:41:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.944+0000 7fd21a088700 1 Processor -- start 2024-03-30T01:41:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.944+0000 7fd21a088700 1 -- start start 2024-03-30T01:41:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.945+0000 7fd21a088700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2141563b0 0x7fd2140c1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.945+0000 7fd21a088700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd21406db70 0x7fd2140c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.945+0000 7fd21a088700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2140c0ff0 con 0x7fd2140706c0 2024-03-30T01:41:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.945+0000 7fd21a088700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21406cfc0 con 0x7fd2141563b0 2024-03-30T01:41:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.945+0000 7fd21a088700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21406d2a0 con 0x7fd21406db70 2024-03-30T01:41:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.947+0000 7fd213fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd21406db70 0x7fd2140c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.947+0000 7fd2137fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd2140706c0 0x7fd214155ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:56582/0 (socket says 172.21.15.31:56582) 2024-03-30T01:41:23.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.947+0000 7fd2137fe700 1 -- 172.21.15.31:0/3791132579 learned_addr learned my addr 172.21.15.31:0/3791132579 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.951+0000 7fd2127fc700 1 -- 172.21.15.31:0/3791132579 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726885220 0 0) 0x7fd2140c0ff0 con 0x7fd2140706c0 2024-03-30T01:41:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.951+0000 7fd2127fc700 1 -- 172.21.15.31:0/3791132579 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1fc0035b0 con 0x7fd2140706c0 2024-03-30T01:41:23.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.951+0000 7fd212ffd700 1 --2- 172.21.15.31:0/3791132579 >> v2:172.21.15.31:3300/0 conn(0x7fd2141563b0 0x7fd2140c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.952+0000 7fd212ffd700 1 -- 172.21.15.31:0/3791132579 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd21406db70 msgr2=0x7fd2140c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.952+0000 7fd212ffd700 1 --2- 172.21.15.31:0/3791132579 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd21406db70 0x7fd2140c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.952+0000 7fd212ffd700 1 -- 172.21.15.31:0/3791132579 >> v1:172.21.15.59:6789/0 conn(0x7fd2140706c0 legacy=0x7fd214155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.952+0000 7fd212ffd700 1 -- 172.21.15.31:0/3791132579 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2140c09a0 con 0x7fd2141563b0 2024-03-30T01:41:23.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.952+0000 7fd212ffd700 1 --2- 172.21.15.31:0/3791132579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2141563b0 0x7fd2140c1480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd208009840 tx=0x7fd208009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b73cd27517f75c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.953+0000 7fd2127fc700 1 -- 172.21.15.31:0/3791132579 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208011830 con 0x7fd2141563b0 2024-03-30T01:41:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.953+0000 7fd2127fc700 1 -- 172.21.15.31:0/3791132579 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208011990 con 0x7fd2141563b0 2024-03-30T01:41:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.953+0000 7fd2127fc700 1 -- 172.21.15.31:0/3791132579 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208011d20 con 0x7fd2141563b0 2024-03-30T01:41:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.954+0000 7fd21a088700 1 -- 172.21.15.31:0/3791132579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2141563b0 msgr2=0x7fd2140c1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.954+0000 7fd21a088700 1 --2- 172.21.15.31:0/3791132579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2141563b0 0x7fd2140c1480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd208009840 tx=0x7fd208009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.955+0000 7fd21a088700 1 -- 172.21.15.31:0/3791132579 shutdown_connections 2024-03-30T01:41:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.955+0000 7fd21a088700 1 --2- 172.21.15.31:0/3791132579 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd21406db70 0x7fd2140c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.955+0000 7fd21a088700 1 --2- 172.21.15.31:0/3791132579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2141563b0 0x7fd2140c1480 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.955+0000 7fd21a088700 1 -- 172.21.15.31:0/3791132579 >> 172.21.15.31:0/3791132579 conn(0x7fd21406aaf0 msgr2=0x7fd21406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:23.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.955+0000 7fd21a088700 1 -- 172.21.15.31:0/3791132579 shutdown_connections 2024-03-30T01:41:23.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.955+0000 7fd21a088700 1 -- 172.21.15.31:0/3791132579 wait complete. 2024-03-30T01:41:23.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.957+0000 7fd21a088700 1 Processor -- start 2024-03-30T01:41:23.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.958+0000 7fd21a088700 1 -- start start 2024-03-30T01:41:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.959+0000 7fd21a088700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21406db70 0x7fd214157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.959+0000 7fd21a088700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2141563b0 0x7fd214140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.959+0000 7fd21a088700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2140bc880 0x7fd214140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.959+0000 7fd21a088700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140aae40 con 0x7fd2140bc880 2024-03-30T01:41:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.959+0000 7fd21a088700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd214141290 con 0x7fd21406db70 2024-03-30T01:41:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.959+0000 7fd21a088700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140a8460 con 0x7fd2141563b0 2024-03-30T01:41:23.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.961+0000 7fd2137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21406db70 0x7fd214157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.962+0000 7fd212ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2141563b0 0x7fd214140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.962+0000 7fd2137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21406db70 0x7fd214157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:39162/0 (socket says 172.21.15.31:39162) 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.962+0000 7fd2137fe700 1 -- 172.21.15.31:0/3280761710 learned_addr learned my addr 172.21.15.31:0/3280761710 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.962+0000 7fd213fff700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2140bc880 0x7fd214140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.963+0000 7fd213fff700 1 -- 172.21.15.31:0/3280761710 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2141563b0 msgr2=0x7fd214140810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.963+0000 7fd213fff700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2141563b0 0x7fd214140810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.963+0000 7fd213fff700 1 -- 172.21.15.31:0/3280761710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21406db70 msgr2=0x7fd214157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.963+0000 7fd213fff700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21406db70 0x7fd214157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.963+0000 7fd213fff700 1 -- 172.21.15.31:0/3280761710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2080094a0 con 0x7fd2140bc880 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.963+0000 7fd213fff700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2140bc880 0x7fd214140d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00ccb0 tx=0x7fd20c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.964+0000 7fd203fff700 1 -- 172.21.15.31:0/3280761710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c003a20 con 0x7fd2140bc880 2024-03-30T01:41:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.964+0000 7fd203fff700 1 -- 172.21.15.31:0/3280761710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20c003b80 con 0x7fd2140bc880 2024-03-30T01:41:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.964+0000 7fd203fff700 1 -- 172.21.15.31:0/3280761710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c004020 con 0x7fd2140bc880 2024-03-30T01:41:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.964+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2140ab050 con 0x7fd2140bc880 2024-03-30T01:41:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.964+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2141583a0 con 0x7fd2140bc880 2024-03-30T01:41:23.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.968+0000 7fd203fff700 1 -- 172.21.15.31:0/3280761710 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd20c004180 con 0x7fd2140bc880 2024-03-30T01:41:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.969+0000 7fd203fff700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd1f806ad60 0x7fd1f806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.969+0000 7fd203fff700 1 -- 172.21.15.31:0/3280761710 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd20c012070 con 0x7fd2140bc880 2024-03-30T01:41:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.970+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f00052f0 con 0x7fd2140bc880 2024-03-30T01:41:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.970+0000 7fd2137fe700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd1f806ad60 0x7fd1f806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.971+0000 7fd2137fe700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd1f806ad60 0x7fd1f806d220 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd2040011e0 tx=0x7fd204009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:23.980+0000 7fd203fff700 1 -- 172.21.15.31:0/3280761710 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd20c054ca0 con 0x7fd2140bc880 2024-03-30T01:41:24.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.162+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6728005870 con 0x7f67481496b0 2024-03-30T01:41:24.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.163+0000 7f672f7fe700 1 -- 172.21.15.31:0/3137778424 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f673c058780 con 0x7f67481496b0 2024-03-30T01:41:24.172 INFO:teuthology.orchestra.run.smithi031.stdout:55834574859 2024-03-30T01:41:24.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.166+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f673406ac90 msgr2=0x7f673406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.166+0000 7f674d7e3700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f673406ac90 0x7f673406d150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6738000c00 tx=0x7f6738010000 comp rx=0 tx=0).stop 2024-03-30T01:41:24.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.166+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67481496b0 msgr2=0x7f67481406c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.166+0000 7f674d7e3700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67481496b0 0x7f67481406c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f673c011960 tx=0x7f673c011c70 comp rx=0 tx=0).stop 2024-03-30T01:41:24.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.167+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 shutdown_connections 2024-03-30T01:41:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.167+0000 7f674d7e3700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67480c5760 0x7f6748140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.167+0000 7f674d7e3700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f673406ac90 0x7f673406d150 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.167+0000 7f674d7e3700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674806a970 0x7f6748149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.167+0000 7f674d7e3700 1 --2- 172.21.15.31:0/3137778424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67481496b0 0x7f67481406c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.167+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 >> 172.21.15.31:0/3137778424 conn(0x7f67480c3a60 msgr2=0x7f6748147810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.167+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 shutdown_connections 2024-03-30T01:41:24.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.167+0000 7f674d7e3700 1 -- 172.21.15.31:0/3137778424 wait complete. 2024-03-30T01:41:24.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.0 2024-03-30T01:41:24.195 DEBUG:teuthology.parallel:result is None 2024-03-30T01:41:24.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.217+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5a580055a0 con 0x7f5a780ab120 2024-03-30T01:41:24.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.218+0000 7f5a757fa700 1 -- 172.21.15.31:0/2159960815 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a6803d030 con 0x7f5a780ab120 2024-03-30T01:41:24.228 INFO:teuthology.orchestra.run.smithi031.stdout:55834574859 2024-03-30T01:41:24.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.222+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a6406ad60 msgr2=0x7f5a6406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.222+0000 7f5a7efce700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a6406ad60 0x7f5a6406d220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a70004ce0 tx=0x7f5a70004c50 comp rx=0 tx=0).stop 2024-03-30T01:41:24.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.222+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a780ab120 msgr2=0x7f5a781f1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.222+0000 7f5a7efce700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a780ab120 0x7f5a781f1cb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800be00 tx=0x7f5a68004750 comp rx=0 tx=0).stop 2024-03-30T01:41:24.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.223+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 shutdown_connections 2024-03-30T01:41:24.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.223+0000 7f5a7efce700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a6406ad60 0x7f5a6406d220 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.223+0000 7f5a7efce700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a78150770 0x7f5a781f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.223+0000 7f5a7efce700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a7806c860 0x7f5a7814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.223+0000 7f5a7efce700 1 --2- 172.21.15.31:0/2159960815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a780ab120 0x7f5a781f1cb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.223+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 >> 172.21.15.31:0/2159960815 conn(0x7f5a7806aaf0 msgr2=0x7f5a780a9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.223+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 shutdown_connections 2024-03-30T01:41:24.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.223+0000 7f5a7efce700 1 -- 172.21.15.31:0/2159960815 wait complete. 2024-03-30T01:41:24.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.3 2024-03-30T01:41:24.255 DEBUG:teuthology.parallel:result is None 2024-03-30T01:41:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.282+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6c040c1ad0 con 0x7f6c0414e2c0 2024-03-30T01:41:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.283+0000 7f6bf2ffd700 1 -- 172.21.15.31:0/2380808182 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c040c1ad0 con 0x7f6c0414e2c0 2024-03-30T01:41:24.290 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2024-03-30T01:41:24.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.285+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6be406ad60 msgr2=0x7f6be406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.285+0000 7f6c08fd7700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6be406ad60 0x7f6be406d220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c040c07b0 tx=0x7f6bf4008030 comp rx=0 tx=0).stop 2024-03-30T01:41:24.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.285+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 msgr2=0x7f6c0414d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.285+0000 7f6c08fd7700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 0x7f6c0414d060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8005930 tx=0x7f6bf800cae0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.286+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 shutdown_connections 2024-03-30T01:41:24.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.286+0000 7f6c08fd7700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6be406ad60 0x7f6be406d220 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.286+0000 7f6c08fd7700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c0414e2c0 0x7f6c0414d060 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.286+0000 7f6c08fd7700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6c0406e000 0x7f6c0414bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.286+0000 7f6c08fd7700 1 --2- 172.21.15.31:0/2380808182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c040c4420 0x7f6c041f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.286+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 >> 172.21.15.31:0/2380808182 conn(0x7f6c040a82d0 msgr2=0x7f6c0406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.286+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 shutdown_connections 2024-03-30T01:41:24.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.286+0000 7f6c08fd7700 1 -- 172.21.15.31:0/2380808182 wait complete. 2024-03-30T01:41:24.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.297+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb5800055a0 con 0x7fb5a0150770 2024-03-30T01:41:24.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.298+0000 7fb587fff700 1 -- 172.21.15.31:0/134511503 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb590069c70 con 0x7fb5a0150770 2024-03-30T01:41:24.305 INFO:teuthology.orchestra.run.smithi031.stdout:55834574859 2024-03-30T01:41:24.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.4 2024-03-30T01:41:24.307 DEBUG:teuthology.parallel:result is None 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb58c06ad60 msgr2=0x7fb58c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb58c06ad60 0x7fb58c06d220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00c23b0 tx=0x7fb594013040 comp rx=0 tx=0).stop 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0150770 msgr2=0x7fb5a01f9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0150770 0x7fb5a01f9990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb590016000 tx=0x7fb590004750 comp rx=0 tx=0).stop 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 shutdown_connections 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb58c06ad60 0x7fb58c06d220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5a0150770 0x7fb5a01f9990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5a0071100 0x7fb5a014a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 --2- 172.21.15.31:0/134511503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5a00ab120 0x7fb5a01f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 >> 172.21.15.31:0/134511503 conn(0x7fb5a006aaf0 msgr2=0x7fb5a00a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 shutdown_connections 2024-03-30T01:41:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.302+0000 7fb5a5e1c700 1 -- 172.21.15.31:0/134511503 wait complete. 2024-03-30T01:41:24.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.313+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7a380c0840 con 0x7f7a3806b400 2024-03-30T01:41:24.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.313+0000 7f7a2e7fc700 1 -- 172.21.15.31:0/115141860 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a28053fe0 con 0x7f7a3806b400 2024-03-30T01:41:24.320 INFO:teuthology.orchestra.run.smithi031.stdout:55834574859 2024-03-30T01:41:24.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.316+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7a1c06ace0 msgr2=0x7f7a1c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.316+0000 7f7a3fdf6700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7a1c06ace0 0x7f7a1c06d1a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7a381e7790 tx=0x7f7a34009210 comp rx=0 tx=0).stop 2024-03-30T01:41:24.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.316+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a3806b400 msgr2=0x7f7a380c0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.316+0000 7f7a3fdf6700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a3806b400 0x7f7a380c0300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c750 tx=0x7f7a2800ca60 comp rx=0 tx=0).stop 2024-03-30T01:41:24.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.317+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 shutdown_connections 2024-03-30T01:41:24.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.317+0000 7f7a3fdf6700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7a1c06ace0 0x7f7a1c06d1a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.317+0000 7f7a3fdf6700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a380c62b0 0x7f7a3814aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.317+0000 7f7a3fdf6700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7a3806b400 0x7f7a380c0300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.317+0000 7f7a3fdf6700 1 --2- 172.21.15.31:0/115141860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a380c4f60 0x7f7a380aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.317+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 >> 172.21.15.31:0/115141860 conn(0x7f7a380ca0d0 msgr2=0x7f7a38154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.317+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 shutdown_connections 2024-03-30T01:41:24.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.317+0000 7f7a3fdf6700 1 -- 172.21.15.31:0/115141860 wait complete. 2024-03-30T01:41:24.329 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.5 2024-03-30T01:41:24.329 DEBUG:teuthology.parallel:result is None 2024-03-30T01:41:24.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.7 2024-03-30T01:41:24.338 DEBUG:teuthology.parallel:result is None 2024-03-30T01:41:24.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.346+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faaa41f9ad0 con 0x7faaa4154d20 2024-03-30T01:41:24.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.346+0000 7faa8bfff700 1 -- 172.21.15.31:0/2271000860 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faaa41f9ad0 con 0x7faaa4154d20 2024-03-30T01:41:24.353 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2024-03-30T01:41:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faa8c06ae20 msgr2=0x7faa8c06d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faa8c06ae20 0x7faa8c06d2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faa9400b240 tx=0x7faa9400b4d0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 msgr2=0x7faaa41e6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa41e6fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faaa406c7a0 tx=0x7faa9c004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 shutdown_connections 2024-03-30T01:41:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faa8c06ae20 0x7faa8c06d2e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faaa4154d20 0x7faaa41e6fc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa40722b0 0x7faaa4144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 --2- 172.21.15.31:0/2271000860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faaa40c3530 0x7faaa406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 >> 172.21.15.31:0/2271000860 conn(0x7faaa40c3a60 msgr2=0x7faaa41533e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fac300a9fb0 con 0x7fac300a5920 2024-03-30T01:41:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 shutdown_connections 2024-03-30T01:41:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7faaa9de9700 1 -- 172.21.15.31:0/2271000860 wait complete. 2024-03-30T01:41:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.349+0000 7fac17fff700 1 -- 172.21.15.31:0/3029627354 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fac300a9fb0 con 0x7fac300a5920 2024-03-30T01:41:24.360 INFO:teuthology.orchestra.run.smithi031.stdout:55834574858 2024-03-30T01:41:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fac1c06ac90 msgr2=0x7fac1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fac1c06ac90 0x7fac1c06d150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac20009fd0 tx=0x7fac20010000 comp rx=0 tx=0).stop 2024-03-30T01:41:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 msgr2=0x7fac301f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 0x7fac301f3380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac2400e980 tx=0x7fac2400ed40 comp rx=0 tx=0).stop 2024-03-30T01:41:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 shutdown_connections 2024-03-30T01:41:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac3006dac0 0x7fac301f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fac1c06ac90 0x7fac1c06d150 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac300c0f20 0x7fac301f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 --2- 172.21.15.31:0/3029627354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac300a5920 0x7fac301f3380 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 >> 172.21.15.31:0/3029627354 conn(0x7fac300a82d0 msgr2=0x7fac301559b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 shutdown_connections 2024-03-30T01:41:24.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.352+0000 7fac36087700 1 -- 172.21.15.31:0/3029627354 wait complete. 2024-03-30T01:41:24.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.1 2024-03-30T01:41:24.371 DEBUG:teuthology.parallel:result is None 2024-03-30T01:41:24.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.2 2024-03-30T01:41:24.374 DEBUG:teuthology.parallel:result is None 2024-03-30T01:41:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.389+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd1f00055a0 con 0x7fd2140bc880 2024-03-30T01:41:24.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.390+0000 7fd203fff700 1 -- 172.21.15.31:0/3280761710 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd20c0582c0 con 0x7fd2140bc880 2024-03-30T01:41:24.396 INFO:teuthology.orchestra.run.smithi031.stdout:55834574859 2024-03-30T01:41:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd1f806ad60 msgr2=0x7fd1f806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd1f806ad60 0x7fd1f806d220 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd2040011e0 tx=0x7fd204009040 comp rx=0 tx=0).stop 2024-03-30T01:41:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2140bc880 msgr2=0x7fd214140d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2140bc880 0x7fd214140d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00ccb0 tx=0x7fd20c00a570 comp rx=0 tx=0).stop 2024-03-30T01:41:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 shutdown_connections 2024-03-30T01:41:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2140bc880 0x7fd214140d50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd1f806ad60 0x7fd1f806d220 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd21406db70 0x7fd214157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 --2- 172.21.15.31:0/3280761710 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2141563b0 0x7fd214140810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 >> 172.21.15.31:0/3280761710 conn(0x7fd21406aaf0 msgr2=0x7fd2140c4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 shutdown_connections 2024-03-30T01:41:24.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.392+0000 7fd21a088700 1 -- 172.21.15.31:0/3280761710 wait complete. 2024-03-30T01:41:24.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.6 2024-03-30T01:41:24.414 DEBUG:teuthology.parallel:result is None 2024-03-30T01:41:24.414 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-30T01:41:24.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:41:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.588+0000 7f181a07f700 1 Processor -- start 2024-03-30T01:41:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.588+0000 7f181a07f700 1 -- start start 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f181a07f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f18140c3180 0x7f181406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f181a07f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f181406a700 0x7f181406f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f181a07f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18140c4710 0x7f18140c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f181a07f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18141547d0 con 0x7f18140c4710 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f181a07f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1814154a90 con 0x7f18140c3180 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f181a07f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1814154d70 con 0x7f181406a700 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f1813fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18140c4710 0x7f18140c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f1813fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18140c4710 0x7f18140c0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37010/0 (socket says 172.21.15.31:37010) 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f1813fff700 1 -- 172.21.15.31:0/4096840760 learned_addr learned my addr 172.21.15.31:0/4096840760 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:24.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f1812ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f181406a700 0x7f181406f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f18137fe700 1 --2- 172.21.15.31:0/4096840760 >> v2:172.21.15.59:3300/0 conn(0x7f18140c3180 0x7f181406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f1813fff700 1 -- 172.21.15.31:0/4096840760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f181406a700 msgr2=0x7f181406f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f1813fff700 1 --2- 172.21.15.31:0/4096840760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f181406a700 0x7f181406f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f1813fff700 1 -- 172.21.15.31:0/4096840760 >> v2:172.21.15.59:3300/0 conn(0x7f18140c3180 msgr2=0x7f181406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f1813fff700 1 --2- 172.21.15.31:0/4096840760 >> v2:172.21.15.59:3300/0 conn(0x7f18140c3180 0x7f181406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.589+0000 7f1813fff700 1 -- 172.21.15.31:0/4096840760 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814155050 con 0x7f18140c4710 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f1813fff700 1 --2- 172.21.15.31:0/4096840760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c4710 0x7f18140c0390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f18140c8000 tx=0x7f180c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b3dcf28e1153f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f18127fc700 1 -- 172.21.15.31:0/4096840760 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00e070 con 0x7f18140c4710 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f18127fc700 1 -- 172.21.15.31:0/4096840760 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c00b910 con 0x7f18140c4710 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f18127fc700 1 -- 172.21.15.31:0/4096840760 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00bcc0 con 0x7f18140c4710 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f181a07f700 1 -- 172.21.15.31:0/4096840760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c4710 msgr2=0x7f18140c0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f181a07f700 1 --2- 172.21.15.31:0/4096840760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c4710 0x7f18140c0390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f18140c8000 tx=0x7f180c00b330 comp rx=0 tx=0).stop 2024-03-30T01:41:24.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f181a07f700 1 -- 172.21.15.31:0/4096840760 shutdown_connections 2024-03-30T01:41:24.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f181a07f700 1 --2- 172.21.15.31:0/4096840760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c4710 0x7f18140c0390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f181a07f700 1 --2- 172.21.15.31:0/4096840760 >> v2:172.21.15.59:3300/0 conn(0x7f18140c3180 0x7f181406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f181a07f700 1 --2- 172.21.15.31:0/4096840760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f181406a700 0x7f181406f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f181a07f700 1 -- 172.21.15.31:0/4096840760 >> 172.21.15.31:0/4096840760 conn(0x7f18140c3a60 msgr2=0x7f18140c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f181a07f700 1 -- 172.21.15.31:0/4096840760 shutdown_connections 2024-03-30T01:41:24.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.590+0000 7f181a07f700 1 -- 172.21.15.31:0/4096840760 wait complete. 2024-03-30T01:41:24.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.592+0000 7f181a07f700 1 Processor -- start 2024-03-30T01:41:24.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.592+0000 7f181a07f700 1 -- start start 2024-03-30T01:41:24.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.593+0000 7f181a07f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f181406a700 0x7f18141ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:24.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.593+0000 7f181a07f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c3180 0x7f18141e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:24.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.593+0000 7f181a07f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18140c4710 0x7f181406de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:24.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.593+0000 7f181a07f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141edc40 con 0x7f18140c3180 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.593+0000 7f181a07f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140abee0 con 0x7f181406a700 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.593+0000 7f181a07f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18140ac1c0 con 0x7f18140c4710 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.593+0000 7f1812ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c3180 0x7f18141e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.593+0000 7f1812ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c3180 0x7f18141e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37018/0 (socket says 172.21.15.31:37018) 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.593+0000 7f1812ffd700 1 -- 172.21.15.31:0/3047544525 learned_addr learned my addr 172.21.15.31:0/3047544525 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f18137fe700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f181406a700 0x7f18141ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f1813fff700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18140c4710 0x7f181406de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f1812ffd700 1 -- 172.21.15.31:0/3047544525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18140c4710 msgr2=0x7f181406de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f1812ffd700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18140c4710 0x7f181406de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f1812ffd700 1 -- 172.21.15.31:0/3047544525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f181406a700 msgr2=0x7f18141ed700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f1812ffd700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f181406a700 0x7f18141ed700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f1812ffd700 1 -- 172.21.15.31:0/3047544525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c009b30 con 0x7f18140c3180 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f1812ffd700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c3180 0x7f18141e9810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f180800e9b0 tx=0x7f180800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f17fbfff700 1 -- 172.21.15.31:0/3047544525 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080096b0 con 0x7f18140c3180 2024-03-30T01:41:24.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f17fbfff700 1 -- 172.21.15.31:0/3047544525 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180800c7f0 con 0x7f18140c3180 2024-03-30T01:41:24.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18141ecf00 con 0x7f18140c3180 2024-03-30T01:41:24.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f17fbfff700 1 -- 172.21.15.31:0/3047544525 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180800cb80 con 0x7f18140c3180 2024-03-30T01:41:24.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.594+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18140abb20 con 0x7f18140c3180 2024-03-30T01:41:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.595+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18141ed420 con 0x7f18140c3180 2024-03-30T01:41:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.596+0000 7f17fbfff700 1 -- 172.21.15.31:0/3047544525 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f180800cce0 con 0x7f18140c3180 2024-03-30T01:41:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.596+0000 7f17fbfff700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f180006ac90 0x7f180006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.596+0000 7f17fbfff700 1 -- 172.21.15.31:0/3047544525 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f18080896e0 con 0x7f18140c3180 2024-03-30T01:41:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.596+0000 7f18137fe700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f180006ac90 0x7f180006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.597+0000 7f18137fe700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f180006ac90 0x7f180006d150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1804009b70 tx=0x7f1804008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:24.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.601+0000 7f17fbfff700 1 -- 172.21.15.31:0/3047544525 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f18141ed420 con 0x7f18140c3180 2024-03-30T01:41:24.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.810+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18141ecd80 con 0x7f180006ac90 2024-03-30T01:41:24.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.820+0000 7f17fbfff700 1 -- 172.21.15.31:0/3047544525 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345175 (secure 0 0 0) 0x7f18040117b0 con 0x7f180006ac90 2024-03-30T01:41:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:24.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f180006ac90 msgr2=0x7f180006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f180006ac90 0x7f180006d150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1804009b70 tx=0x7f1804008040 comp rx=0 tx=0).stop 2024-03-30T01:41:24.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c3180 msgr2=0x7f18141e9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:24.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c3180 0x7f18141e9810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f180800e9b0 tx=0x7f180800ed70 comp rx=0 tx=0).stop 2024-03-30T01:41:24.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 shutdown_connections 2024-03-30T01:41:24.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f180006ac90 0x7f180006d150 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18140c4710 0x7f181406de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f181406a700 0x7f18141ed700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 --2- 172.21.15.31:0/3047544525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18140c3180 0x7f18141e9810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.823+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 >> 172.21.15.31:0/3047544525 conn(0x7f18140c3a60 msgr2=0x7f1814152340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.824+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 shutdown_connections 2024-03-30T01:41:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:24.824+0000 7f181a07f700 1 -- 172.21.15.31:0/3047544525 wait complete. 2024-03-30T01:41:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:41:24.844 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-03-30T01:41:23.813518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219364,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755356,"statfs":{"total":773094113280,"available":772869484544,"internally_reserved":0,"allocated":3280896,"data_stored":1987195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011282"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965171+0000","last_change":"2024-03-30T01:40:58.441027+0000","last_active":"2024-03-30T01:41:02.965171+0000","last_peered":"2024-03-30T01:41:02.965171+0000","last_clean":"2024-03-30T01:41:02.965171+0000","last_became_active":"2024-03-30T01:40:58.440741+0000","last_became_peered":"2024-03-30T01:40:58.440741+0000","last_unstale":"2024-03-30T01:41:02.965171+0000","last_undegraded":"2024-03-30T01:41:02.965171+0000","last_fullsized":"2024-03-30T01:41:02.965171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868566+0000","last_change":"2024-03-30T01:40:58.426588+0000","last_active":"2024-03-30T01:41:02.868566+0000","last_peered":"2024-03-30T01:41:02.868566+0000","last_clean":"2024-03-30T01:41:02.868566+0000","last_became_active":"2024-03-30T01:40:58.426336+0000","last_became_peered":"2024-03-30T01:40:58.426336+0000","last_unstale":"2024-03-30T01:41:02.868566+0000","last_undegraded":"2024-03-30T01:41:02.868566+0000","last_fullsized":"2024-03-30T01:41:02.868566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869813+0000","last_change":"2024-03-30T01:40:58.434632+0000","last_active":"2024-03-30T01:41:02.869813+0000","last_peered":"2024-03-30T01:41:02.869813+0000","last_clean":"2024-03-30T01:41:02.869813+0000","last_became_active":"2024-03-30T01:40:58.434339+0000","last_became_peered":"2024-03-30T01:40:58.434339+0000","last_unstale":"2024-03-30T01:41:02.869813+0000","last_undegraded":"2024-03-30T01:41:02.869813+0000","last_fullsized":"2024-03-30T01:41:02.869813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:35.665153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868405+0000","last_change":"2024-03-30T01:40:58.428659+0000","last_active":"2024-03-30T01:41:02.868405+0000","last_peered":"2024-03-30T01:41:02.868405+0000","last_clean":"2024-03-30T01:41:02.868405+0000","last_became_active":"2024-03-30T01:40:58.426016+0000","last_became_peered":"2024-03-30T01:40:58.426016+0000","last_unstale":"2024-03-30T01:41:02.868405+0000","last_undegraded":"2024-03-30T01:41:02.868405+0000","last_fullsized":"2024-03-30T01:41:02.868405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964596+0000","last_change":"2024-03-30T01:40:58.440079+0000","last_active":"2024-03-30T01:41:02.964596+0000","last_peered":"2024-03-30T01:41:02.964596+0000","last_clean":"2024-03-30T01:41:02.964596+0000","last_became_active":"2024-03-30T01:40:58.439661+0000","last_became_peered":"2024-03-30T01:40:58.439661+0000","last_unstale":"2024-03-30T01:41:02.964596+0000","last_undegraded":"2024-03-30T01:41:02.964596+0000","last_fullsized":"2024-03-30T01:41:02.964596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868344+0000","last_change":"2024-03-30T01:40:58.426692+0000","last_active":"2024-03-30T01:41:02.868344+0000","last_peered":"2024-03-30T01:41:02.868344+0000","last_clean":"2024-03-30T01:41:02.868344+0000","last_became_active":"2024-03-30T01:40:58.426370+0000","last_became_peered":"2024-03-30T01:40:58.426370+0000","last_unstale":"2024-03-30T01:41:02.868344+0000","last_undegraded":"2024-03-30T01:41:02.868344+0000","last_fullsized":"2024-03-30T01:41:02.868344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:51.791864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964482+0000","last_change":"2024-03-30T01:40:58.440389+0000","last_active":"2024-03-30T01:41:02.964482+0000","last_peered":"2024-03-30T01:41:02.964482+0000","last_clean":"2024-03-30T01:41:02.964482+0000","last_became_active":"2024-03-30T01:40:58.439965+0000","last_became_peered":"2024-03-30T01:40:58.439965+0000","last_unstale":"2024-03-30T01:41:02.964482+0000","last_undegraded":"2024-03-30T01:41:02.964482+0000","last_fullsized":"2024-03-30T01:41:02.964482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:57.923292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868477+0000","last_change":"2024-03-30T01:40:58.429571+0000","last_active":"2024-03-30T01:41:02.868477+0000","last_peered":"2024-03-30T01:41:02.868477+0000","last_clean":"2024-03-30T01:41:02.868477+0000","last_became_active":"2024-03-30T01:40:58.429144+0000","last_became_peered":"2024-03-30T01:40:58.429144+0000","last_unstale":"2024-03-30T01:41:02.868477+0000","last_undegraded":"2024-03-30T01:41:02.868477+0000","last_fullsized":"2024-03-30T01:41:02.868477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965163+0000","last_change":"2024-03-30T01:40:58.441005+0000","last_active":"2024-03-30T01:41:02.965163+0000","last_peered":"2024-03-30T01:41:02.965163+0000","last_clean":"2024-03-30T01:41:02.965163+0000","last_became_active":"2024-03-30T01:40:58.440409+0000","last_became_peered":"2024-03-30T01:40:58.440409+0000","last_unstale":"2024-03-30T01:41:02.965163+0000","last_undegraded":"2024-03-30T01:41:02.965163+0000","last_fullsized":"2024-03-30T01:41:02.965163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:48:57.608199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872597+0000","last_change":"2024-03-30T01:40:58.429626+0000","last_active":"2024-03-30T01:41:02.872597+0000","last_peered":"2024-03-30T01:41:02.872597+0000","last_clean":"2024-03-30T01:41:02.872597+0000","last_became_active":"2024-03-30T01:40:58.428916+0000","last_became_peered":"2024-03-30T01:40:58.428916+0000","last_unstale":"2024-03-30T01:41:02.872597+0000","last_undegraded":"2024-03-30T01:41:02.872597+0000","last_fullsized":"2024-03-30T01:41:02.872597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872572+0000","last_change":"2024-03-30T01:40:58.430546+0000","last_active":"2024-03-30T01:41:02.872572+0000","last_peered":"2024-03-30T01:41:02.872572+0000","last_clean":"2024-03-30T01:41:02.872572+0000","last_became_active":"2024-03-30T01:40:58.430232+0000","last_became_peered":"2024-03-30T01:40:58.430232+0000","last_unstale":"2024-03-30T01:41:02.872572+0000","last_undegraded":"2024-03-30T01:41:02.872572+0000","last_fullsized":"2024-03-30T01:41:02.872572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864576+0000","last_change":"2024-03-30T01:40:58.422212+0000","last_active":"2024-03-30T01:41:02.864576+0000","last_peered":"2024-03-30T01:41:02.864576+0000","last_clean":"2024-03-30T01:41:02.864576+0000","last_became_active":"2024-03-30T01:40:58.421812+0000","last_became_peered":"2024-03-30T01:40:58.421812+0000","last_unstale":"2024-03-30T01:41:02.864576+0000","last_undegraded":"2024-03-30T01:41:02.864576+0000","last_fullsized":"2024-03-30T01:41:02.864576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:30.774569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868720+0000","last_change":"2024-03-30T01:40:58.418447+0000","last_active":"2024-03-30T01:41:02.868720+0000","last_peered":"2024-03-30T01:41:02.868720+0000","last_clean":"2024-03-30T01:41:02.868720+0000","last_became_active":"2024-03-30T01:40:58.418146+0000","last_became_peered":"2024-03-30T01:40:58.418146+0000","last_unstale":"2024-03-30T01:41:02.868720+0000","last_undegraded":"2024-03-30T01:41:02.868720+0000","last_fullsized":"2024-03-30T01:41:02.868720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869653+0000","last_change":"2024-03-30T01:40:58.430002+0000","last_active":"2024-03-30T01:41:02.869653+0000","last_peered":"2024-03-30T01:41:02.869653+0000","last_clean":"2024-03-30T01:41:02.869653+0000","last_became_active":"2024-03-30T01:40:58.429743+0000","last_became_peered":"2024-03-30T01:40:58.429743+0000","last_unstale":"2024-03-30T01:41:02.869653+0000","last_undegraded":"2024-03-30T01:41:02.869653+0000","last_fullsized":"2024-03-30T01:41:02.869653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:54.449729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868389+0000","last_change":"2024-03-30T01:40:58.425393+0000","last_active":"2024-03-30T01:41:02.868389+0000","last_peered":"2024-03-30T01:41:02.868389+0000","last_clean":"2024-03-30T01:41:02.868389+0000","last_became_active":"2024-03-30T01:40:58.424890+0000","last_became_peered":"2024-03-30T01:40:58.424890+0000","last_unstale":"2024-03-30T01:41:02.868389+0000","last_undegraded":"2024-03-30T01:41:02.868389+0000","last_fullsized":"2024-03-30T01:41:02.868389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868218+0000","last_change":"2024-03-30T01:40:58.426453+0000","last_active":"2024-03-30T01:41:02.868218+0000","last_peered":"2024-03-30T01:41:02.868218+0000","last_clean":"2024-03-30T01:41:02.868218+0000","last_became_active":"2024-03-30T01:40:58.425360+0000","last_became_peered":"2024-03-30T01:40:58.425360+0000","last_unstale":"2024-03-30T01:41:02.868218+0000","last_undegraded":"2024-03-30T01:41:02.868218+0000","last_fullsized":"2024-03-30T01:41:02.868218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:28.675589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868517+0000","last_change":"2024-03-30T01:40:58.433354+0000","last_active":"2024-03-30T01:41:02.868517+0000","last_peered":"2024-03-30T01:41:02.868517+0000","last_clean":"2024-03-30T01:41:02.868517+0000","last_became_active":"2024-03-30T01:40:58.433034+0000","last_became_peered":"2024-03-30T01:40:58.433034+0000","last_unstale":"2024-03-30T01:41:02.868517+0000","last_undegraded":"2024-03-30T01:41:02.868517+0000","last_fullsized":"2024-03-30T01:41:02.868517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:30.394391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868781+0000","last_change":"2024-03-30T01:40:58.433603+0000","last_active":"2024-03-30T01:41:02.868781+0000","last_peered":"2024-03-30T01:41:02.868781+0000","last_clean":"2024-03-30T01:41:02.868781+0000","last_became_active":"2024-03-30T01:40:58.433303+0000","last_became_peered":"2024-03-30T01:40:58.433303+0000","last_unstale":"2024-03-30T01:41:02.868781+0000","last_undegraded":"2024-03-30T01:41:02.868781+0000","last_fullsized":"2024-03-30T01:41:02.868781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:05:33.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965044+0000","last_change":"2024-03-30T01:40:58.436807+0000","last_active":"2024-03-30T01:41:02.965044+0000","last_peered":"2024-03-30T01:41:02.965044+0000","last_clean":"2024-03-30T01:41:02.965044+0000","last_became_active":"2024-03-30T01:40:58.436528+0000","last_became_peered":"2024-03-30T01:40:58.436528+0000","last_unstale":"2024-03-30T01:41:02.965044+0000","last_undegraded":"2024-03-30T01:41:02.965044+0000","last_fullsized":"2024-03-30T01:41:02.965044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:21.062997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972825+0000","last_change":"2024-03-30T01:41:02.970253+0000","last_active":"2024-03-30T01:41:02.972825+0000","last_peered":"2024-03-30T01:41:02.972825+0000","last_clean":"2024-03-30T01:41:02.972825+0000","last_became_active":"2024-03-30T01:41:02.969854+0000","last_became_peered":"2024-03-30T01:41:02.969854+0000","last_unstale":"2024-03-30T01:41:02.972825+0000","last_undegraded":"2024-03-30T01:41:02.972825+0000","last_fullsized":"2024-03-30T01:41:02.972825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:57:18.590955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.786878+0000","last_change":"2024-03-30T01:40:57.963998+0000","last_active":"2024-03-30T01:41:02.786878+0000","last_peered":"2024-03-30T01:41:02.786878+0000","last_clean":"2024-03-30T01:41:02.786878+0000","last_became_active":"2024-03-30T01:40:57.963535+0000","last_became_peered":"2024-03-30T01:40:57.963535+0000","last_unstale":"2024-03-30T01:41:02.786878+0000","last_undegraded":"2024-03-30T01:41:02.786878+0000","last_fullsized":"2024-03-30T01:41:02.786878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:40:49.324287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868907+0000","last_change":"2024-03-30T01:40:58.418336+0000","last_active":"2024-03-30T01:41:02.868907+0000","last_peered":"2024-03-30T01:41:02.868907+0000","last_clean":"2024-03-30T01:41:02.868907+0000","last_became_active":"2024-03-30T01:40:58.418057+0000","last_became_peered":"2024-03-30T01:40:58.418057+0000","last_unstale":"2024-03-30T01:41:02.868907+0000","last_undegraded":"2024-03-30T01:41:02.868907+0000","last_fullsized":"2024-03-30T01:41:02.868907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:52.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864399+0000","last_change":"2024-03-30T01:41:02.791579+0000","last_active":"2024-03-30T01:41:02.864399+0000","last_peered":"2024-03-30T01:41:02.864399+0000","last_clean":"2024-03-30T01:41:02.864399+0000","last_became_active":"2024-03-30T01:41:02.791102+0000","last_became_peered":"2024-03-30T01:41:02.791102+0000","last_unstale":"2024-03-30T01:41:02.864399+0000","last_undegraded":"2024-03-30T01:41:02.864399+0000","last_fullsized":"2024-03-30T01:41:02.864399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:34.215813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.776056+0000","last_change":"2024-03-30T01:40:57.963840+0000","last_active":"2024-03-30T01:41:02.776056+0000","last_peered":"2024-03-30T01:41:02.776056+0000","last_clean":"2024-03-30T01:41:02.776056+0000","last_became_active":"2024-03-30T01:40:57.962842+0000","last_became_peered":"2024-03-30T01:40:57.962842+0000","last_unstale":"2024-03-30T01:41:02.776056+0000","last_undegraded":"2024-03-30T01:41:02.776056+0000","last_fullsized":"2024-03-30T01:41:02.776056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869931+0000","last_change":"2024-03-30T01:40:58.435910+0000","last_active":"2024-03-30T01:41:02.869931+0000","last_peered":"2024-03-30T01:41:02.869931+0000","last_clean":"2024-03-30T01:41:02.869931+0000","last_became_active":"2024-03-30T01:40:58.435402+0000","last_became_peered":"2024-03-30T01:40:58.435402+0000","last_unstale":"2024-03-30T01:41:02.869931+0000","last_undegraded":"2024-03-30T01:41:02.869931+0000","last_fullsized":"2024-03-30T01:41:02.869931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864534+0000","last_change":"2024-03-30T01:41:02.789877+0000","last_active":"2024-03-30T01:41:02.864534+0000","last_peered":"2024-03-30T01:41:02.864534+0000","last_clean":"2024-03-30T01:41:02.864534+0000","last_became_active":"2024-03-30T01:41:02.789440+0000","last_became_peered":"2024-03-30T01:41:02.789440+0000","last_unstale":"2024-03-30T01:41:02.864534+0000","last_undegraded":"2024-03-30T01:41:02.864534+0000","last_fullsized":"2024-03-30T01:41:02.864534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:23.954119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964766+0000","last_change":"2024-03-30T01:40:57.968973+0000","last_active":"2024-03-30T01:41:02.964766+0000","last_peered":"2024-03-30T01:41:02.964766+0000","last_clean":"2024-03-30T01:41:02.964766+0000","last_became_active":"2024-03-30T01:40:57.968415+0000","last_became_peered":"2024-03-30T01:40:57.968415+0000","last_unstale":"2024-03-30T01:41:02.964766+0000","last_undegraded":"2024-03-30T01:41:02.964766+0000","last_fullsized":"2024-03-30T01:41:02.964766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868378+0000","last_change":"2024-03-30T01:40:58.431466+0000","last_active":"2024-03-30T01:41:02.868378+0000","last_peered":"2024-03-30T01:41:02.868378+0000","last_clean":"2024-03-30T01:41:02.868378+0000","last_became_active":"2024-03-30T01:40:58.431122+0000","last_became_peered":"2024-03-30T01:40:58.431122+0000","last_unstale":"2024-03-30T01:41:02.868378+0000","last_undegraded":"2024-03-30T01:41:02.868378+0000","last_fullsized":"2024-03-30T01:41:02.868378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:21.062997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864511+0000","last_change":"2024-03-30T01:41:02.789299+0000","last_active":"2024-03-30T01:41:02.864511+0000","last_peered":"2024-03-30T01:41:02.864511+0000","last_clean":"2024-03-30T01:41:02.864511+0000","last_became_active":"2024-03-30T01:41:02.788769+0000","last_became_peered":"2024-03-30T01:41:02.788769+0000","last_unstale":"2024-03-30T01:41:02.864511+0000","last_undegraded":"2024-03-30T01:41:02.864511+0000","last_fullsized":"2024-03-30T01:41:02.864511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:18.933245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964688+0000","last_change":"2024-03-30T01:40:57.970239+0000","last_active":"2024-03-30T01:41:02.964688+0000","last_peered":"2024-03-30T01:41:02.964688+0000","last_clean":"2024-03-30T01:41:02.964688+0000","last_became_active":"2024-03-30T01:40:57.969868+0000","last_became_peered":"2024-03-30T01:40:57.969868+0000","last_unstale":"2024-03-30T01:41:02.964688+0000","last_undegraded":"2024-03-30T01:41:02.964688+0000","last_fullsized":"2024-03-30T01:41:02.964688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:34.662924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868462+0000","last_change":"2024-03-30T01:40:58.431864+0000","last_active":"2024-03-30T01:41:02.868462+0000","last_peered":"2024-03-30T01:41:02.868462+0000","last_clean":"2024-03-30T01:41:02.868462+0000","last_became_active":"2024-03-30T01:40:58.431101+0000","last_became_peered":"2024-03-30T01:40:58.431101+0000","last_unstale":"2024-03-30T01:41:02.868462+0000","last_undegraded":"2024-03-30T01:41:02.868462+0000","last_fullsized":"2024-03-30T01:41:02.868462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:43.041425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868886+0000","last_change":"2024-03-30T01:41:02.797411+0000","last_active":"2024-03-30T01:41:02.868886+0000","last_peered":"2024-03-30T01:41:02.868886+0000","last_clean":"2024-03-30T01:41:02.868886+0000","last_became_active":"2024-03-30T01:41:02.797018+0000","last_became_peered":"2024-03-30T01:41:02.797018+0000","last_unstale":"2024-03-30T01:41:02.868886+0000","last_undegraded":"2024-03-30T01:41:02.868886+0000","last_fullsized":"2024-03-30T01:41:02.868886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:18:03.263600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.775328+0000","last_change":"2024-03-30T01:40:57.963372+0000","last_active":"2024-03-30T01:41:02.775328+0000","last_peered":"2024-03-30T01:41:02.775328+0000","last_clean":"2024-03-30T01:41:02.775328+0000","last_became_active":"2024-03-30T01:40:57.963033+0000","last_became_peered":"2024-03-30T01:40:57.963033+0000","last_unstale":"2024-03-30T01:41:02.775328+0000","last_undegraded":"2024-03-30T01:41:02.775328+0000","last_fullsized":"2024-03-30T01:41:02.775328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864523+0000","last_change":"2024-03-30T01:40:58.427338+0000","last_active":"2024-03-30T01:41:02.864523+0000","last_peered":"2024-03-30T01:41:02.864523+0000","last_clean":"2024-03-30T01:41:02.864523+0000","last_became_active":"2024-03-30T01:40:58.427023+0000","last_became_peered":"2024-03-30T01:40:58.427023+0000","last_unstale":"2024-03-30T01:41:02.864523+0000","last_undegraded":"2024-03-30T01:41:02.864523+0000","last_fullsized":"2024-03-30T01:41:02.864523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:35.665153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868735+0000","last_change":"2024-03-30T01:41:02.801387+0000","last_active":"2024-03-30T01:41:02.868735+0000","last_peered":"2024-03-30T01:41:02.868735+0000","last_clean":"2024-03-30T01:41:02.868735+0000","last_became_active":"2024-03-30T01:41:02.800907+0000","last_became_peered":"2024-03-30T01:41:02.800907+0000","last_unstale":"2024-03-30T01:41:02.868735+0000","last_undegraded":"2024-03-30T01:41:02.868735+0000","last_fullsized":"2024-03-30T01:41:02.868735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:43.218702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770418+0000","last_change":"2024-03-30T01:40:57.965707+0000","last_active":"2024-03-30T01:41:02.770418+0000","last_peered":"2024-03-30T01:41:02.770418+0000","last_clean":"2024-03-30T01:41:02.770418+0000","last_became_active":"2024-03-30T01:40:57.965371+0000","last_became_peered":"2024-03-30T01:40:57.965371+0000","last_unstale":"2024-03-30T01:41:02.770418+0000","last_undegraded":"2024-03-30T01:41:02.770418+0000","last_fullsized":"2024-03-30T01:41:02.770418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869759+0000","last_change":"2024-03-30T01:40:58.432647+0000","last_active":"2024-03-30T01:41:02.869759+0000","last_peered":"2024-03-30T01:41:02.869759+0000","last_clean":"2024-03-30T01:41:02.869759+0000","last_became_active":"2024-03-30T01:40:58.432292+0000","last_became_peered":"2024-03-30T01:40:58.432292+0000","last_unstale":"2024-03-30T01:41:02.869759+0000","last_undegraded":"2024-03-30T01:41:02.869759+0000","last_fullsized":"2024-03-30T01:41:02.869759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872608+0000","last_change":"2024-03-30T01:41:02.798539+0000","last_active":"2024-03-30T01:41:02.872608+0000","last_peered":"2024-03-30T01:41:02.872608+0000","last_clean":"2024-03-30T01:41:02.872608+0000","last_became_active":"2024-03-30T01:41:02.798146+0000","last_became_peered":"2024-03-30T01:41:02.798146+0000","last_unstale":"2024-03-30T01:41:02.872608+0000","last_undegraded":"2024-03-30T01:41:02.872608+0000","last_fullsized":"2024-03-30T01:41:02.872608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:18.933245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.773948+0000","last_change":"2024-03-30T01:40:57.959181+0000","last_active":"2024-03-30T01:41:02.773948+0000","last_peered":"2024-03-30T01:41:02.773948+0000","last_clean":"2024-03-30T01:41:02.773948+0000","last_became_active":"2024-03-30T01:40:57.958838+0000","last_became_peered":"2024-03-30T01:40:57.958838+0000","last_unstale":"2024-03-30T01:41:02.773948+0000","last_undegraded":"2024-03-30T01:41:02.773948+0000","last_fullsized":"2024-03-30T01:41:02.773948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:04.688535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868351+0000","last_change":"2024-03-30T01:40:58.429817+0000","last_active":"2024-03-30T01:41:02.868351+0000","last_peered":"2024-03-30T01:41:02.868351+0000","last_clean":"2024-03-30T01:41:02.868351+0000","last_became_active":"2024-03-30T01:40:58.429505+0000","last_became_peered":"2024-03-30T01:40:58.429505+0000","last_unstale":"2024-03-30T01:41:02.868351+0000","last_undegraded":"2024-03-30T01:41:02.868351+0000","last_fullsized":"2024-03-30T01:41:02.868351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:30:35.170155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.970928+0000","last_change":"2024-03-30T01:41:02.967897+0000","last_active":"2024-03-30T01:41:02.970928+0000","last_peered":"2024-03-30T01:41:02.970928+0000","last_clean":"2024-03-30T01:41:02.970928+0000","last_became_active":"2024-03-30T01:41:02.967524+0000","last_became_peered":"2024-03-30T01:41:02.967524+0000","last_unstale":"2024-03-30T01:41:02.970928+0000","last_undegraded":"2024-03-30T01:41:02.970928+0000","last_fullsized":"2024-03-30T01:41:02.970928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:58.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.778673+0000","last_change":"2024-03-30T01:40:57.964382+0000","last_active":"2024-03-30T01:41:02.778673+0000","last_peered":"2024-03-30T01:41:02.778673+0000","last_clean":"2024-03-30T01:41:02.778673+0000","last_became_active":"2024-03-30T01:40:57.963879+0000","last_became_peered":"2024-03-30T01:40:57.963879+0000","last_unstale":"2024-03-30T01:41:02.778673+0000","last_undegraded":"2024-03-30T01:41:02.778673+0000","last_fullsized":"2024-03-30T01:41:02.778673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868435+0000","last_change":"2024-03-30T01:40:58.425195+0000","last_active":"2024-03-30T01:41:02.868435+0000","last_peered":"2024-03-30T01:41:02.868435+0000","last_clean":"2024-03-30T01:41:02.868435+0000","last_became_active":"2024-03-30T01:40:58.424555+0000","last_became_peered":"2024-03-30T01:40:58.424555+0000","last_unstale":"2024-03-30T01:41:02.868435+0000","last_undegraded":"2024-03-30T01:41:02.868435+0000","last_fullsized":"2024-03-30T01:41:02.868435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972778+0000","last_change":"2024-03-30T01:41:02.968243+0000","last_active":"2024-03-30T01:41:02.972778+0000","last_peered":"2024-03-30T01:41:02.972778+0000","last_clean":"2024-03-30T01:41:02.972778+0000","last_became_active":"2024-03-30T01:41:02.967682+0000","last_became_peered":"2024-03-30T01:41:02.967682+0000","last_unstale":"2024-03-30T01:41:02.972778+0000","last_undegraded":"2024-03-30T01:41:02.972778+0000","last_fullsized":"2024-03-30T01:41:02.972778+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:40.634879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770699+0000","last_change":"2024-03-30T01:40:57.959485+0000","last_active":"2024-03-30T01:41:02.770699+0000","last_peered":"2024-03-30T01:41:02.770699+0000","last_clean":"2024-03-30T01:41:02.770699+0000","last_became_active":"2024-03-30T01:40:57.959180+0000","last_became_peered":"2024-03-30T01:40:57.959180+0000","last_unstale":"2024-03-30T01:41:02.770699+0000","last_undegraded":"2024-03-30T01:41:02.770699+0000","last_fullsized":"2024-03-30T01:41:02.770699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868505+0000","last_change":"2024-03-30T01:40:58.432524+0000","last_active":"2024-03-30T01:41:02.868505+0000","last_peered":"2024-03-30T01:41:02.868505+0000","last_clean":"2024-03-30T01:41:02.868505+0000","last_became_active":"2024-03-30T01:40:58.432225+0000","last_became_peered":"2024-03-30T01:40:58.432225+0000","last_unstale":"2024-03-30T01:41:02.868505+0000","last_undegraded":"2024-03-30T01:41:02.868505+0000","last_fullsized":"2024-03-30T01:41:02.868505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:30:35.170155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868329+0000","last_change":"2024-03-30T01:41:02.796380+0000","last_active":"2024-03-30T01:41:02.868329+0000","last_peered":"2024-03-30T01:41:02.868329+0000","last_clean":"2024-03-30T01:41:02.868329+0000","last_became_active":"2024-03-30T01:41:02.795984+0000","last_became_peered":"2024-03-30T01:41:02.795984+0000","last_unstale":"2024-03-30T01:41:02.868329+0000","last_undegraded":"2024-03-30T01:41:02.868329+0000","last_fullsized":"2024-03-30T01:41:02.868329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:13.399650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.781391+0000","last_change":"2024-03-30T01:40:57.958458+0000","last_active":"2024-03-30T01:41:02.781391+0000","last_peered":"2024-03-30T01:41:02.781391+0000","last_clean":"2024-03-30T01:41:02.781391+0000","last_became_active":"2024-03-30T01:40:57.957887+0000","last_became_peered":"2024-03-30T01:40:57.957887+0000","last_unstale":"2024-03-30T01:41:02.781391+0000","last_undegraded":"2024-03-30T01:41:02.781391+0000","last_fullsized":"2024-03-30T01:41:02.781391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869948+0000","last_change":"2024-03-30T01:40:58.436050+0000","last_active":"2024-03-30T01:41:02.869948+0000","last_peered":"2024-03-30T01:41:02.869948+0000","last_clean":"2024-03-30T01:41:02.869948+0000","last_became_active":"2024-03-30T01:40:58.435014+0000","last_became_peered":"2024-03-30T01:40:58.435014+0000","last_unstale":"2024-03-30T01:41:02.869948+0000","last_undegraded":"2024-03-30T01:41:02.869948+0000","last_fullsized":"2024-03-30T01:41:02.869948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:52.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864562+0000","last_change":"2024-03-30T01:41:02.784245+0000","last_active":"2024-03-30T01:41:02.864562+0000","last_peered":"2024-03-30T01:41:02.864562+0000","last_clean":"2024-03-30T01:41:02.864562+0000","last_became_active":"2024-03-30T01:41:02.783538+0000","last_became_peered":"2024-03-30T01:41:02.783538+0000","last_unstale":"2024-03-30T01:41:02.864562+0000","last_undegraded":"2024-03-30T01:41:02.864562+0000","last_fullsized":"2024-03-30T01:41:02.864562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:58.270483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.777921+0000","last_change":"2024-03-30T01:40:57.967602+0000","last_active":"2024-03-30T01:41:02.777921+0000","last_peered":"2024-03-30T01:41:02.777921+0000","last_clean":"2024-03-30T01:41:02.777921+0000","last_became_active":"2024-03-30T01:40:57.967239+0000","last_became_peered":"2024-03-30T01:40:57.967239+0000","last_unstale":"2024-03-30T01:41:02.777921+0000","last_undegraded":"2024-03-30T01:41:02.777921+0000","last_fullsized":"2024-03-30T01:41:02.777921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:36.274096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864600+0000","last_change":"2024-03-30T01:40:58.430901+0000","last_active":"2024-03-30T01:41:02.864600+0000","last_peered":"2024-03-30T01:41:02.864600+0000","last_clean":"2024-03-30T01:41:02.864600+0000","last_became_active":"2024-03-30T01:40:58.430595+0000","last_became_peered":"2024-03-30T01:40:58.430595+0000","last_unstale":"2024-03-30T01:41:02.864600+0000","last_undegraded":"2024-03-30T01:41:02.864600+0000","last_fullsized":"2024-03-30T01:41:02.864600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:37.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868468+0000","last_change":"2024-03-30T01:41:02.795440+0000","last_active":"2024-03-30T01:41:02.868468+0000","last_peered":"2024-03-30T01:41:02.868468+0000","last_clean":"2024-03-30T01:41:02.868468+0000","last_became_active":"2024-03-30T01:41:02.795052+0000","last_became_peered":"2024-03-30T01:41:02.795052+0000","last_unstale":"2024-03-30T01:41:02.868468+0000","last_undegraded":"2024-03-30T01:41:02.868468+0000","last_fullsized":"2024-03-30T01:41:02.868468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:23.954119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T01:41:13.464226+0000","last_change":"2024-03-30T01:40:57.966563+0000","last_active":"2024-03-30T01:41:13.464226+0000","last_peered":"2024-03-30T01:41:13.464226+0000","last_clean":"2024-03-30T01:41:13.464226+0000","last_became_active":"2024-03-30T01:40:57.966235+0000","last_became_peered":"2024-03-30T01:40:57.966235+0000","last_unstale":"2024-03-30T01:41:13.464226+0000","last_undegraded":"2024-03-30T01:41:13.464226+0000","last_fullsized":"2024-03-30T01:41:13.464226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964895+0000","last_change":"2024-03-30T01:40:58.436063+0000","last_active":"2024-03-30T01:41:02.964895+0000","last_peered":"2024-03-30T01:41:02.964895+0000","last_clean":"2024-03-30T01:41:02.964895+0000","last_became_active":"2024-03-30T01:40:58.435716+0000","last_became_peered":"2024-03-30T01:40:58.435716+0000","last_unstale":"2024-03-30T01:41:02.964895+0000","last_undegraded":"2024-03-30T01:41:02.964895+0000","last_fullsized":"2024-03-30T01:41:02.964895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:30.394391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.973943+0000","last_change":"2024-03-30T01:41:02.969718+0000","last_active":"2024-03-30T01:41:02.973943+0000","last_peered":"2024-03-30T01:41:02.973943+0000","last_clean":"2024-03-30T01:41:02.973943+0000","last_became_active":"2024-03-30T01:41:02.969246+0000","last_became_peered":"2024-03-30T01:41:02.969246+0000","last_unstale":"2024-03-30T01:41:02.973943+0000","last_undegraded":"2024-03-30T01:41:02.973943+0000","last_fullsized":"2024-03-30T01:41:02.973943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:34.173820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.786047+0000","last_change":"2024-03-30T01:40:57.971645+0000","last_active":"2024-03-30T01:41:02.786047+0000","last_peered":"2024-03-30T01:41:02.786047+0000","last_clean":"2024-03-30T01:41:02.786047+0000","last_became_active":"2024-03-30T01:40:57.971274+0000","last_became_peered":"2024-03-30T01:40:57.971274+0000","last_unstale":"2024-03-30T01:41:02.786047+0000","last_undegraded":"2024-03-30T01:41:02.786047+0000","last_fullsized":"2024-03-30T01:41:02.786047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:09.583893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869665+0000","last_change":"2024-03-30T01:40:58.431398+0000","last_active":"2024-03-30T01:41:02.869665+0000","last_peered":"2024-03-30T01:41:02.869665+0000","last_clean":"2024-03-30T01:41:02.869665+0000","last_became_active":"2024-03-30T01:40:58.431120+0000","last_became_peered":"2024-03-30T01:40:58.431120+0000","last_unstale":"2024-03-30T01:41:02.869665+0000","last_undegraded":"2024-03-30T01:41:02.869665+0000","last_fullsized":"2024-03-30T01:41:02.869665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968671+0000","last_change":"2024-03-30T01:41:02.966425+0000","last_active":"2024-03-30T01:41:02.968671+0000","last_peered":"2024-03-30T01:41:02.968671+0000","last_clean":"2024-03-30T01:41:02.968671+0000","last_became_active":"2024-03-30T01:41:02.965978+0000","last_became_peered":"2024-03-30T01:41:02.965978+0000","last_unstale":"2024-03-30T01:41:02.968671+0000","last_undegraded":"2024-03-30T01:41:02.968671+0000","last_fullsized":"2024-03-30T01:41:02.968671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780686+0000","last_change":"2024-03-30T01:40:57.956691+0000","last_active":"2024-03-30T01:41:02.780686+0000","last_peered":"2024-03-30T01:41:02.780686+0000","last_clean":"2024-03-30T01:41:02.780686+0000","last_became_active":"2024-03-30T01:40:57.956387+0000","last_became_peered":"2024-03-30T01:40:57.956387+0000","last_unstale":"2024-03-30T01:41:02.780686+0000","last_undegraded":"2024-03-30T01:41:02.780686+0000","last_fullsized":"2024-03-30T01:41:02.780686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964728+0000","last_change":"2024-03-30T01:40:58.435147+0000","last_active":"2024-03-30T01:41:02.964728+0000","last_peered":"2024-03-30T01:41:02.964728+0000","last_clean":"2024-03-30T01:41:02.964728+0000","last_became_active":"2024-03-30T01:40:58.434835+0000","last_became_peered":"2024-03-30T01:40:58.434835+0000","last_unstale":"2024-03-30T01:41:02.964728+0000","last_undegraded":"2024-03-30T01:41:02.964728+0000","last_fullsized":"2024-03-30T01:41:02.964728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:43.041425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972786+0000","last_change":"2024-03-30T01:41:02.968714+0000","last_active":"2024-03-30T01:41:02.972786+0000","last_peered":"2024-03-30T01:41:02.972786+0000","last_clean":"2024-03-30T01:41:02.972786+0000","last_became_active":"2024-03-30T01:41:02.967728+0000","last_became_peered":"2024-03-30T01:41:02.967728+0000","last_unstale":"2024-03-30T01:41:02.972786+0000","last_undegraded":"2024-03-30T01:41:02.972786+0000","last_fullsized":"2024-03-30T01:41:02.972786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:57:35.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770059+0000","last_change":"2024-03-30T01:40:57.964975+0000","last_active":"2024-03-30T01:41:02.770059+0000","last_peered":"2024-03-30T01:41:02.770059+0000","last_clean":"2024-03-30T01:41:02.770059+0000","last_became_active":"2024-03-30T01:40:57.964458+0000","last_became_peered":"2024-03-30T01:40:57.964458+0000","last_unstale":"2024-03-30T01:41:02.770059+0000","last_undegraded":"2024-03-30T01:41:02.770059+0000","last_fullsized":"2024-03-30T01:41:02.770059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872388+0000","last_change":"2024-03-30T01:40:58.425912+0000","last_active":"2024-03-30T01:41:02.872388+0000","last_peered":"2024-03-30T01:41:02.872388+0000","last_clean":"2024-03-30T01:41:02.872388+0000","last_became_active":"2024-03-30T01:40:58.425596+0000","last_became_peered":"2024-03-30T01:40:58.425596+0000","last_unstale":"2024-03-30T01:41:02.872388+0000","last_undegraded":"2024-03-30T01:41:02.872388+0000","last_fullsized":"2024-03-30T01:41:02.872388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869825+0000","last_change":"2024-03-30T01:41:02.793602+0000","last_active":"2024-03-30T01:41:02.869825+0000","last_peered":"2024-03-30T01:41:02.869825+0000","last_clean":"2024-03-30T01:41:02.869825+0000","last_became_active":"2024-03-30T01:41:02.793146+0000","last_became_peered":"2024-03-30T01:41:02.793146+0000","last_unstale":"2024-03-30T01:41:02.869825+0000","last_undegraded":"2024-03-30T01:41:02.869825+0000","last_fullsized":"2024-03-30T01:41:02.869825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782115+0000","last_change":"2024-03-30T01:40:57.962830+0000","last_active":"2024-03-30T01:41:02.782115+0000","last_peered":"2024-03-30T01:41:02.782115+0000","last_clean":"2024-03-30T01:41:02.782115+0000","last_became_active":"2024-03-30T01:40:57.961073+0000","last_became_peered":"2024-03-30T01:40:57.961073+0000","last_unstale":"2024-03-30T01:41:02.782115+0000","last_undegraded":"2024-03-30T01:41:02.782115+0000","last_fullsized":"2024-03-30T01:41:02.782115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964632+0000","last_change":"2024-03-30T01:40:58.435030+0000","last_active":"2024-03-30T01:41:02.964632+0000","last_peered":"2024-03-30T01:41:02.964632+0000","last_clean":"2024-03-30T01:41:02.964632+0000","last_became_active":"2024-03-30T01:40:58.434660+0000","last_became_peered":"2024-03-30T01:40:58.434660+0000","last_unstale":"2024-03-30T01:41:02.964632+0000","last_undegraded":"2024-03-30T01:41:02.964632+0000","last_fullsized":"2024-03-30T01:41:02.964632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:37.385758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864431+0000","last_change":"2024-03-30T01:41:02.796839+0000","last_active":"2024-03-30T01:41:02.864431+0000","last_peered":"2024-03-30T01:41:02.864431+0000","last_clean":"2024-03-30T01:41:02.864431+0000","last_became_active":"2024-03-30T01:41:02.796400+0000","last_became_peered":"2024-03-30T01:41:02.796400+0000","last_unstale":"2024-03-30T01:41:02.864431+0000","last_undegraded":"2024-03-30T01:41:02.864431+0000","last_fullsized":"2024-03-30T01:41:02.864431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:06:05.412311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.775105+0000","last_change":"2024-03-30T01:40:57.963776+0000","last_active":"2024-03-30T01:41:02.775105+0000","last_peered":"2024-03-30T01:41:02.775105+0000","last_clean":"2024-03-30T01:41:02.775105+0000","last_became_active":"2024-03-30T01:40:57.963473+0000","last_became_peered":"2024-03-30T01:40:57.963473+0000","last_unstale":"2024-03-30T01:41:02.775105+0000","last_undegraded":"2024-03-30T01:41:02.775105+0000","last_fullsized":"2024-03-30T01:41:02.775105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:09.583893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964956+0000","last_change":"2024-03-30T01:40:58.440136+0000","last_active":"2024-03-30T01:41:02.964956+0000","last_peered":"2024-03-30T01:41:02.964956+0000","last_clean":"2024-03-30T01:41:02.964956+0000","last_became_active":"2024-03-30T01:40:58.439831+0000","last_became_peered":"2024-03-30T01:40:58.439831+0000","last_unstale":"2024-03-30T01:41:02.964956+0000","last_undegraded":"2024-03-30T01:41:02.964956+0000","last_fullsized":"2024-03-30T01:41:02.964956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:28.675589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868455+0000","last_change":"2024-03-30T01:41:02.790100+0000","last_active":"2024-03-30T01:41:02.868455+0000","last_peered":"2024-03-30T01:41:02.868455+0000","last_clean":"2024-03-30T01:41:02.868455+0000","last_became_active":"2024-03-30T01:41:02.789660+0000","last_became_peered":"2024-03-30T01:41:02.789660+0000","last_unstale":"2024-03-30T01:41:02.868455+0000","last_undegraded":"2024-03-30T01:41:02.868455+0000","last_fullsized":"2024-03-30T01:41:02.868455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:22:34.848382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774905+0000","last_change":"2024-03-30T01:40:57.962434+0000","last_active":"2024-03-30T01:41:02.774905+0000","last_peered":"2024-03-30T01:41:02.774905+0000","last_clean":"2024-03-30T01:41:02.774905+0000","last_became_active":"2024-03-30T01:40:57.961987+0000","last_became_peered":"2024-03-30T01:40:57.961987+0000","last_unstale":"2024-03-30T01:41:02.774905+0000","last_undegraded":"2024-03-30T01:41:02.774905+0000","last_fullsized":"2024-03-30T01:41:02.774905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:42.056286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868476+0000","last_change":"2024-03-30T01:40:58.419381+0000","last_active":"2024-03-30T01:41:02.868476+0000","last_peered":"2024-03-30T01:41:02.868476+0000","last_clean":"2024-03-30T01:41:02.868476+0000","last_became_active":"2024-03-30T01:40:58.419079+0000","last_became_peered":"2024-03-30T01:40:58.419079+0000","last_unstale":"2024-03-30T01:41:02.868476+0000","last_undegraded":"2024-03-30T01:41:02.868476+0000","last_fullsized":"2024-03-30T01:41:02.868476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868479+0000","last_change":"2024-03-30T01:41:02.795347+0000","last_active":"2024-03-30T01:41:02.868479+0000","last_peered":"2024-03-30T01:41:02.868479+0000","last_clean":"2024-03-30T01:41:02.868479+0000","last_became_active":"2024-03-30T01:41:02.794998+0000","last_became_peered":"2024-03-30T01:41:02.794998+0000","last_unstale":"2024-03-30T01:41:02.868479+0000","last_undegraded":"2024-03-30T01:41:02.868479+0000","last_fullsized":"2024-03-30T01:41:02.868479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:19.384530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780572+0000","last_change":"2024-03-30T01:40:57.967506+0000","last_active":"2024-03-30T01:41:02.780572+0000","last_peered":"2024-03-30T01:41:02.780572+0000","last_clean":"2024-03-30T01:41:02.780572+0000","last_became_active":"2024-03-30T01:40:57.967219+0000","last_became_peered":"2024-03-30T01:40:57.967219+0000","last_unstale":"2024-03-30T01:41:02.780572+0000","last_undegraded":"2024-03-30T01:41:02.780572+0000","last_fullsized":"2024-03-30T01:41:02.780572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864547+0000","last_change":"2024-03-30T01:40:58.434945+0000","last_active":"2024-03-30T01:41:02.864547+0000","last_peered":"2024-03-30T01:41:02.864547+0000","last_clean":"2024-03-30T01:41:02.864547+0000","last_became_active":"2024-03-30T01:40:58.434344+0000","last_became_peered":"2024-03-30T01:40:58.434344+0000","last_unstale":"2024-03-30T01:41:02.864547+0000","last_undegraded":"2024-03-30T01:41:02.864547+0000","last_fullsized":"2024-03-30T01:41:02.864547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:43.058516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868577+0000","last_change":"2024-03-30T01:41:02.796689+0000","last_active":"2024-03-30T01:41:02.868577+0000","last_peered":"2024-03-30T01:41:02.868577+0000","last_clean":"2024-03-30T01:41:02.868577+0000","last_became_active":"2024-03-30T01:41:02.796285+0000","last_became_peered":"2024-03-30T01:41:02.796285+0000","last_unstale":"2024-03-30T01:41:02.868577+0000","last_undegraded":"2024-03-30T01:41:02.868577+0000","last_fullsized":"2024-03-30T01:41:02.868577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868364+0000","last_change":"2024-03-30T01:40:58.426059+0000","last_active":"2024-03-30T01:41:02.868364+0000","last_peered":"2024-03-30T01:41:02.868364+0000","last_clean":"2024-03-30T01:41:02.868364+0000","last_became_active":"2024-03-30T01:40:58.424722+0000","last_became_peered":"2024-03-30T01:40:58.424722+0000","last_unstale":"2024-03-30T01:41:02.868364+0000","last_undegraded":"2024-03-30T01:41:02.868364+0000","last_fullsized":"2024-03-30T01:41:02.868364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782192+0000","last_change":"2024-03-30T01:40:57.964575+0000","last_active":"2024-03-30T01:41:02.782192+0000","last_peered":"2024-03-30T01:41:02.782192+0000","last_clean":"2024-03-30T01:41:02.782192+0000","last_became_active":"2024-03-30T01:40:57.962016+0000","last_became_peered":"2024-03-30T01:40:57.962016+0000","last_unstale":"2024-03-30T01:41:02.782192+0000","last_undegraded":"2024-03-30T01:41:02.782192+0000","last_fullsized":"2024-03-30T01:41:02.782192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:04.688535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.974026+0000","last_change":"2024-03-30T01:41:02.969088+0000","last_active":"2024-03-30T01:41:02.974026+0000","last_peered":"2024-03-30T01:41:02.974026+0000","last_clean":"2024-03-30T01:41:02.974026+0000","last_became_active":"2024-03-30T01:41:02.968624+0000","last_became_peered":"2024-03-30T01:41:02.968624+0000","last_unstale":"2024-03-30T01:41:02.974026+0000","last_undegraded":"2024-03-30T01:41:02.974026+0000","last_fullsized":"2024-03-30T01:41:02.974026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:34.265123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868700+0000","last_change":"2024-03-30T01:40:58.429787+0000","last_active":"2024-03-30T01:41:02.868700+0000","last_peered":"2024-03-30T01:41:02.868700+0000","last_clean":"2024-03-30T01:41:02.868700+0000","last_became_active":"2024-03-30T01:40:58.429037+0000","last_became_peered":"2024-03-30T01:40:58.429037+0000","last_unstale":"2024-03-30T01:41:02.868700+0000","last_undegraded":"2024-03-30T01:41:02.868700+0000","last_fullsized":"2024-03-30T01:41:02.868700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:54.449729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780784+0000","last_change":"2024-03-30T01:40:57.954275+0000","last_active":"2024-03-30T01:41:02.780784+0000","last_peered":"2024-03-30T01:41:02.780784+0000","last_clean":"2024-03-30T01:41:02.780784+0000","last_became_active":"2024-03-30T01:40:57.953987+0000","last_became_peered":"2024-03-30T01:40:57.953987+0000","last_unstale":"2024-03-30T01:41:02.780784+0000","last_undegraded":"2024-03-30T01:41:02.780784+0000","last_fullsized":"2024-03-30T01:41:02.780784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964197+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:41:02.964197+0000","last_peered":"2024-03-30T01:41:02.964197+0000","last_clean":"2024-03-30T01:41:02.964197+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:41:02.964197+0000","last_undegraded":"2024-03-30T01:41:02.964197+0000","last_fullsized":"2024-03-30T01:41:02.964197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972468+0000","last_change":"2024-03-30T01:41:02.969064+0000","last_active":"2024-03-30T01:41:02.972468+0000","last_peered":"2024-03-30T01:41:02.972468+0000","last_clean":"2024-03-30T01:41:02.972468+0000","last_became_active":"2024-03-30T01:41:02.968333+0000","last_became_peered":"2024-03-30T01:41:02.968333+0000","last_unstale":"2024-03-30T01:41:02.972468+0000","last_undegraded":"2024-03-30T01:41:02.972468+0000","last_fullsized":"2024-03-30T01:41:02.972468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:34.173820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869624+0000","last_change":"2024-03-30T01:40:58.432432+0000","last_active":"2024-03-30T01:41:02.869624+0000","last_peered":"2024-03-30T01:41:02.869624+0000","last_clean":"2024-03-30T01:41:02.869624+0000","last_became_active":"2024-03-30T01:40:58.432158+0000","last_became_peered":"2024-03-30T01:40:58.432158+0000","last_unstale":"2024-03-30T01:41:02.869624+0000","last_undegraded":"2024-03-30T01:41:02.869624+0000","last_fullsized":"2024-03-30T01:41:02.869624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964478+0000","last_change":"2024-03-30T01:40:57.966931+0000","last_active":"2024-03-30T01:41:02.964478+0000","last_peered":"2024-03-30T01:41:02.964478+0000","last_clean":"2024-03-30T01:41:02.964478+0000","last_became_active":"2024-03-30T01:40:57.966589+0000","last_became_peered":"2024-03-30T01:40:57.966589+0000","last_unstale":"2024-03-30T01:41:02.964478+0000","last_undegraded":"2024-03-30T01:41:02.964478+0000","last_fullsized":"2024-03-30T01:41:02.964478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:15.934995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968515+0000","last_change":"2024-03-30T01:41:02.966273+0000","last_active":"2024-03-30T01:41:02.968515+0000","last_peered":"2024-03-30T01:41:02.968515+0000","last_clean":"2024-03-30T01:41:02.968515+0000","last_became_active":"2024-03-30T01:41:02.965857+0000","last_became_peered":"2024-03-30T01:41:02.965857+0000","last_unstale":"2024-03-30T01:41:02.968515+0000","last_undegraded":"2024-03-30T01:41:02.968515+0000","last_fullsized":"2024-03-30T01:41:02.968515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:58.270483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964736+0000","last_change":"2024-03-30T01:40:57.964957+0000","last_active":"2024-03-30T01:41:02.964736+0000","last_peered":"2024-03-30T01:41:02.964736+0000","last_clean":"2024-03-30T01:41:02.964736+0000","last_became_active":"2024-03-30T01:40:57.963985+0000","last_became_peered":"2024-03-30T01:40:57.963985+0000","last_unstale":"2024-03-30T01:41:02.964736+0000","last_undegraded":"2024-03-30T01:41:02.964736+0000","last_fullsized":"2024-03-30T01:41:02.964736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:05:32.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868496+0000","last_change":"2024-03-30T01:40:58.425295+0000","last_active":"2024-03-30T01:41:02.868496+0000","last_peered":"2024-03-30T01:41:02.868496+0000","last_clean":"2024-03-30T01:41:02.868496+0000","last_became_active":"2024-03-30T01:40:58.424967+0000","last_became_peered":"2024-03-30T01:40:58.424967+0000","last_unstale":"2024-03-30T01:41:02.868496+0000","last_undegraded":"2024-03-30T01:41:02.868496+0000","last_fullsized":"2024-03-30T01:41:02.868496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.975320+0000","last_change":"2024-03-30T01:41:02.970612+0000","last_active":"2024-03-30T01:41:02.975320+0000","last_peered":"2024-03-30T01:41:02.975320+0000","last_clean":"2024-03-30T01:41:02.975320+0000","last_became_active":"2024-03-30T01:41:02.969823+0000","last_became_peered":"2024-03-30T01:41:02.969823+0000","last_unstale":"2024-03-30T01:41:02.975320+0000","last_undegraded":"2024-03-30T01:41:02.975320+0000","last_fullsized":"2024-03-30T01:41:02.975320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:25:35.463484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T01:41:03.568727+0000","last_change":"2024-03-30T01:40:57.968553+0000","last_active":"2024-03-30T01:41:03.568727+0000","last_peered":"2024-03-30T01:41:03.568727+0000","last_clean":"2024-03-30T01:41:03.568727+0000","last_became_active":"2024-03-30T01:40:57.968315+0000","last_became_peered":"2024-03-30T01:40:57.968315+0000","last_unstale":"2024-03-30T01:41:03.568727+0000","last_undegraded":"2024-03-30T01:41:03.568727+0000","last_fullsized":"2024-03-30T01:41:03.568727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:34.662924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869030+0000","last_change":"2024-03-30T01:40:58.429594+0000","last_active":"2024-03-30T01:41:02.869030+0000","last_peered":"2024-03-30T01:41:02.869030+0000","last_clean":"2024-03-30T01:41:02.869030+0000","last_became_active":"2024-03-30T01:40:58.429296+0000","last_became_peered":"2024-03-30T01:40:58.429296+0000","last_unstale":"2024-03-30T01:41:02.869030+0000","last_undegraded":"2024-03-30T01:41:02.869030+0000","last_fullsized":"2024-03-30T01:41:02.869030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.974419+0000","last_change":"2024-03-30T01:41:02.970239+0000","last_active":"2024-03-30T01:41:02.974419+0000","last_peered":"2024-03-30T01:41:02.974419+0000","last_clean":"2024-03-30T01:41:02.974419+0000","last_became_active":"2024-03-30T01:41:02.969823+0000","last_became_peered":"2024-03-30T01:41:02.969823+0000","last_unstale":"2024-03-30T01:41:02.974419+0000","last_undegraded":"2024-03-30T01:41:02.974419+0000","last_fullsized":"2024-03-30T01:41:02.974419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:00.285894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770593+0000","last_change":"2024-03-30T01:40:57.963709+0000","last_active":"2024-03-30T01:41:02.770593+0000","last_peered":"2024-03-30T01:41:02.770593+0000","last_clean":"2024-03-30T01:41:02.770593+0000","last_became_active":"2024-03-30T01:40:57.963374+0000","last_became_peered":"2024-03-30T01:40:57.963374+0000","last_unstale":"2024-03-30T01:41:02.770593+0000","last_undegraded":"2024-03-30T01:41:02.770593+0000","last_fullsized":"2024-03-30T01:41:02.770593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869920+0000","last_change":"2024-03-30T01:40:58.430636+0000","last_active":"2024-03-30T01:41:02.869920+0000","last_peered":"2024-03-30T01:41:02.869920+0000","last_clean":"2024-03-30T01:41:02.869920+0000","last_became_active":"2024-03-30T01:40:58.430400+0000","last_became_peered":"2024-03-30T01:40:58.430400+0000","last_unstale":"2024-03-30T01:41:02.869920+0000","last_undegraded":"2024-03-30T01:41:02.869920+0000","last_fullsized":"2024-03-30T01:41:02.869920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:44:58.613670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864624+0000","last_change":"2024-03-30T01:41:02.788933+0000","last_active":"2024-03-30T01:41:02.864624+0000","last_peered":"2024-03-30T01:41:02.864624+0000","last_clean":"2024-03-30T01:41:02.864624+0000","last_became_active":"2024-03-30T01:41:02.788466+0000","last_became_peered":"2024-03-30T01:41:02.788466+0000","last_unstale":"2024-03-30T01:41:02.864624+0000","last_undegraded":"2024-03-30T01:41:02.864624+0000","last_fullsized":"2024-03-30T01:41:02.864624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964182+0000","last_change":"2024-03-30T01:40:57.962449+0000","last_active":"2024-03-30T01:41:02.964182+0000","last_peered":"2024-03-30T01:41:02.964182+0000","last_clean":"2024-03-30T01:41:02.964182+0000","last_became_active":"2024-03-30T01:40:57.962192+0000","last_became_peered":"2024-03-30T01:40:57.962192+0000","last_unstale":"2024-03-30T01:41:02.964182+0000","last_undegraded":"2024-03-30T01:41:02.964182+0000","last_fullsized":"2024-03-30T01:41:02.964182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:36.274096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864497+0000","last_change":"2024-03-30T01:40:58.420796+0000","last_active":"2024-03-30T01:41:02.864497+0000","last_peered":"2024-03-30T01:41:02.864497+0000","last_clean":"2024-03-30T01:41:02.864497+0000","last_became_active":"2024-03-30T01:40:58.420441+0000","last_became_peered":"2024-03-30T01:40:58.420441+0000","last_unstale":"2024-03-30T01:41:02.864497+0000","last_undegraded":"2024-03-30T01:41:02.864497+0000","last_fullsized":"2024-03-30T01:41:02.864497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:16.937225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.969938+0000","last_change":"2024-03-30T01:41:02.966462+0000","last_active":"2024-03-30T01:41:02.969938+0000","last_peered":"2024-03-30T01:41:02.969938+0000","last_clean":"2024-03-30T01:41:02.969938+0000","last_became_active":"2024-03-30T01:41:02.966056+0000","last_became_peered":"2024-03-30T01:41:02.966056+0000","last_unstale":"2024-03-30T01:41:02.969938+0000","last_undegraded":"2024-03-30T01:41:02.969938+0000","last_fullsized":"2024-03-30T01:41:02.969938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:29.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774672+0000","last_change":"2024-03-30T01:40:57.967648+0000","last_active":"2024-03-30T01:41:02.774672+0000","last_peered":"2024-03-30T01:41:02.774672+0000","last_clean":"2024-03-30T01:41:02.774672+0000","last_became_active":"2024-03-30T01:40:57.967362+0000","last_became_peered":"2024-03-30T01:40:57.967362+0000","last_unstale":"2024-03-30T01:41:02.774672+0000","last_undegraded":"2024-03-30T01:41:02.774672+0000","last_fullsized":"2024-03-30T01:41:02.774672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868291+0000","last_change":"2024-03-30T01:40:58.432296+0000","last_active":"2024-03-30T01:41:02.868291+0000","last_peered":"2024-03-30T01:41:02.868291+0000","last_clean":"2024-03-30T01:41:02.868291+0000","last_became_active":"2024-03-30T01:40:58.432045+0000","last_became_peered":"2024-03-30T01:40:58.432045+0000","last_unstale":"2024-03-30T01:41:02.868291+0000","last_undegraded":"2024-03-30T01:41:02.868291+0000","last_fullsized":"2024-03-30T01:41:02.868291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864586+0000","last_change":"2024-03-30T01:41:02.796892+0000","last_active":"2024-03-30T01:41:02.864586+0000","last_peered":"2024-03-30T01:41:02.864586+0000","last_clean":"2024-03-30T01:41:02.864586+0000","last_became_active":"2024-03-30T01:41:02.796347+0000","last_became_peered":"2024-03-30T01:41:02.796347+0000","last_unstale":"2024-03-30T01:41:02.864586+0000","last_undegraded":"2024-03-30T01:41:02.864586+0000","last_fullsized":"2024-03-30T01:41:02.864586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:20.847856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774635+0000","last_change":"2024-03-30T01:40:57.969391+0000","last_active":"2024-03-30T01:41:02.774635+0000","last_peered":"2024-03-30T01:41:02.774635+0000","last_clean":"2024-03-30T01:41:02.774635+0000","last_became_active":"2024-03-30T01:40:57.969107+0000","last_became_peered":"2024-03-30T01:40:57.969107+0000","last_unstale":"2024-03-30T01:41:02.774635+0000","last_undegraded":"2024-03-30T01:41:02.774635+0000","last_fullsized":"2024-03-30T01:41:02.774635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868769+0000","last_change":"2024-03-30T01:40:58.430179+0000","last_active":"2024-03-30T01:41:02.868769+0000","last_peered":"2024-03-30T01:41:02.868769+0000","last_clean":"2024-03-30T01:41:02.868769+0000","last_became_active":"2024-03-30T01:40:58.429919+0000","last_became_peered":"2024-03-30T01:40:58.429919+0000","last_unstale":"2024-03-30T01:41:02.868769+0000","last_undegraded":"2024-03-30T01:41:02.868769+0000","last_fullsized":"2024-03-30T01:41:02.868769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864611+0000","last_change":"2024-03-30T01:41:02.792167+0000","last_active":"2024-03-30T01:41:02.864611+0000","last_peered":"2024-03-30T01:41:02.864611+0000","last_clean":"2024-03-30T01:41:02.864611+0000","last_became_active":"2024-03-30T01:41:02.791805+0000","last_became_peered":"2024-03-30T01:41:02.791805+0000","last_unstale":"2024-03-30T01:41:02.864611+0000","last_undegraded":"2024-03-30T01:41:02.864611+0000","last_fullsized":"2024-03-30T01:41:02.864611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964766+0000","last_change":"2024-03-30T01:40:57.964733+0000","last_active":"2024-03-30T01:41:02.964766+0000","last_peered":"2024-03-30T01:41:02.964766+0000","last_clean":"2024-03-30T01:41:02.964766+0000","last_became_active":"2024-03-30T01:40:57.964430+0000","last_became_peered":"2024-03-30T01:40:57.964430+0000","last_unstale":"2024-03-30T01:41:02.964766+0000","last_undegraded":"2024-03-30T01:41:02.964766+0000","last_fullsized":"2024-03-30T01:41:02.964766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:42.056286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872480+0000","last_change":"2024-03-30T01:40:58.420811+0000","last_active":"2024-03-30T01:41:02.872480+0000","last_peered":"2024-03-30T01:41:02.872480+0000","last_clean":"2024-03-30T01:41:02.872480+0000","last_became_active":"2024-03-30T01:40:58.420491+0000","last_became_peered":"2024-03-30T01:40:58.420491+0000","last_unstale":"2024-03-30T01:41:02.872480+0000","last_undegraded":"2024-03-30T01:41:02.872480+0000","last_fullsized":"2024-03-30T01:41:02.872480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968342+0000","last_change":"2024-03-30T01:41:02.966275+0000","last_active":"2024-03-30T01:41:02.968342+0000","last_peered":"2024-03-30T01:41:02.968342+0000","last_clean":"2024-03-30T01:41:02.968342+0000","last_became_active":"2024-03-30T01:41:02.965934+0000","last_became_peered":"2024-03-30T01:41:02.965934+0000","last_unstale":"2024-03-30T01:41:02.968342+0000","last_undegraded":"2024-03-30T01:41:02.968342+0000","last_fullsized":"2024-03-30T01:41:02.968342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:39.954419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.783325+0000","last_change":"2024-03-30T01:40:57.962835+0000","last_active":"2024-03-30T01:41:02.783325+0000","last_peered":"2024-03-30T01:41:02.783325+0000","last_clean":"2024-03-30T01:41:02.783325+0000","last_became_active":"2024-03-30T01:40:57.961552+0000","last_became_peered":"2024-03-30T01:40:57.961552+0000","last_unstale":"2024-03-30T01:41:02.783325+0000","last_undegraded":"2024-03-30T01:41:02.783325+0000","last_fullsized":"2024-03-30T01:41:02.783325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964874+0000","last_change":"2024-03-30T01:40:58.432622+0000","last_active":"2024-03-30T01:41:02.964874+0000","last_peered":"2024-03-30T01:41:02.964874+0000","last_clean":"2024-03-30T01:41:02.964874+0000","last_became_active":"2024-03-30T01:40:58.432336+0000","last_became_peered":"2024-03-30T01:40:58.432336+0000","last_unstale":"2024-03-30T01:41:02.964874+0000","last_undegraded":"2024-03-30T01:41:02.964874+0000","last_fullsized":"2024-03-30T01:41:02.964874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:24:50.793081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869687+0000","last_change":"2024-03-30T01:41:02.799702+0000","last_active":"2024-03-30T01:41:02.869687+0000","last_peered":"2024-03-30T01:41:02.869687+0000","last_clean":"2024-03-30T01:41:02.869687+0000","last_became_active":"2024-03-30T01:41:02.799189+0000","last_became_peered":"2024-03-30T01:41:02.799189+0000","last_unstale":"2024-03-30T01:41:02.869687+0000","last_undegraded":"2024-03-30T01:41:02.869687+0000","last_fullsized":"2024-03-30T01:41:02.869687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.777920+0000","last_change":"2024-03-30T01:40:57.964577+0000","last_active":"2024-03-30T01:41:02.777920+0000","last_peered":"2024-03-30T01:41:02.777920+0000","last_clean":"2024-03-30T01:41:02.777920+0000","last_became_active":"2024-03-30T01:40:57.964277+0000","last_became_peered":"2024-03-30T01:40:57.964277+0000","last_unstale":"2024-03-30T01:41:02.777920+0000","last_undegraded":"2024-03-30T01:41:02.777920+0000","last_fullsized":"2024-03-30T01:41:02.777920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:15.934995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868392+0000","last_change":"2024-03-30T01:40:58.428923+0000","last_active":"2024-03-30T01:41:02.868392+0000","last_peered":"2024-03-30T01:41:02.868392+0000","last_clean":"2024-03-30T01:41:02.868392+0000","last_became_active":"2024-03-30T01:40:58.428204+0000","last_became_peered":"2024-03-30T01:40:58.428204+0000","last_unstale":"2024-03-30T01:41:02.868392+0000","last_undegraded":"2024-03-30T01:41:02.868392+0000","last_fullsized":"2024-03-30T01:41:02.868392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.970293+0000","last_change":"2024-03-30T01:41:02.968043+0000","last_active":"2024-03-30T01:41:02.970293+0000","last_peered":"2024-03-30T01:41:02.970293+0000","last_clean":"2024-03-30T01:41:02.970293+0000","last_became_active":"2024-03-30T01:41:02.967586+0000","last_became_peered":"2024-03-30T01:41:02.967586+0000","last_unstale":"2024-03-30T01:41:02.970293+0000","last_undegraded":"2024-03-30T01:41:02.970293+0000","last_fullsized":"2024-03-30T01:41:02.970293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:29.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782245+0000","last_change":"2024-03-30T01:40:57.962668+0000","last_active":"2024-03-30T01:41:02.782245+0000","last_peered":"2024-03-30T01:41:02.782245+0000","last_clean":"2024-03-30T01:41:02.782245+0000","last_became_active":"2024-03-30T01:40:57.962255+0000","last_became_peered":"2024-03-30T01:40:57.962255+0000","last_unstale":"2024-03-30T01:41:02.782245+0000","last_undegraded":"2024-03-30T01:41:02.782245+0000","last_fullsized":"2024-03-30T01:41:02.782245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965085+0000","last_change":"2024-03-30T01:40:58.441825+0000","last_active":"2024-03-30T01:41:02.965085+0000","last_peered":"2024-03-30T01:41:02.965085+0000","last_clean":"2024-03-30T01:41:02.965085+0000","last_became_active":"2024-03-30T01:40:58.441575+0000","last_became_peered":"2024-03-30T01:40:58.441575+0000","last_unstale":"2024-03-30T01:41:02.965085+0000","last_undegraded":"2024-03-30T01:41:02.965085+0000","last_fullsized":"2024-03-30T01:41:02.965085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:51.791864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872404+0000","last_change":"2024-03-30T01:40:58.425945+0000","last_active":"2024-03-30T01:41:02.872404+0000","last_peered":"2024-03-30T01:41:02.872404+0000","last_clean":"2024-03-30T01:41:02.872404+0000","last_became_active":"2024-03-30T01:40:58.425607+0000","last_became_peered":"2024-03-30T01:40:58.425607+0000","last_unstale":"2024-03-30T01:41:02.872404+0000","last_undegraded":"2024-03-30T01:41:02.872404+0000","last_fullsized":"2024-03-30T01:41:02.872404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868697+0000","last_change":"2024-03-30T01:40:58.427012+0000","last_active":"2024-03-30T01:41:02.868697+0000","last_peered":"2024-03-30T01:41:02.868697+0000","last_clean":"2024-03-30T01:41:02.868697+0000","last_became_active":"2024-03-30T01:40:58.426750+0000","last_became_peered":"2024-03-30T01:40:58.426750+0000","last_unstale":"2024-03-30T01:41:02.868697+0000","last_undegraded":"2024-03-30T01:41:02.868697+0000","last_fullsized":"2024-03-30T01:41:02.868697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:37.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868317+0000","last_change":"2024-03-30T01:40:58.426120+0000","last_active":"2024-03-30T01:41:02.868317+0000","last_peered":"2024-03-30T01:41:02.868317+0000","last_clean":"2024-03-30T01:41:02.868317+0000","last_became_active":"2024-03-30T01:40:58.425776+0000","last_became_peered":"2024-03-30T01:40:58.425776+0000","last_unstale":"2024-03-30T01:41:02.868317+0000","last_undegraded":"2024-03-30T01:41:02.868317+0000","last_fullsized":"2024-03-30T01:41:02.868317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869771+0000","last_change":"2024-03-30T01:40:58.436516+0000","last_active":"2024-03-30T01:41:02.869771+0000","last_peered":"2024-03-30T01:41:02.869771+0000","last_clean":"2024-03-30T01:41:02.869771+0000","last_became_active":"2024-03-30T01:40:58.436247+0000","last_became_peered":"2024-03-30T01:40:58.436247+0000","last_unstale":"2024-03-30T01:41:02.869771+0000","last_undegraded":"2024-03-30T01:41:02.869771+0000","last_fullsized":"2024-03-30T01:41:02.869771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868868+0000","last_change":"2024-03-30T01:40:58.415507+0000","last_active":"2024-03-30T01:41:02.868868+0000","last_peered":"2024-03-30T01:41:02.868868+0000","last_clean":"2024-03-30T01:41:02.868868+0000","last_became_active":"2024-03-30T01:40:58.415197+0000","last_became_peered":"2024-03-30T01:40:58.415197+0000","last_unstale":"2024-03-30T01:41:02.868868+0000","last_undegraded":"2024-03-30T01:41:02.868868+0000","last_fullsized":"2024-03-30T01:41:02.868868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868506+0000","last_change":"2024-03-30T01:40:58.425962+0000","last_active":"2024-03-30T01:41:02.868506+0000","last_peered":"2024-03-30T01:41:02.868506+0000","last_clean":"2024-03-30T01:41:02.868506+0000","last_became_active":"2024-03-30T01:40:58.425667+0000","last_became_peered":"2024-03-30T01:40:58.425667+0000","last_unstale":"2024-03-30T01:41:02.868506+0000","last_undegraded":"2024-03-30T01:41:02.868506+0000","last_fullsized":"2024-03-30T01:41:02.868506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872585+0000","last_change":"2024-03-30T01:40:58.429802+0000","last_active":"2024-03-30T01:41:02.872585+0000","last_peered":"2024-03-30T01:41:02.872585+0000","last_clean":"2024-03-30T01:41:02.872585+0000","last_became_active":"2024-03-30T01:40:58.429296+0000","last_became_peered":"2024-03-30T01:40:58.429296+0000","last_unstale":"2024-03-30T01:41:02.872585+0000","last_undegraded":"2024-03-30T01:41:02.872585+0000","last_fullsized":"2024-03-30T01:41:02.872585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869051+0000","last_change":"2024-03-30T01:40:58.420564+0000","last_active":"2024-03-30T01:41:02.869051+0000","last_peered":"2024-03-30T01:41:02.869051+0000","last_clean":"2024-03-30T01:41:02.869051+0000","last_became_active":"2024-03-30T01:40:58.420286+0000","last_became_peered":"2024-03-30T01:40:58.420286+0000","last_unstale":"2024-03-30T01:41:02.869051+0000","last_undegraded":"2024-03-30T01:41:02.869051+0000","last_fullsized":"2024-03-30T01:41:02.869051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868288+0000","last_change":"2024-03-30T01:40:58.418904+0000","last_active":"2024-03-30T01:41:02.868288+0000","last_peered":"2024-03-30T01:41:02.868288+0000","last_clean":"2024-03-30T01:41:02.868288+0000","last_became_active":"2024-03-30T01:40:58.418351+0000","last_became_peered":"2024-03-30T01:40:58.418351+0000","last_unstale":"2024-03-30T01:41:02.868288+0000","last_undegraded":"2024-03-30T01:41:02.868288+0000","last_fullsized":"2024-03-30T01:41:02.868288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868595+0000","last_change":"2024-03-30T01:40:58.416220+0000","last_active":"2024-03-30T01:41:02.868595+0000","last_peered":"2024-03-30T01:41:02.868595+0000","last_clean":"2024-03-30T01:41:02.868595+0000","last_became_active":"2024-03-30T01:40:58.415937+0000","last_became_peered":"2024-03-30T01:40:58.415937+0000","last_unstale":"2024-03-30T01:41:02.868595+0000","last_undegraded":"2024-03-30T01:41:02.868595+0000","last_fullsized":"2024-03-30T01:41:02.868595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868332+0000","last_change":"2024-03-30T01:40:58.433268+0000","last_active":"2024-03-30T01:41:02.868332+0000","last_peered":"2024-03-30T01:41:02.868332+0000","last_clean":"2024-03-30T01:41:02.868332+0000","last_became_active":"2024-03-30T01:40:58.432963+0000","last_became_peered":"2024-03-30T01:40:58.432963+0000","last_unstale":"2024-03-30T01:41:02.868332+0000","last_undegraded":"2024-03-30T01:41:02.868332+0000","last_fullsized":"2024-03-30T01:41:02.868332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869784+0000","last_change":"2024-03-30T01:40:58.440462+0000","last_active":"2024-03-30T01:41:02.869784+0000","last_peered":"2024-03-30T01:41:02.869784+0000","last_clean":"2024-03-30T01:41:02.869784+0000","last_became_active":"2024-03-30T01:40:58.440144+0000","last_became_peered":"2024-03-30T01:40:58.440144+0000","last_unstale":"2024-03-30T01:41:02.869784+0000","last_undegraded":"2024-03-30T01:41:02.869784+0000","last_fullsized":"2024-03-30T01:41:02.869784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872499+0000","last_change":"2024-03-30T01:40:58.422820+0000","last_active":"2024-03-30T01:41:02.872499+0000","last_peered":"2024-03-30T01:41:02.872499+0000","last_clean":"2024-03-30T01:41:02.872499+0000","last_became_active":"2024-03-30T01:40:58.420834+0000","last_became_peered":"2024-03-30T01:40:58.420834+0000","last_unstale":"2024-03-30T01:41:02.872499+0000","last_undegraded":"2024-03-30T01:41:02.872499+0000","last_fullsized":"2024-03-30T01:41:02.872499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.858000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.99}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.974}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.504}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":856064,"data_stored":689817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.478999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.7400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.3970000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.5919999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7480000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.8410000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.8970000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:41:24.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:41:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.017+0000 7f6cf7136700 1 Processor -- start 2024-03-30T01:41:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.017+0000 7f6cf7136700 1 -- start start 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cf7136700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6cf014eb70 0x7f6cf00c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cf7136700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf0070e20 con 0x7f6cf00c1b20 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cf7136700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf006ae10 con 0x7f6cf00718c0 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cf7136700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf00727c0 con 0x7f6cf014eb70 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6ceffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6cf00c1b20 0x7f6cf006a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:34584/0 (socket says 172.21.15.31:34584) 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6ceffff700 1 -- 172.21.15.31:0/3227772230 learned_addr learned my addr 172.21.15.31:0/3227772230 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cf4ed2700 1 --2- 172.21.15.31:0/3227772230 >> v2:172.21.15.59:3300/0 conn(0x7f6cf014eb70 0x7f6cf00c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545692136 0 0) 0x7f6cf0070e20 con 0x7f6cf00c1b20 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd80035b0 con 0x7f6cf00c1b20 2024-03-30T01:41:25.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2283751857 0 0) 0x7f6cd80035b0 con 0x7f6cf00c1b20 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 >> v2:172.21.15.59:3300/0 conn(0x7f6cf014eb70 msgr2=0x7f6cf00c28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 --2- 172.21.15.31:0/3227772230 >> v2:172.21.15.59:3300/0 conn(0x7f6cf014eb70 0x7f6cf00c28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 >> v1:172.21.15.59:6789/0 conn(0x7f6cf00718c0 legacy=0x7f6cf015bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf0071070 con 0x7f6cf00c1b20 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f6ce40017f0 con 0x7f6cf00c1b20 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 >> v1:172.21.15.31:6789/0 conn(0x7f6cf00c1b20 legacy=0x7f6cf006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.018+0000 7f6cef7fe700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd8003a00 0x7f6cd8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cef7fe700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cd8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cef7fe700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6cd8005330 0x7f6cd800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd800c1d0 con 0x7f6cd8004850 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd800c340 con 0x7f6cd8003a00 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd800c500 con 0x7f6cd8005330 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6ceffff700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cd8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cf4ed2700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd8003a00 0x7f6cd8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cf56d3700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6cd8005330 0x7f6cd800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6ceffff700 1 -- 172.21.15.31:0/3227772230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6cd8005330 msgr2=0x7f6cd800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6ceffff700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6cd8005330 0x7f6cd800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6ceffff700 1 -- 172.21.15.31:0/3227772230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd8003a00 msgr2=0x7f6cd8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6ceffff700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cd8003a00 0x7f6cd8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6ceffff700 1 -- 172.21.15.31:0/3227772230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd80035b0 con 0x7f6cd8004850 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cf56d3700 1 -- 172.21.15.31:0/3227772230 reap_dead start 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6ceffff700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cd8004cd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c210 tx=0x7f6cd8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e03175da1f6a23 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4007c00 con 0x7f6cd8004850 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce400cdd0 con 0x7f6cd8004850 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.019+0000 7f6cef7fe700 1 -- 172.21.15.31:0/3227772230 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4007310 con 0x7f6cd8004850 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.020+0000 7f6cf7136700 1 -- 172.21.15.31:0/3227772230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 msgr2=0x7f6cd8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.020+0000 7f6cf7136700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cd8004cd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c210 tx=0x7f6cd8005880 comp rx=0 tx=0).stop 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.020+0000 7f6cf7136700 1 -- 172.21.15.31:0/3227772230 shutdown_connections 2024-03-30T01:41:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.020+0000 7f6cf7136700 1 --2- 172.21.15.31:0/3227772230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cd8004cd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.020+0000 7f6cf7136700 1 -- 172.21.15.31:0/3227772230 >> 172.21.15.31:0/3227772230 conn(0x7f6cf00c3190 msgr2=0x7f6cf00c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.020+0000 7f6cf7136700 1 -- 172.21.15.31:0/3227772230 shutdown_connections 2024-03-30T01:41:25.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.020+0000 7f6cf7136700 1 -- 172.21.15.31:0/3227772230 wait complete. 2024-03-30T01:41:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.021+0000 7f6cf7136700 1 Processor -- start 2024-03-30T01:41:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.021+0000 7f6cf7136700 1 -- start start 2024-03-30T01:41:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf7136700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cf00c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf7136700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6cf0153be0 0x7f6cf0154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf7136700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf0154b60 0x7f6cf0154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf7136700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00c5b60 con 0x7f6cd8004850 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf7136700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00c6b00 con 0x7f6cf0154b60 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf7136700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf01e6000 con 0x7f6cf0153be0 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf4ed2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cf00c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf4ed2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cf00c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37044/0 (socket says 172.21.15.31:37044) 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf4ed2700 1 -- 172.21.15.31:0/3814389348 learned_addr learned my addr 172.21.15.31:0/3814389348 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6cf56d3700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf0154b60 0x7f6cf0154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.022+0000 7f6ceffff700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6cf0153be0 0x7f6cf0154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6cf4ed2700 1 -- 172.21.15.31:0/3814389348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6cf0153be0 msgr2=0x7f6cf0154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6cf4ed2700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6cf0153be0 0x7f6cf0154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6cf4ed2700 1 -- 172.21.15.31:0/3814389348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf0154b60 msgr2=0x7f6cf0154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6cf4ed2700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf0154b60 0x7f6cf0154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6cf4ed2700 1 -- 172.21.15.31:0/3814389348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf01e62e0 con 0x7f6cd8004850 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6cf4ed2700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cf00c6f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0011a00 tx=0x7f6ce0011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6ced7fa700 1 -- 172.21.15.31:0/3814389348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0011e40 con 0x7f6cd8004850 2024-03-30T01:41:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6ced7fa700 1 -- 172.21.15.31:0/3814389348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0004240 con 0x7f6cd8004850 2024-03-30T01:41:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf00c4220 con 0x7f6cd8004850 2024-03-30T01:41:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6ced7fa700 1 -- 172.21.15.31:0/3814389348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000f730 con 0x7f6cd8004850 2024-03-30T01:41:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.023+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf0155520 con 0x7f6cd8004850 2024-03-30T01:41:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.024+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf00c4590 con 0x7f6cd8004850 2024-03-30T01:41:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.025+0000 7f6ced7fa700 1 -- 172.21.15.31:0/3814389348 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6ce000f890 con 0x7f6cd8004850 2024-03-30T01:41:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.025+0000 7f6ced7fa700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6cdc06ac90 0x7f6cdc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.025+0000 7f6ced7fa700 1 -- 172.21.15.31:0/3814389348 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6ce0089a20 con 0x7f6cd8004850 2024-03-30T01:41:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.025+0000 7f6ceffff700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6cdc06ac90 0x7f6cdc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.026+0000 7f6ceffff700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6cdc06ac90 0x7f6cdc06d150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000ef0 tx=0x7f6ce4007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:25.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.030+0000 7f6ced7fa700 1 -- 172.21.15.31:0/3814389348 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6ce0054910 con 0x7f6cd8004850 2024-03-30T01:41:25.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.238+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cf0154780 con 0x7f6cdc06ac90 2024-03-30T01:41:25.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.248+0000 7f6ced7fa700 1 -- 172.21.15.31:0/3814389348 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345175 (secure 0 0 0) 0x7f6cf0154780 con 0x7f6cdc06ac90 2024-03-30T01:41:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.252+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6cdc06ac90 msgr2=0x7f6cdc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.252+0000 7f6cf7136700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6cdc06ac90 0x7f6cdc06d150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000ef0 tx=0x7f6ce4007000 comp rx=0 tx=0).stop 2024-03-30T01:41:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.252+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 msgr2=0x7f6cf00c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.252+0000 7f6cf7136700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cf00c6f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0011a00 tx=0x7f6ce0011d10 comp rx=0 tx=0).stop 2024-03-30T01:41:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.253+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 shutdown_connections 2024-03-30T01:41:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.253+0000 7f6cf7136700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cf00c6f20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.253+0000 7f6cf7136700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6cdc06ac90 0x7f6cdc06d150 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.253+0000 7f6cf7136700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cf0154b60 0x7f6cf0154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.253+0000 7f6cf7136700 1 --2- 172.21.15.31:0/3814389348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6cf0153be0 0x7f6cf0154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.253+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 >> 172.21.15.31:0/3814389348 conn(0x7f6cf00c3190 msgr2=0x7f6cf00c2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.253+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 shutdown_connections 2024-03-30T01:41:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.253+0000 7f6cf7136700 1 -- 172.21.15.31:0/3814389348 wait complete. 2024-03-30T01:41:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:41:25.272 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-03-30T01:41:23.813518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219364,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755356,"statfs":{"total":773094113280,"available":772869484544,"internally_reserved":0,"allocated":3280896,"data_stored":1987195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011282"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965171+0000","last_change":"2024-03-30T01:40:58.441027+0000","last_active":"2024-03-30T01:41:02.965171+0000","last_peered":"2024-03-30T01:41:02.965171+0000","last_clean":"2024-03-30T01:41:02.965171+0000","last_became_active":"2024-03-30T01:40:58.440741+0000","last_became_peered":"2024-03-30T01:40:58.440741+0000","last_unstale":"2024-03-30T01:41:02.965171+0000","last_undegraded":"2024-03-30T01:41:02.965171+0000","last_fullsized":"2024-03-30T01:41:02.965171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868566+0000","last_change":"2024-03-30T01:40:58.426588+0000","last_active":"2024-03-30T01:41:02.868566+0000","last_peered":"2024-03-30T01:41:02.868566+0000","last_clean":"2024-03-30T01:41:02.868566+0000","last_became_active":"2024-03-30T01:40:58.426336+0000","last_became_peered":"2024-03-30T01:40:58.426336+0000","last_unstale":"2024-03-30T01:41:02.868566+0000","last_undegraded":"2024-03-30T01:41:02.868566+0000","last_fullsized":"2024-03-30T01:41:02.868566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869813+0000","last_change":"2024-03-30T01:40:58.434632+0000","last_active":"2024-03-30T01:41:02.869813+0000","last_peered":"2024-03-30T01:41:02.869813+0000","last_clean":"2024-03-30T01:41:02.869813+0000","last_became_active":"2024-03-30T01:40:58.434339+0000","last_became_peered":"2024-03-30T01:40:58.434339+0000","last_unstale":"2024-03-30T01:41:02.869813+0000","last_undegraded":"2024-03-30T01:41:02.869813+0000","last_fullsized":"2024-03-30T01:41:02.869813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:35.665153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868405+0000","last_change":"2024-03-30T01:40:58.428659+0000","last_active":"2024-03-30T01:41:02.868405+0000","last_peered":"2024-03-30T01:41:02.868405+0000","last_clean":"2024-03-30T01:41:02.868405+0000","last_became_active":"2024-03-30T01:40:58.426016+0000","last_became_peered":"2024-03-30T01:40:58.426016+0000","last_unstale":"2024-03-30T01:41:02.868405+0000","last_undegraded":"2024-03-30T01:41:02.868405+0000","last_fullsized":"2024-03-30T01:41:02.868405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964596+0000","last_change":"2024-03-30T01:40:58.440079+0000","last_active":"2024-03-30T01:41:02.964596+0000","last_peered":"2024-03-30T01:41:02.964596+0000","last_clean":"2024-03-30T01:41:02.964596+0000","last_became_active":"2024-03-30T01:40:58.439661+0000","last_became_peered":"2024-03-30T01:40:58.439661+0000","last_unstale":"2024-03-30T01:41:02.964596+0000","last_undegraded":"2024-03-30T01:41:02.964596+0000","last_fullsized":"2024-03-30T01:41:02.964596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868344+0000","last_change":"2024-03-30T01:40:58.426692+0000","last_active":"2024-03-30T01:41:02.868344+0000","last_peered":"2024-03-30T01:41:02.868344+0000","last_clean":"2024-03-30T01:41:02.868344+0000","last_became_active":"2024-03-30T01:40:58.426370+0000","last_became_peered":"2024-03-30T01:40:58.426370+0000","last_unstale":"2024-03-30T01:41:02.868344+0000","last_undegraded":"2024-03-30T01:41:02.868344+0000","last_fullsized":"2024-03-30T01:41:02.868344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:51.791864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964482+0000","last_change":"2024-03-30T01:40:58.440389+0000","last_active":"2024-03-30T01:41:02.964482+0000","last_peered":"2024-03-30T01:41:02.964482+0000","last_clean":"2024-03-30T01:41:02.964482+0000","last_became_active":"2024-03-30T01:40:58.439965+0000","last_became_peered":"2024-03-30T01:40:58.439965+0000","last_unstale":"2024-03-30T01:41:02.964482+0000","last_undegraded":"2024-03-30T01:41:02.964482+0000","last_fullsized":"2024-03-30T01:41:02.964482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:12:57.923292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868477+0000","last_change":"2024-03-30T01:40:58.429571+0000","last_active":"2024-03-30T01:41:02.868477+0000","last_peered":"2024-03-30T01:41:02.868477+0000","last_clean":"2024-03-30T01:41:02.868477+0000","last_became_active":"2024-03-30T01:40:58.429144+0000","last_became_peered":"2024-03-30T01:40:58.429144+0000","last_unstale":"2024-03-30T01:41:02.868477+0000","last_undegraded":"2024-03-30T01:41:02.868477+0000","last_fullsized":"2024-03-30T01:41:02.868477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965163+0000","last_change":"2024-03-30T01:40:58.441005+0000","last_active":"2024-03-30T01:41:02.965163+0000","last_peered":"2024-03-30T01:41:02.965163+0000","last_clean":"2024-03-30T01:41:02.965163+0000","last_became_active":"2024-03-30T01:40:58.440409+0000","last_became_peered":"2024-03-30T01:40:58.440409+0000","last_unstale":"2024-03-30T01:41:02.965163+0000","last_undegraded":"2024-03-30T01:41:02.965163+0000","last_fullsized":"2024-03-30T01:41:02.965163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:48:57.608199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872597+0000","last_change":"2024-03-30T01:40:58.429626+0000","last_active":"2024-03-30T01:41:02.872597+0000","last_peered":"2024-03-30T01:41:02.872597+0000","last_clean":"2024-03-30T01:41:02.872597+0000","last_became_active":"2024-03-30T01:40:58.428916+0000","last_became_peered":"2024-03-30T01:40:58.428916+0000","last_unstale":"2024-03-30T01:41:02.872597+0000","last_undegraded":"2024-03-30T01:41:02.872597+0000","last_fullsized":"2024-03-30T01:41:02.872597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872572+0000","last_change":"2024-03-30T01:40:58.430546+0000","last_active":"2024-03-30T01:41:02.872572+0000","last_peered":"2024-03-30T01:41:02.872572+0000","last_clean":"2024-03-30T01:41:02.872572+0000","last_became_active":"2024-03-30T01:40:58.430232+0000","last_became_peered":"2024-03-30T01:40:58.430232+0000","last_unstale":"2024-03-30T01:41:02.872572+0000","last_undegraded":"2024-03-30T01:41:02.872572+0000","last_fullsized":"2024-03-30T01:41:02.872572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864576+0000","last_change":"2024-03-30T01:40:58.422212+0000","last_active":"2024-03-30T01:41:02.864576+0000","last_peered":"2024-03-30T01:41:02.864576+0000","last_clean":"2024-03-30T01:41:02.864576+0000","last_became_active":"2024-03-30T01:40:58.421812+0000","last_became_peered":"2024-03-30T01:40:58.421812+0000","last_unstale":"2024-03-30T01:41:02.864576+0000","last_undegraded":"2024-03-30T01:41:02.864576+0000","last_fullsized":"2024-03-30T01:41:02.864576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:30.774569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868720+0000","last_change":"2024-03-30T01:40:58.418447+0000","last_active":"2024-03-30T01:41:02.868720+0000","last_peered":"2024-03-30T01:41:02.868720+0000","last_clean":"2024-03-30T01:41:02.868720+0000","last_became_active":"2024-03-30T01:40:58.418146+0000","last_became_peered":"2024-03-30T01:40:58.418146+0000","last_unstale":"2024-03-30T01:41:02.868720+0000","last_undegraded":"2024-03-30T01:41:02.868720+0000","last_fullsized":"2024-03-30T01:41:02.868720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869653+0000","last_change":"2024-03-30T01:40:58.430002+0000","last_active":"2024-03-30T01:41:02.869653+0000","last_peered":"2024-03-30T01:41:02.869653+0000","last_clean":"2024-03-30T01:41:02.869653+0000","last_became_active":"2024-03-30T01:40:58.429743+0000","last_became_peered":"2024-03-30T01:40:58.429743+0000","last_unstale":"2024-03-30T01:41:02.869653+0000","last_undegraded":"2024-03-30T01:41:02.869653+0000","last_fullsized":"2024-03-30T01:41:02.869653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:54.449729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868389+0000","last_change":"2024-03-30T01:40:58.425393+0000","last_active":"2024-03-30T01:41:02.868389+0000","last_peered":"2024-03-30T01:41:02.868389+0000","last_clean":"2024-03-30T01:41:02.868389+0000","last_became_active":"2024-03-30T01:40:58.424890+0000","last_became_peered":"2024-03-30T01:40:58.424890+0000","last_unstale":"2024-03-30T01:41:02.868389+0000","last_undegraded":"2024-03-30T01:41:02.868389+0000","last_fullsized":"2024-03-30T01:41:02.868389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868218+0000","last_change":"2024-03-30T01:40:58.426453+0000","last_active":"2024-03-30T01:41:02.868218+0000","last_peered":"2024-03-30T01:41:02.868218+0000","last_clean":"2024-03-30T01:41:02.868218+0000","last_became_active":"2024-03-30T01:40:58.425360+0000","last_became_peered":"2024-03-30T01:40:58.425360+0000","last_unstale":"2024-03-30T01:41:02.868218+0000","last_undegraded":"2024-03-30T01:41:02.868218+0000","last_fullsized":"2024-03-30T01:41:02.868218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:28.675589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868517+0000","last_change":"2024-03-30T01:40:58.433354+0000","last_active":"2024-03-30T01:41:02.868517+0000","last_peered":"2024-03-30T01:41:02.868517+0000","last_clean":"2024-03-30T01:41:02.868517+0000","last_became_active":"2024-03-30T01:40:58.433034+0000","last_became_peered":"2024-03-30T01:40:58.433034+0000","last_unstale":"2024-03-30T01:41:02.868517+0000","last_undegraded":"2024-03-30T01:41:02.868517+0000","last_fullsized":"2024-03-30T01:41:02.868517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:30.394391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868781+0000","last_change":"2024-03-30T01:40:58.433603+0000","last_active":"2024-03-30T01:41:02.868781+0000","last_peered":"2024-03-30T01:41:02.868781+0000","last_clean":"2024-03-30T01:41:02.868781+0000","last_became_active":"2024-03-30T01:40:58.433303+0000","last_became_peered":"2024-03-30T01:40:58.433303+0000","last_unstale":"2024-03-30T01:41:02.868781+0000","last_undegraded":"2024-03-30T01:41:02.868781+0000","last_fullsized":"2024-03-30T01:41:02.868781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:05:33.160354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965044+0000","last_change":"2024-03-30T01:40:58.436807+0000","last_active":"2024-03-30T01:41:02.965044+0000","last_peered":"2024-03-30T01:41:02.965044+0000","last_clean":"2024-03-30T01:41:02.965044+0000","last_became_active":"2024-03-30T01:40:58.436528+0000","last_became_peered":"2024-03-30T01:40:58.436528+0000","last_unstale":"2024-03-30T01:41:02.965044+0000","last_undegraded":"2024-03-30T01:41:02.965044+0000","last_fullsized":"2024-03-30T01:41:02.965044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:21.062997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972825+0000","last_change":"2024-03-30T01:41:02.970253+0000","last_active":"2024-03-30T01:41:02.972825+0000","last_peered":"2024-03-30T01:41:02.972825+0000","last_clean":"2024-03-30T01:41:02.972825+0000","last_became_active":"2024-03-30T01:41:02.969854+0000","last_became_peered":"2024-03-30T01:41:02.969854+0000","last_unstale":"2024-03-30T01:41:02.972825+0000","last_undegraded":"2024-03-30T01:41:02.972825+0000","last_fullsized":"2024-03-30T01:41:02.972825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:57:18.590955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.786878+0000","last_change":"2024-03-30T01:40:57.963998+0000","last_active":"2024-03-30T01:41:02.786878+0000","last_peered":"2024-03-30T01:41:02.786878+0000","last_clean":"2024-03-30T01:41:02.786878+0000","last_became_active":"2024-03-30T01:40:57.963535+0000","last_became_peered":"2024-03-30T01:40:57.963535+0000","last_unstale":"2024-03-30T01:41:02.786878+0000","last_undegraded":"2024-03-30T01:41:02.786878+0000","last_fullsized":"2024-03-30T01:41:02.786878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:40:49.324287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868907+0000","last_change":"2024-03-30T01:40:58.418336+0000","last_active":"2024-03-30T01:41:02.868907+0000","last_peered":"2024-03-30T01:41:02.868907+0000","last_clean":"2024-03-30T01:41:02.868907+0000","last_became_active":"2024-03-30T01:40:58.418057+0000","last_became_peered":"2024-03-30T01:40:58.418057+0000","last_unstale":"2024-03-30T01:41:02.868907+0000","last_undegraded":"2024-03-30T01:41:02.868907+0000","last_fullsized":"2024-03-30T01:41:02.868907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:52.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864399+0000","last_change":"2024-03-30T01:41:02.791579+0000","last_active":"2024-03-30T01:41:02.864399+0000","last_peered":"2024-03-30T01:41:02.864399+0000","last_clean":"2024-03-30T01:41:02.864399+0000","last_became_active":"2024-03-30T01:41:02.791102+0000","last_became_peered":"2024-03-30T01:41:02.791102+0000","last_unstale":"2024-03-30T01:41:02.864399+0000","last_undegraded":"2024-03-30T01:41:02.864399+0000","last_fullsized":"2024-03-30T01:41:02.864399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:34.215813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.776056+0000","last_change":"2024-03-30T01:40:57.963840+0000","last_active":"2024-03-30T01:41:02.776056+0000","last_peered":"2024-03-30T01:41:02.776056+0000","last_clean":"2024-03-30T01:41:02.776056+0000","last_became_active":"2024-03-30T01:40:57.962842+0000","last_became_peered":"2024-03-30T01:40:57.962842+0000","last_unstale":"2024-03-30T01:41:02.776056+0000","last_undegraded":"2024-03-30T01:41:02.776056+0000","last_fullsized":"2024-03-30T01:41:02.776056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869931+0000","last_change":"2024-03-30T01:40:58.435910+0000","last_active":"2024-03-30T01:41:02.869931+0000","last_peered":"2024-03-30T01:41:02.869931+0000","last_clean":"2024-03-30T01:41:02.869931+0000","last_became_active":"2024-03-30T01:40:58.435402+0000","last_became_peered":"2024-03-30T01:40:58.435402+0000","last_unstale":"2024-03-30T01:41:02.869931+0000","last_undegraded":"2024-03-30T01:41:02.869931+0000","last_fullsized":"2024-03-30T01:41:02.869931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864534+0000","last_change":"2024-03-30T01:41:02.789877+0000","last_active":"2024-03-30T01:41:02.864534+0000","last_peered":"2024-03-30T01:41:02.864534+0000","last_clean":"2024-03-30T01:41:02.864534+0000","last_became_active":"2024-03-30T01:41:02.789440+0000","last_became_peered":"2024-03-30T01:41:02.789440+0000","last_unstale":"2024-03-30T01:41:02.864534+0000","last_undegraded":"2024-03-30T01:41:02.864534+0000","last_fullsized":"2024-03-30T01:41:02.864534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:23.954119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964766+0000","last_change":"2024-03-30T01:40:57.968973+0000","last_active":"2024-03-30T01:41:02.964766+0000","last_peered":"2024-03-30T01:41:02.964766+0000","last_clean":"2024-03-30T01:41:02.964766+0000","last_became_active":"2024-03-30T01:40:57.968415+0000","last_became_peered":"2024-03-30T01:40:57.968415+0000","last_unstale":"2024-03-30T01:41:02.964766+0000","last_undegraded":"2024-03-30T01:41:02.964766+0000","last_fullsized":"2024-03-30T01:41:02.964766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868378+0000","last_change":"2024-03-30T01:40:58.431466+0000","last_active":"2024-03-30T01:41:02.868378+0000","last_peered":"2024-03-30T01:41:02.868378+0000","last_clean":"2024-03-30T01:41:02.868378+0000","last_became_active":"2024-03-30T01:40:58.431122+0000","last_became_peered":"2024-03-30T01:40:58.431122+0000","last_unstale":"2024-03-30T01:41:02.868378+0000","last_undegraded":"2024-03-30T01:41:02.868378+0000","last_fullsized":"2024-03-30T01:41:02.868378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:21.062997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864511+0000","last_change":"2024-03-30T01:41:02.789299+0000","last_active":"2024-03-30T01:41:02.864511+0000","last_peered":"2024-03-30T01:41:02.864511+0000","last_clean":"2024-03-30T01:41:02.864511+0000","last_became_active":"2024-03-30T01:41:02.788769+0000","last_became_peered":"2024-03-30T01:41:02.788769+0000","last_unstale":"2024-03-30T01:41:02.864511+0000","last_undegraded":"2024-03-30T01:41:02.864511+0000","last_fullsized":"2024-03-30T01:41:02.864511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:18.933245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964688+0000","last_change":"2024-03-30T01:40:57.970239+0000","last_active":"2024-03-30T01:41:02.964688+0000","last_peered":"2024-03-30T01:41:02.964688+0000","last_clean":"2024-03-30T01:41:02.964688+0000","last_became_active":"2024-03-30T01:40:57.969868+0000","last_became_peered":"2024-03-30T01:40:57.969868+0000","last_unstale":"2024-03-30T01:41:02.964688+0000","last_undegraded":"2024-03-30T01:41:02.964688+0000","last_fullsized":"2024-03-30T01:41:02.964688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:34.662924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868462+0000","last_change":"2024-03-30T01:40:58.431864+0000","last_active":"2024-03-30T01:41:02.868462+0000","last_peered":"2024-03-30T01:41:02.868462+0000","last_clean":"2024-03-30T01:41:02.868462+0000","last_became_active":"2024-03-30T01:40:58.431101+0000","last_became_peered":"2024-03-30T01:40:58.431101+0000","last_unstale":"2024-03-30T01:41:02.868462+0000","last_undegraded":"2024-03-30T01:41:02.868462+0000","last_fullsized":"2024-03-30T01:41:02.868462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:43.041425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868886+0000","last_change":"2024-03-30T01:41:02.797411+0000","last_active":"2024-03-30T01:41:02.868886+0000","last_peered":"2024-03-30T01:41:02.868886+0000","last_clean":"2024-03-30T01:41:02.868886+0000","last_became_active":"2024-03-30T01:41:02.797018+0000","last_became_peered":"2024-03-30T01:41:02.797018+0000","last_unstale":"2024-03-30T01:41:02.868886+0000","last_undegraded":"2024-03-30T01:41:02.868886+0000","last_fullsized":"2024-03-30T01:41:02.868886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:18:03.263600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.775328+0000","last_change":"2024-03-30T01:40:57.963372+0000","last_active":"2024-03-30T01:41:02.775328+0000","last_peered":"2024-03-30T01:41:02.775328+0000","last_clean":"2024-03-30T01:41:02.775328+0000","last_became_active":"2024-03-30T01:40:57.963033+0000","last_became_peered":"2024-03-30T01:40:57.963033+0000","last_unstale":"2024-03-30T01:41:02.775328+0000","last_undegraded":"2024-03-30T01:41:02.775328+0000","last_fullsized":"2024-03-30T01:41:02.775328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864523+0000","last_change":"2024-03-30T01:40:58.427338+0000","last_active":"2024-03-30T01:41:02.864523+0000","last_peered":"2024-03-30T01:41:02.864523+0000","last_clean":"2024-03-30T01:41:02.864523+0000","last_became_active":"2024-03-30T01:40:58.427023+0000","last_became_peered":"2024-03-30T01:40:58.427023+0000","last_unstale":"2024-03-30T01:41:02.864523+0000","last_undegraded":"2024-03-30T01:41:02.864523+0000","last_fullsized":"2024-03-30T01:41:02.864523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:35.665153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868735+0000","last_change":"2024-03-30T01:41:02.801387+0000","last_active":"2024-03-30T01:41:02.868735+0000","last_peered":"2024-03-30T01:41:02.868735+0000","last_clean":"2024-03-30T01:41:02.868735+0000","last_became_active":"2024-03-30T01:41:02.800907+0000","last_became_peered":"2024-03-30T01:41:02.800907+0000","last_unstale":"2024-03-30T01:41:02.868735+0000","last_undegraded":"2024-03-30T01:41:02.868735+0000","last_fullsized":"2024-03-30T01:41:02.868735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:43.218702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770418+0000","last_change":"2024-03-30T01:40:57.965707+0000","last_active":"2024-03-30T01:41:02.770418+0000","last_peered":"2024-03-30T01:41:02.770418+0000","last_clean":"2024-03-30T01:41:02.770418+0000","last_became_active":"2024-03-30T01:40:57.965371+0000","last_became_peered":"2024-03-30T01:40:57.965371+0000","last_unstale":"2024-03-30T01:41:02.770418+0000","last_undegraded":"2024-03-30T01:41:02.770418+0000","last_fullsized":"2024-03-30T01:41:02.770418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869759+0000","last_change":"2024-03-30T01:40:58.432647+0000","last_active":"2024-03-30T01:41:02.869759+0000","last_peered":"2024-03-30T01:41:02.869759+0000","last_clean":"2024-03-30T01:41:02.869759+0000","last_became_active":"2024-03-30T01:40:58.432292+0000","last_became_peered":"2024-03-30T01:40:58.432292+0000","last_unstale":"2024-03-30T01:41:02.869759+0000","last_undegraded":"2024-03-30T01:41:02.869759+0000","last_fullsized":"2024-03-30T01:41:02.869759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872608+0000","last_change":"2024-03-30T01:41:02.798539+0000","last_active":"2024-03-30T01:41:02.872608+0000","last_peered":"2024-03-30T01:41:02.872608+0000","last_clean":"2024-03-30T01:41:02.872608+0000","last_became_active":"2024-03-30T01:41:02.798146+0000","last_became_peered":"2024-03-30T01:41:02.798146+0000","last_unstale":"2024-03-30T01:41:02.872608+0000","last_undegraded":"2024-03-30T01:41:02.872608+0000","last_fullsized":"2024-03-30T01:41:02.872608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:19:18.933245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.773948+0000","last_change":"2024-03-30T01:40:57.959181+0000","last_active":"2024-03-30T01:41:02.773948+0000","last_peered":"2024-03-30T01:41:02.773948+0000","last_clean":"2024-03-30T01:41:02.773948+0000","last_became_active":"2024-03-30T01:40:57.958838+0000","last_became_peered":"2024-03-30T01:40:57.958838+0000","last_unstale":"2024-03-30T01:41:02.773948+0000","last_undegraded":"2024-03-30T01:41:02.773948+0000","last_fullsized":"2024-03-30T01:41:02.773948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:04.688535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868351+0000","last_change":"2024-03-30T01:40:58.429817+0000","last_active":"2024-03-30T01:41:02.868351+0000","last_peered":"2024-03-30T01:41:02.868351+0000","last_clean":"2024-03-30T01:41:02.868351+0000","last_became_active":"2024-03-30T01:40:58.429505+0000","last_became_peered":"2024-03-30T01:40:58.429505+0000","last_unstale":"2024-03-30T01:41:02.868351+0000","last_undegraded":"2024-03-30T01:41:02.868351+0000","last_fullsized":"2024-03-30T01:41:02.868351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:30:35.170155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.970928+0000","last_change":"2024-03-30T01:41:02.967897+0000","last_active":"2024-03-30T01:41:02.970928+0000","last_peered":"2024-03-30T01:41:02.970928+0000","last_clean":"2024-03-30T01:41:02.970928+0000","last_became_active":"2024-03-30T01:41:02.967524+0000","last_became_peered":"2024-03-30T01:41:02.967524+0000","last_unstale":"2024-03-30T01:41:02.970928+0000","last_undegraded":"2024-03-30T01:41:02.970928+0000","last_fullsized":"2024-03-30T01:41:02.970928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:53:58.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.778673+0000","last_change":"2024-03-30T01:40:57.964382+0000","last_active":"2024-03-30T01:41:02.778673+0000","last_peered":"2024-03-30T01:41:02.778673+0000","last_clean":"2024-03-30T01:41:02.778673+0000","last_became_active":"2024-03-30T01:40:57.963879+0000","last_became_peered":"2024-03-30T01:40:57.963879+0000","last_unstale":"2024-03-30T01:41:02.778673+0000","last_undegraded":"2024-03-30T01:41:02.778673+0000","last_fullsized":"2024-03-30T01:41:02.778673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868435+0000","last_change":"2024-03-30T01:40:58.425195+0000","last_active":"2024-03-30T01:41:02.868435+0000","last_peered":"2024-03-30T01:41:02.868435+0000","last_clean":"2024-03-30T01:41:02.868435+0000","last_became_active":"2024-03-30T01:40:58.424555+0000","last_became_peered":"2024-03-30T01:40:58.424555+0000","last_unstale":"2024-03-30T01:41:02.868435+0000","last_undegraded":"2024-03-30T01:41:02.868435+0000","last_fullsized":"2024-03-30T01:41:02.868435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972778+0000","last_change":"2024-03-30T01:41:02.968243+0000","last_active":"2024-03-30T01:41:02.972778+0000","last_peered":"2024-03-30T01:41:02.972778+0000","last_clean":"2024-03-30T01:41:02.972778+0000","last_became_active":"2024-03-30T01:41:02.967682+0000","last_became_peered":"2024-03-30T01:41:02.967682+0000","last_unstale":"2024-03-30T01:41:02.972778+0000","last_undegraded":"2024-03-30T01:41:02.972778+0000","last_fullsized":"2024-03-30T01:41:02.972778+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:40.634879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770699+0000","last_change":"2024-03-30T01:40:57.959485+0000","last_active":"2024-03-30T01:41:02.770699+0000","last_peered":"2024-03-30T01:41:02.770699+0000","last_clean":"2024-03-30T01:41:02.770699+0000","last_became_active":"2024-03-30T01:40:57.959180+0000","last_became_peered":"2024-03-30T01:40:57.959180+0000","last_unstale":"2024-03-30T01:41:02.770699+0000","last_undegraded":"2024-03-30T01:41:02.770699+0000","last_fullsized":"2024-03-30T01:41:02.770699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868505+0000","last_change":"2024-03-30T01:40:58.432524+0000","last_active":"2024-03-30T01:41:02.868505+0000","last_peered":"2024-03-30T01:41:02.868505+0000","last_clean":"2024-03-30T01:41:02.868505+0000","last_became_active":"2024-03-30T01:40:58.432225+0000","last_became_peered":"2024-03-30T01:40:58.432225+0000","last_unstale":"2024-03-30T01:41:02.868505+0000","last_undegraded":"2024-03-30T01:41:02.868505+0000","last_fullsized":"2024-03-30T01:41:02.868505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:30:35.170155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868329+0000","last_change":"2024-03-30T01:41:02.796380+0000","last_active":"2024-03-30T01:41:02.868329+0000","last_peered":"2024-03-30T01:41:02.868329+0000","last_clean":"2024-03-30T01:41:02.868329+0000","last_became_active":"2024-03-30T01:41:02.795984+0000","last_became_peered":"2024-03-30T01:41:02.795984+0000","last_unstale":"2024-03-30T01:41:02.868329+0000","last_undegraded":"2024-03-30T01:41:02.868329+0000","last_fullsized":"2024-03-30T01:41:02.868329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:13.399650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.781391+0000","last_change":"2024-03-30T01:40:57.958458+0000","last_active":"2024-03-30T01:41:02.781391+0000","last_peered":"2024-03-30T01:41:02.781391+0000","last_clean":"2024-03-30T01:41:02.781391+0000","last_became_active":"2024-03-30T01:40:57.957887+0000","last_became_peered":"2024-03-30T01:40:57.957887+0000","last_unstale":"2024-03-30T01:41:02.781391+0000","last_undegraded":"2024-03-30T01:41:02.781391+0000","last_fullsized":"2024-03-30T01:41:02.781391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869948+0000","last_change":"2024-03-30T01:40:58.436050+0000","last_active":"2024-03-30T01:41:02.869948+0000","last_peered":"2024-03-30T01:41:02.869948+0000","last_clean":"2024-03-30T01:41:02.869948+0000","last_became_active":"2024-03-30T01:40:58.435014+0000","last_became_peered":"2024-03-30T01:40:58.435014+0000","last_unstale":"2024-03-30T01:41:02.869948+0000","last_undegraded":"2024-03-30T01:41:02.869948+0000","last_fullsized":"2024-03-30T01:41:02.869948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:52.270432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864562+0000","last_change":"2024-03-30T01:41:02.784245+0000","last_active":"2024-03-30T01:41:02.864562+0000","last_peered":"2024-03-30T01:41:02.864562+0000","last_clean":"2024-03-30T01:41:02.864562+0000","last_became_active":"2024-03-30T01:41:02.783538+0000","last_became_peered":"2024-03-30T01:41:02.783538+0000","last_unstale":"2024-03-30T01:41:02.864562+0000","last_undegraded":"2024-03-30T01:41:02.864562+0000","last_fullsized":"2024-03-30T01:41:02.864562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:58.270483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.777921+0000","last_change":"2024-03-30T01:40:57.967602+0000","last_active":"2024-03-30T01:41:02.777921+0000","last_peered":"2024-03-30T01:41:02.777921+0000","last_clean":"2024-03-30T01:41:02.777921+0000","last_became_active":"2024-03-30T01:40:57.967239+0000","last_became_peered":"2024-03-30T01:40:57.967239+0000","last_unstale":"2024-03-30T01:41:02.777921+0000","last_undegraded":"2024-03-30T01:41:02.777921+0000","last_fullsized":"2024-03-30T01:41:02.777921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:36.274096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864600+0000","last_change":"2024-03-30T01:40:58.430901+0000","last_active":"2024-03-30T01:41:02.864600+0000","last_peered":"2024-03-30T01:41:02.864600+0000","last_clean":"2024-03-30T01:41:02.864600+0000","last_became_active":"2024-03-30T01:40:58.430595+0000","last_became_peered":"2024-03-30T01:40:58.430595+0000","last_unstale":"2024-03-30T01:41:02.864600+0000","last_undegraded":"2024-03-30T01:41:02.864600+0000","last_fullsized":"2024-03-30T01:41:02.864600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:37.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868468+0000","last_change":"2024-03-30T01:41:02.795440+0000","last_active":"2024-03-30T01:41:02.868468+0000","last_peered":"2024-03-30T01:41:02.868468+0000","last_clean":"2024-03-30T01:41:02.868468+0000","last_became_active":"2024-03-30T01:41:02.795052+0000","last_became_peered":"2024-03-30T01:41:02.795052+0000","last_unstale":"2024-03-30T01:41:02.868468+0000","last_undegraded":"2024-03-30T01:41:02.868468+0000","last_fullsized":"2024-03-30T01:41:02.868468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:13:23.954119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T01:41:13.464226+0000","last_change":"2024-03-30T01:40:57.966563+0000","last_active":"2024-03-30T01:41:13.464226+0000","last_peered":"2024-03-30T01:41:13.464226+0000","last_clean":"2024-03-30T01:41:13.464226+0000","last_became_active":"2024-03-30T01:40:57.966235+0000","last_became_peered":"2024-03-30T01:40:57.966235+0000","last_unstale":"2024-03-30T01:41:13.464226+0000","last_undegraded":"2024-03-30T01:41:13.464226+0000","last_fullsized":"2024-03-30T01:41:13.464226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:37:03.068287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964895+0000","last_change":"2024-03-30T01:40:58.436063+0000","last_active":"2024-03-30T01:41:02.964895+0000","last_peered":"2024-03-30T01:41:02.964895+0000","last_clean":"2024-03-30T01:41:02.964895+0000","last_became_active":"2024-03-30T01:40:58.435716+0000","last_became_peered":"2024-03-30T01:40:58.435716+0000","last_unstale":"2024-03-30T01:41:02.964895+0000","last_undegraded":"2024-03-30T01:41:02.964895+0000","last_fullsized":"2024-03-30T01:41:02.964895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:20:30.394391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.973943+0000","last_change":"2024-03-30T01:41:02.969718+0000","last_active":"2024-03-30T01:41:02.973943+0000","last_peered":"2024-03-30T01:41:02.973943+0000","last_clean":"2024-03-30T01:41:02.973943+0000","last_became_active":"2024-03-30T01:41:02.969246+0000","last_became_peered":"2024-03-30T01:41:02.969246+0000","last_unstale":"2024-03-30T01:41:02.973943+0000","last_undegraded":"2024-03-30T01:41:02.973943+0000","last_fullsized":"2024-03-30T01:41:02.973943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:34.173820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.786047+0000","last_change":"2024-03-30T01:40:57.971645+0000","last_active":"2024-03-30T01:41:02.786047+0000","last_peered":"2024-03-30T01:41:02.786047+0000","last_clean":"2024-03-30T01:41:02.786047+0000","last_became_active":"2024-03-30T01:40:57.971274+0000","last_became_peered":"2024-03-30T01:40:57.971274+0000","last_unstale":"2024-03-30T01:41:02.786047+0000","last_undegraded":"2024-03-30T01:41:02.786047+0000","last_fullsized":"2024-03-30T01:41:02.786047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:09.583893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869665+0000","last_change":"2024-03-30T01:40:58.431398+0000","last_active":"2024-03-30T01:41:02.869665+0000","last_peered":"2024-03-30T01:41:02.869665+0000","last_clean":"2024-03-30T01:41:02.869665+0000","last_became_active":"2024-03-30T01:40:58.431120+0000","last_became_peered":"2024-03-30T01:40:58.431120+0000","last_unstale":"2024-03-30T01:41:02.869665+0000","last_undegraded":"2024-03-30T01:41:02.869665+0000","last_fullsized":"2024-03-30T01:41:02.869665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968671+0000","last_change":"2024-03-30T01:41:02.966425+0000","last_active":"2024-03-30T01:41:02.968671+0000","last_peered":"2024-03-30T01:41:02.968671+0000","last_clean":"2024-03-30T01:41:02.968671+0000","last_became_active":"2024-03-30T01:41:02.965978+0000","last_became_peered":"2024-03-30T01:41:02.965978+0000","last_unstale":"2024-03-30T01:41:02.968671+0000","last_undegraded":"2024-03-30T01:41:02.968671+0000","last_fullsized":"2024-03-30T01:41:02.968671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780686+0000","last_change":"2024-03-30T01:40:57.956691+0000","last_active":"2024-03-30T01:41:02.780686+0000","last_peered":"2024-03-30T01:41:02.780686+0000","last_clean":"2024-03-30T01:41:02.780686+0000","last_became_active":"2024-03-30T01:40:57.956387+0000","last_became_peered":"2024-03-30T01:40:57.956387+0000","last_unstale":"2024-03-30T01:41:02.780686+0000","last_undegraded":"2024-03-30T01:41:02.780686+0000","last_fullsized":"2024-03-30T01:41:02.780686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964728+0000","last_change":"2024-03-30T01:40:58.435147+0000","last_active":"2024-03-30T01:41:02.964728+0000","last_peered":"2024-03-30T01:41:02.964728+0000","last_clean":"2024-03-30T01:41:02.964728+0000","last_became_active":"2024-03-30T01:40:58.434835+0000","last_became_peered":"2024-03-30T01:40:58.434835+0000","last_unstale":"2024-03-30T01:41:02.964728+0000","last_undegraded":"2024-03-30T01:41:02.964728+0000","last_fullsized":"2024-03-30T01:41:02.964728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:43.041425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972786+0000","last_change":"2024-03-30T01:41:02.968714+0000","last_active":"2024-03-30T01:41:02.972786+0000","last_peered":"2024-03-30T01:41:02.972786+0000","last_clean":"2024-03-30T01:41:02.972786+0000","last_became_active":"2024-03-30T01:41:02.967728+0000","last_became_peered":"2024-03-30T01:41:02.967728+0000","last_unstale":"2024-03-30T01:41:02.972786+0000","last_undegraded":"2024-03-30T01:41:02.972786+0000","last_fullsized":"2024-03-30T01:41:02.972786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:57:35.718883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770059+0000","last_change":"2024-03-30T01:40:57.964975+0000","last_active":"2024-03-30T01:41:02.770059+0000","last_peered":"2024-03-30T01:41:02.770059+0000","last_clean":"2024-03-30T01:41:02.770059+0000","last_became_active":"2024-03-30T01:40:57.964458+0000","last_became_peered":"2024-03-30T01:40:57.964458+0000","last_unstale":"2024-03-30T01:41:02.770059+0000","last_undegraded":"2024-03-30T01:41:02.770059+0000","last_fullsized":"2024-03-30T01:41:02.770059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872388+0000","last_change":"2024-03-30T01:40:58.425912+0000","last_active":"2024-03-30T01:41:02.872388+0000","last_peered":"2024-03-30T01:41:02.872388+0000","last_clean":"2024-03-30T01:41:02.872388+0000","last_became_active":"2024-03-30T01:40:58.425596+0000","last_became_peered":"2024-03-30T01:40:58.425596+0000","last_unstale":"2024-03-30T01:41:02.872388+0000","last_undegraded":"2024-03-30T01:41:02.872388+0000","last_fullsized":"2024-03-30T01:41:02.872388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869825+0000","last_change":"2024-03-30T01:41:02.793602+0000","last_active":"2024-03-30T01:41:02.869825+0000","last_peered":"2024-03-30T01:41:02.869825+0000","last_clean":"2024-03-30T01:41:02.869825+0000","last_became_active":"2024-03-30T01:41:02.793146+0000","last_became_peered":"2024-03-30T01:41:02.793146+0000","last_unstale":"2024-03-30T01:41:02.869825+0000","last_undegraded":"2024-03-30T01:41:02.869825+0000","last_fullsized":"2024-03-30T01:41:02.869825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782115+0000","last_change":"2024-03-30T01:40:57.962830+0000","last_active":"2024-03-30T01:41:02.782115+0000","last_peered":"2024-03-30T01:41:02.782115+0000","last_clean":"2024-03-30T01:41:02.782115+0000","last_became_active":"2024-03-30T01:40:57.961073+0000","last_became_peered":"2024-03-30T01:40:57.961073+0000","last_unstale":"2024-03-30T01:41:02.782115+0000","last_undegraded":"2024-03-30T01:41:02.782115+0000","last_fullsized":"2024-03-30T01:41:02.782115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964632+0000","last_change":"2024-03-30T01:40:58.435030+0000","last_active":"2024-03-30T01:41:02.964632+0000","last_peered":"2024-03-30T01:41:02.964632+0000","last_clean":"2024-03-30T01:41:02.964632+0000","last_became_active":"2024-03-30T01:40:58.434660+0000","last_became_peered":"2024-03-30T01:40:58.434660+0000","last_unstale":"2024-03-30T01:41:02.964632+0000","last_undegraded":"2024-03-30T01:41:02.964632+0000","last_fullsized":"2024-03-30T01:41:02.964632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:37.385758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864431+0000","last_change":"2024-03-30T01:41:02.796839+0000","last_active":"2024-03-30T01:41:02.864431+0000","last_peered":"2024-03-30T01:41:02.864431+0000","last_clean":"2024-03-30T01:41:02.864431+0000","last_became_active":"2024-03-30T01:41:02.796400+0000","last_became_peered":"2024-03-30T01:41:02.796400+0000","last_unstale":"2024-03-30T01:41:02.864431+0000","last_undegraded":"2024-03-30T01:41:02.864431+0000","last_fullsized":"2024-03-30T01:41:02.864431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:06:05.412311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.775105+0000","last_change":"2024-03-30T01:40:57.963776+0000","last_active":"2024-03-30T01:41:02.775105+0000","last_peered":"2024-03-30T01:41:02.775105+0000","last_clean":"2024-03-30T01:41:02.775105+0000","last_became_active":"2024-03-30T01:40:57.963473+0000","last_became_peered":"2024-03-30T01:40:57.963473+0000","last_unstale":"2024-03-30T01:41:02.775105+0000","last_undegraded":"2024-03-30T01:41:02.775105+0000","last_fullsized":"2024-03-30T01:41:02.775105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:09.583893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964956+0000","last_change":"2024-03-30T01:40:58.440136+0000","last_active":"2024-03-30T01:41:02.964956+0000","last_peered":"2024-03-30T01:41:02.964956+0000","last_clean":"2024-03-30T01:41:02.964956+0000","last_became_active":"2024-03-30T01:40:58.439831+0000","last_became_peered":"2024-03-30T01:40:58.439831+0000","last_unstale":"2024-03-30T01:41:02.964956+0000","last_undegraded":"2024-03-30T01:41:02.964956+0000","last_fullsized":"2024-03-30T01:41:02.964956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:57:28.675589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868455+0000","last_change":"2024-03-30T01:41:02.790100+0000","last_active":"2024-03-30T01:41:02.868455+0000","last_peered":"2024-03-30T01:41:02.868455+0000","last_clean":"2024-03-30T01:41:02.868455+0000","last_became_active":"2024-03-30T01:41:02.789660+0000","last_became_peered":"2024-03-30T01:41:02.789660+0000","last_unstale":"2024-03-30T01:41:02.868455+0000","last_undegraded":"2024-03-30T01:41:02.868455+0000","last_fullsized":"2024-03-30T01:41:02.868455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:22:34.848382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774905+0000","last_change":"2024-03-30T01:40:57.962434+0000","last_active":"2024-03-30T01:41:02.774905+0000","last_peered":"2024-03-30T01:41:02.774905+0000","last_clean":"2024-03-30T01:41:02.774905+0000","last_became_active":"2024-03-30T01:40:57.961987+0000","last_became_peered":"2024-03-30T01:40:57.961987+0000","last_unstale":"2024-03-30T01:41:02.774905+0000","last_undegraded":"2024-03-30T01:41:02.774905+0000","last_fullsized":"2024-03-30T01:41:02.774905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:42.056286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868476+0000","last_change":"2024-03-30T01:40:58.419381+0000","last_active":"2024-03-30T01:41:02.868476+0000","last_peered":"2024-03-30T01:41:02.868476+0000","last_clean":"2024-03-30T01:41:02.868476+0000","last_became_active":"2024-03-30T01:40:58.419079+0000","last_became_peered":"2024-03-30T01:40:58.419079+0000","last_unstale":"2024-03-30T01:41:02.868476+0000","last_undegraded":"2024-03-30T01:41:02.868476+0000","last_fullsized":"2024-03-30T01:41:02.868476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868479+0000","last_change":"2024-03-30T01:41:02.795347+0000","last_active":"2024-03-30T01:41:02.868479+0000","last_peered":"2024-03-30T01:41:02.868479+0000","last_clean":"2024-03-30T01:41:02.868479+0000","last_became_active":"2024-03-30T01:41:02.794998+0000","last_became_peered":"2024-03-30T01:41:02.794998+0000","last_unstale":"2024-03-30T01:41:02.868479+0000","last_undegraded":"2024-03-30T01:41:02.868479+0000","last_fullsized":"2024-03-30T01:41:02.868479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:19.384530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780572+0000","last_change":"2024-03-30T01:40:57.967506+0000","last_active":"2024-03-30T01:41:02.780572+0000","last_peered":"2024-03-30T01:41:02.780572+0000","last_clean":"2024-03-30T01:41:02.780572+0000","last_became_active":"2024-03-30T01:40:57.967219+0000","last_became_peered":"2024-03-30T01:40:57.967219+0000","last_unstale":"2024-03-30T01:41:02.780572+0000","last_undegraded":"2024-03-30T01:41:02.780572+0000","last_fullsized":"2024-03-30T01:41:02.780572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864547+0000","last_change":"2024-03-30T01:40:58.434945+0000","last_active":"2024-03-30T01:41:02.864547+0000","last_peered":"2024-03-30T01:41:02.864547+0000","last_clean":"2024-03-30T01:41:02.864547+0000","last_became_active":"2024-03-30T01:40:58.434344+0000","last_became_peered":"2024-03-30T01:40:58.434344+0000","last_unstale":"2024-03-30T01:41:02.864547+0000","last_undegraded":"2024-03-30T01:41:02.864547+0000","last_fullsized":"2024-03-30T01:41:02.864547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:43.058516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868577+0000","last_change":"2024-03-30T01:41:02.796689+0000","last_active":"2024-03-30T01:41:02.868577+0000","last_peered":"2024-03-30T01:41:02.868577+0000","last_clean":"2024-03-30T01:41:02.868577+0000","last_became_active":"2024-03-30T01:41:02.796285+0000","last_became_peered":"2024-03-30T01:41:02.796285+0000","last_unstale":"2024-03-30T01:41:02.868577+0000","last_undegraded":"2024-03-30T01:41:02.868577+0000","last_fullsized":"2024-03-30T01:41:02.868577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868364+0000","last_change":"2024-03-30T01:40:58.426059+0000","last_active":"2024-03-30T01:41:02.868364+0000","last_peered":"2024-03-30T01:41:02.868364+0000","last_clean":"2024-03-30T01:41:02.868364+0000","last_became_active":"2024-03-30T01:40:58.424722+0000","last_became_peered":"2024-03-30T01:40:58.424722+0000","last_unstale":"2024-03-30T01:41:02.868364+0000","last_undegraded":"2024-03-30T01:41:02.868364+0000","last_fullsized":"2024-03-30T01:41:02.868364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782192+0000","last_change":"2024-03-30T01:40:57.964575+0000","last_active":"2024-03-30T01:41:02.782192+0000","last_peered":"2024-03-30T01:41:02.782192+0000","last_clean":"2024-03-30T01:41:02.782192+0000","last_became_active":"2024-03-30T01:40:57.962016+0000","last_became_peered":"2024-03-30T01:40:57.962016+0000","last_unstale":"2024-03-30T01:41:02.782192+0000","last_undegraded":"2024-03-30T01:41:02.782192+0000","last_fullsized":"2024-03-30T01:41:02.782192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:04.688535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.974026+0000","last_change":"2024-03-30T01:41:02.969088+0000","last_active":"2024-03-30T01:41:02.974026+0000","last_peered":"2024-03-30T01:41:02.974026+0000","last_clean":"2024-03-30T01:41:02.974026+0000","last_became_active":"2024-03-30T01:41:02.968624+0000","last_became_peered":"2024-03-30T01:41:02.968624+0000","last_unstale":"2024-03-30T01:41:02.974026+0000","last_undegraded":"2024-03-30T01:41:02.974026+0000","last_fullsized":"2024-03-30T01:41:02.974026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:34.265123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868700+0000","last_change":"2024-03-30T01:40:58.429787+0000","last_active":"2024-03-30T01:41:02.868700+0000","last_peered":"2024-03-30T01:41:02.868700+0000","last_clean":"2024-03-30T01:41:02.868700+0000","last_became_active":"2024-03-30T01:40:58.429037+0000","last_became_peered":"2024-03-30T01:40:58.429037+0000","last_unstale":"2024-03-30T01:41:02.868700+0000","last_undegraded":"2024-03-30T01:41:02.868700+0000","last_fullsized":"2024-03-30T01:41:02.868700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:54.449729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.780784+0000","last_change":"2024-03-30T01:40:57.954275+0000","last_active":"2024-03-30T01:41:02.780784+0000","last_peered":"2024-03-30T01:41:02.780784+0000","last_clean":"2024-03-30T01:41:02.780784+0000","last_became_active":"2024-03-30T01:40:57.953987+0000","last_became_peered":"2024-03-30T01:40:57.953987+0000","last_unstale":"2024-03-30T01:41:02.780784+0000","last_undegraded":"2024-03-30T01:41:02.780784+0000","last_fullsized":"2024-03-30T01:41:02.780784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964197+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:41:02.964197+0000","last_peered":"2024-03-30T01:41:02.964197+0000","last_clean":"2024-03-30T01:41:02.964197+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:41:02.964197+0000","last_undegraded":"2024-03-30T01:41:02.964197+0000","last_fullsized":"2024-03-30T01:41:02.964197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.972468+0000","last_change":"2024-03-30T01:41:02.969064+0000","last_active":"2024-03-30T01:41:02.972468+0000","last_peered":"2024-03-30T01:41:02.972468+0000","last_clean":"2024-03-30T01:41:02.972468+0000","last_became_active":"2024-03-30T01:41:02.968333+0000","last_became_peered":"2024-03-30T01:41:02.968333+0000","last_unstale":"2024-03-30T01:41:02.972468+0000","last_undegraded":"2024-03-30T01:41:02.972468+0000","last_fullsized":"2024-03-30T01:41:02.972468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:18:34.173820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869624+0000","last_change":"2024-03-30T01:40:58.432432+0000","last_active":"2024-03-30T01:41:02.869624+0000","last_peered":"2024-03-30T01:41:02.869624+0000","last_clean":"2024-03-30T01:41:02.869624+0000","last_became_active":"2024-03-30T01:40:58.432158+0000","last_became_peered":"2024-03-30T01:40:58.432158+0000","last_unstale":"2024-03-30T01:41:02.869624+0000","last_undegraded":"2024-03-30T01:41:02.869624+0000","last_fullsized":"2024-03-30T01:41:02.869624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964478+0000","last_change":"2024-03-30T01:40:57.966931+0000","last_active":"2024-03-30T01:41:02.964478+0000","last_peered":"2024-03-30T01:41:02.964478+0000","last_clean":"2024-03-30T01:41:02.964478+0000","last_became_active":"2024-03-30T01:40:57.966589+0000","last_became_peered":"2024-03-30T01:40:57.966589+0000","last_unstale":"2024-03-30T01:41:02.964478+0000","last_undegraded":"2024-03-30T01:41:02.964478+0000","last_fullsized":"2024-03-30T01:41:02.964478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:15.934995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968515+0000","last_change":"2024-03-30T01:41:02.966273+0000","last_active":"2024-03-30T01:41:02.968515+0000","last_peered":"2024-03-30T01:41:02.968515+0000","last_clean":"2024-03-30T01:41:02.968515+0000","last_became_active":"2024-03-30T01:41:02.965857+0000","last_became_peered":"2024-03-30T01:41:02.965857+0000","last_unstale":"2024-03-30T01:41:02.968515+0000","last_undegraded":"2024-03-30T01:41:02.968515+0000","last_fullsized":"2024-03-30T01:41:02.968515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:52:58.270483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964736+0000","last_change":"2024-03-30T01:40:57.964957+0000","last_active":"2024-03-30T01:41:02.964736+0000","last_peered":"2024-03-30T01:41:02.964736+0000","last_clean":"2024-03-30T01:41:02.964736+0000","last_became_active":"2024-03-30T01:40:57.963985+0000","last_became_peered":"2024-03-30T01:40:57.963985+0000","last_unstale":"2024-03-30T01:41:02.964736+0000","last_undegraded":"2024-03-30T01:41:02.964736+0000","last_fullsized":"2024-03-30T01:41:02.964736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:05:32.158124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868496+0000","last_change":"2024-03-30T01:40:58.425295+0000","last_active":"2024-03-30T01:41:02.868496+0000","last_peered":"2024-03-30T01:41:02.868496+0000","last_clean":"2024-03-30T01:41:02.868496+0000","last_became_active":"2024-03-30T01:40:58.424967+0000","last_became_peered":"2024-03-30T01:40:58.424967+0000","last_unstale":"2024-03-30T01:41:02.868496+0000","last_undegraded":"2024-03-30T01:41:02.868496+0000","last_fullsized":"2024-03-30T01:41:02.868496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.975320+0000","last_change":"2024-03-30T01:41:02.970612+0000","last_active":"2024-03-30T01:41:02.975320+0000","last_peered":"2024-03-30T01:41:02.975320+0000","last_clean":"2024-03-30T01:41:02.975320+0000","last_became_active":"2024-03-30T01:41:02.969823+0000","last_became_peered":"2024-03-30T01:41:02.969823+0000","last_unstale":"2024-03-30T01:41:02.975320+0000","last_undegraded":"2024-03-30T01:41:02.975320+0000","last_fullsized":"2024-03-30T01:41:02.975320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:25:35.463484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-30T01:41:03.568727+0000","last_change":"2024-03-30T01:40:57.968553+0000","last_active":"2024-03-30T01:41:03.568727+0000","last_peered":"2024-03-30T01:41:03.568727+0000","last_clean":"2024-03-30T01:41:03.568727+0000","last_became_active":"2024-03-30T01:40:57.968315+0000","last_became_peered":"2024-03-30T01:40:57.968315+0000","last_unstale":"2024-03-30T01:41:03.568727+0000","last_undegraded":"2024-03-30T01:41:03.568727+0000","last_fullsized":"2024-03-30T01:41:03.568727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:02:34.662924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869030+0000","last_change":"2024-03-30T01:40:58.429594+0000","last_active":"2024-03-30T01:41:02.869030+0000","last_peered":"2024-03-30T01:41:02.869030+0000","last_clean":"2024-03-30T01:41:02.869030+0000","last_became_active":"2024-03-30T01:40:58.429296+0000","last_became_peered":"2024-03-30T01:40:58.429296+0000","last_unstale":"2024-03-30T01:41:02.869030+0000","last_undegraded":"2024-03-30T01:41:02.869030+0000","last_fullsized":"2024-03-30T01:41:02.869030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.974419+0000","last_change":"2024-03-30T01:41:02.970239+0000","last_active":"2024-03-30T01:41:02.974419+0000","last_peered":"2024-03-30T01:41:02.974419+0000","last_clean":"2024-03-30T01:41:02.974419+0000","last_became_active":"2024-03-30T01:41:02.969823+0000","last_became_peered":"2024-03-30T01:41:02.969823+0000","last_unstale":"2024-03-30T01:41:02.974419+0000","last_undegraded":"2024-03-30T01:41:02.974419+0000","last_fullsized":"2024-03-30T01:41:02.974419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:40:00.285894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.770593+0000","last_change":"2024-03-30T01:40:57.963709+0000","last_active":"2024-03-30T01:41:02.770593+0000","last_peered":"2024-03-30T01:41:02.770593+0000","last_clean":"2024-03-30T01:41:02.770593+0000","last_became_active":"2024-03-30T01:40:57.963374+0000","last_became_peered":"2024-03-30T01:40:57.963374+0000","last_unstale":"2024-03-30T01:41:02.770593+0000","last_undegraded":"2024-03-30T01:41:02.770593+0000","last_fullsized":"2024-03-30T01:41:02.770593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869920+0000","last_change":"2024-03-30T01:40:58.430636+0000","last_active":"2024-03-30T01:41:02.869920+0000","last_peered":"2024-03-30T01:41:02.869920+0000","last_clean":"2024-03-30T01:41:02.869920+0000","last_became_active":"2024-03-30T01:40:58.430400+0000","last_became_peered":"2024-03-30T01:40:58.430400+0000","last_unstale":"2024-03-30T01:41:02.869920+0000","last_undegraded":"2024-03-30T01:41:02.869920+0000","last_fullsized":"2024-03-30T01:41:02.869920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:44:58.613670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864624+0000","last_change":"2024-03-30T01:41:02.788933+0000","last_active":"2024-03-30T01:41:02.864624+0000","last_peered":"2024-03-30T01:41:02.864624+0000","last_clean":"2024-03-30T01:41:02.864624+0000","last_became_active":"2024-03-30T01:41:02.788466+0000","last_became_peered":"2024-03-30T01:41:02.788466+0000","last_unstale":"2024-03-30T01:41:02.864624+0000","last_undegraded":"2024-03-30T01:41:02.864624+0000","last_fullsized":"2024-03-30T01:41:02.864624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964182+0000","last_change":"2024-03-30T01:40:57.962449+0000","last_active":"2024-03-30T01:41:02.964182+0000","last_peered":"2024-03-30T01:41:02.964182+0000","last_clean":"2024-03-30T01:41:02.964182+0000","last_became_active":"2024-03-30T01:40:57.962192+0000","last_became_peered":"2024-03-30T01:40:57.962192+0000","last_unstale":"2024-03-30T01:41:02.964182+0000","last_undegraded":"2024-03-30T01:41:02.964182+0000","last_fullsized":"2024-03-30T01:41:02.964182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:36.274096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864497+0000","last_change":"2024-03-30T01:40:58.420796+0000","last_active":"2024-03-30T01:41:02.864497+0000","last_peered":"2024-03-30T01:41:02.864497+0000","last_clean":"2024-03-30T01:41:02.864497+0000","last_became_active":"2024-03-30T01:40:58.420441+0000","last_became_peered":"2024-03-30T01:40:58.420441+0000","last_unstale":"2024-03-30T01:41:02.864497+0000","last_undegraded":"2024-03-30T01:41:02.864497+0000","last_fullsized":"2024-03-30T01:41:02.864497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:16.937225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.969938+0000","last_change":"2024-03-30T01:41:02.966462+0000","last_active":"2024-03-30T01:41:02.969938+0000","last_peered":"2024-03-30T01:41:02.969938+0000","last_clean":"2024-03-30T01:41:02.969938+0000","last_became_active":"2024-03-30T01:41:02.966056+0000","last_became_peered":"2024-03-30T01:41:02.966056+0000","last_unstale":"2024-03-30T01:41:02.969938+0000","last_undegraded":"2024-03-30T01:41:02.969938+0000","last_fullsized":"2024-03-30T01:41:02.969938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:29.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774672+0000","last_change":"2024-03-30T01:40:57.967648+0000","last_active":"2024-03-30T01:41:02.774672+0000","last_peered":"2024-03-30T01:41:02.774672+0000","last_clean":"2024-03-30T01:41:02.774672+0000","last_became_active":"2024-03-30T01:40:57.967362+0000","last_became_peered":"2024-03-30T01:40:57.967362+0000","last_unstale":"2024-03-30T01:41:02.774672+0000","last_undegraded":"2024-03-30T01:41:02.774672+0000","last_fullsized":"2024-03-30T01:41:02.774672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:33:42.276774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868291+0000","last_change":"2024-03-30T01:40:58.432296+0000","last_active":"2024-03-30T01:41:02.868291+0000","last_peered":"2024-03-30T01:41:02.868291+0000","last_clean":"2024-03-30T01:41:02.868291+0000","last_became_active":"2024-03-30T01:40:58.432045+0000","last_became_peered":"2024-03-30T01:40:58.432045+0000","last_unstale":"2024-03-30T01:41:02.868291+0000","last_undegraded":"2024-03-30T01:41:02.868291+0000","last_fullsized":"2024-03-30T01:41:02.868291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864586+0000","last_change":"2024-03-30T01:41:02.796892+0000","last_active":"2024-03-30T01:41:02.864586+0000","last_peered":"2024-03-30T01:41:02.864586+0000","last_clean":"2024-03-30T01:41:02.864586+0000","last_became_active":"2024-03-30T01:41:02.796347+0000","last_became_peered":"2024-03-30T01:41:02.796347+0000","last_unstale":"2024-03-30T01:41:02.864586+0000","last_undegraded":"2024-03-30T01:41:02.864586+0000","last_fullsized":"2024-03-30T01:41:02.864586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:20.847856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.774635+0000","last_change":"2024-03-30T01:40:57.969391+0000","last_active":"2024-03-30T01:41:02.774635+0000","last_peered":"2024-03-30T01:41:02.774635+0000","last_clean":"2024-03-30T01:41:02.774635+0000","last_became_active":"2024-03-30T01:40:57.969107+0000","last_became_peered":"2024-03-30T01:40:57.969107+0000","last_unstale":"2024-03-30T01:41:02.774635+0000","last_undegraded":"2024-03-30T01:41:02.774635+0000","last_fullsized":"2024-03-30T01:41:02.774635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:29.772340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868769+0000","last_change":"2024-03-30T01:40:58.430179+0000","last_active":"2024-03-30T01:41:02.868769+0000","last_peered":"2024-03-30T01:41:02.868769+0000","last_clean":"2024-03-30T01:41:02.868769+0000","last_became_active":"2024-03-30T01:40:58.429919+0000","last_became_peered":"2024-03-30T01:40:58.429919+0000","last_unstale":"2024-03-30T01:41:02.868769+0000","last_undegraded":"2024-03-30T01:41:02.868769+0000","last_fullsized":"2024-03-30T01:41:02.868769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.864611+0000","last_change":"2024-03-30T01:41:02.792167+0000","last_active":"2024-03-30T01:41:02.864611+0000","last_peered":"2024-03-30T01:41:02.864611+0000","last_clean":"2024-03-30T01:41:02.864611+0000","last_became_active":"2024-03-30T01:41:02.791805+0000","last_became_peered":"2024-03-30T01:41:02.791805+0000","last_unstale":"2024-03-30T01:41:02.864611+0000","last_undegraded":"2024-03-30T01:41:02.864611+0000","last_fullsized":"2024-03-30T01:41:02.864611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:53.733547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964766+0000","last_change":"2024-03-30T01:40:57.964733+0000","last_active":"2024-03-30T01:41:02.964766+0000","last_peered":"2024-03-30T01:41:02.964766+0000","last_clean":"2024-03-30T01:41:02.964766+0000","last_became_active":"2024-03-30T01:40:57.964430+0000","last_became_peered":"2024-03-30T01:40:57.964430+0000","last_unstale":"2024-03-30T01:41:02.964766+0000","last_undegraded":"2024-03-30T01:41:02.964766+0000","last_fullsized":"2024-03-30T01:41:02.964766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:42:42.056286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872480+0000","last_change":"2024-03-30T01:40:58.420811+0000","last_active":"2024-03-30T01:41:02.872480+0000","last_peered":"2024-03-30T01:41:02.872480+0000","last_clean":"2024-03-30T01:41:02.872480+0000","last_became_active":"2024-03-30T01:40:58.420491+0000","last_became_peered":"2024-03-30T01:40:58.420491+0000","last_unstale":"2024-03-30T01:41:02.872480+0000","last_undegraded":"2024-03-30T01:41:02.872480+0000","last_fullsized":"2024-03-30T01:41:02.872480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.968342+0000","last_change":"2024-03-30T01:41:02.966275+0000","last_active":"2024-03-30T01:41:02.968342+0000","last_peered":"2024-03-30T01:41:02.968342+0000","last_clean":"2024-03-30T01:41:02.968342+0000","last_became_active":"2024-03-30T01:41:02.965934+0000","last_became_peered":"2024-03-30T01:41:02.965934+0000","last_unstale":"2024-03-30T01:41:02.968342+0000","last_undegraded":"2024-03-30T01:41:02.968342+0000","last_fullsized":"2024-03-30T01:41:02.968342+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:39.954419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.783325+0000","last_change":"2024-03-30T01:40:57.962835+0000","last_active":"2024-03-30T01:41:02.783325+0000","last_peered":"2024-03-30T01:41:02.783325+0000","last_clean":"2024-03-30T01:41:02.783325+0000","last_became_active":"2024-03-30T01:40:57.961552+0000","last_became_peered":"2024-03-30T01:40:57.961552+0000","last_unstale":"2024-03-30T01:41:02.783325+0000","last_undegraded":"2024-03-30T01:41:02.783325+0000","last_fullsized":"2024-03-30T01:41:02.783325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:36.399345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.964874+0000","last_change":"2024-03-30T01:40:58.432622+0000","last_active":"2024-03-30T01:41:02.964874+0000","last_peered":"2024-03-30T01:41:02.964874+0000","last_clean":"2024-03-30T01:41:02.964874+0000","last_became_active":"2024-03-30T01:40:58.432336+0000","last_became_peered":"2024-03-30T01:40:58.432336+0000","last_unstale":"2024-03-30T01:41:02.964874+0000","last_undegraded":"2024-03-30T01:41:02.964874+0000","last_fullsized":"2024-03-30T01:41:02.964874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:24:50.793081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869687+0000","last_change":"2024-03-30T01:41:02.799702+0000","last_active":"2024-03-30T01:41:02.869687+0000","last_peered":"2024-03-30T01:41:02.869687+0000","last_clean":"2024-03-30T01:41:02.869687+0000","last_became_active":"2024-03-30T01:41:02.799189+0000","last_became_peered":"2024-03-30T01:41:02.799189+0000","last_unstale":"2024-03-30T01:41:02.869687+0000","last_undegraded":"2024-03-30T01:41:02.869687+0000","last_fullsized":"2024-03-30T01:41:02.869687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:06:50.244168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.777920+0000","last_change":"2024-03-30T01:40:57.964577+0000","last_active":"2024-03-30T01:41:02.777920+0000","last_peered":"2024-03-30T01:41:02.777920+0000","last_clean":"2024-03-30T01:41:02.777920+0000","last_became_active":"2024-03-30T01:40:57.964277+0000","last_became_peered":"2024-03-30T01:40:57.964277+0000","last_unstale":"2024-03-30T01:41:02.777920+0000","last_undegraded":"2024-03-30T01:41:02.777920+0000","last_fullsized":"2024-03-30T01:41:02.777920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:15.934995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868392+0000","last_change":"2024-03-30T01:40:58.428923+0000","last_active":"2024-03-30T01:41:02.868392+0000","last_peered":"2024-03-30T01:41:02.868392+0000","last_clean":"2024-03-30T01:41:02.868392+0000","last_became_active":"2024-03-30T01:40:58.428204+0000","last_became_peered":"2024-03-30T01:40:58.428204+0000","last_unstale":"2024-03-30T01:41:02.868392+0000","last_undegraded":"2024-03-30T01:41:02.868392+0000","last_fullsized":"2024-03-30T01:41:02.868392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.970293+0000","last_change":"2024-03-30T01:41:02.968043+0000","last_active":"2024-03-30T01:41:02.970293+0000","last_peered":"2024-03-30T01:41:02.970293+0000","last_clean":"2024-03-30T01:41:02.970293+0000","last_became_active":"2024-03-30T01:41:02.967586+0000","last_became_peered":"2024-03-30T01:41:02.967586+0000","last_unstale":"2024-03-30T01:41:02.970293+0000","last_undegraded":"2024-03-30T01:41:02.970293+0000","last_fullsized":"2024-03-30T01:41:02.970293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:41:01.760510+0000","last_clean_scrub_stamp":"2024-03-30T01:41:01.760510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:57:29.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.782245+0000","last_change":"2024-03-30T01:40:57.962668+0000","last_active":"2024-03-30T01:41:02.782245+0000","last_peered":"2024-03-30T01:41:02.782245+0000","last_clean":"2024-03-30T01:41:02.782245+0000","last_became_active":"2024-03-30T01:40:57.962255+0000","last_became_peered":"2024-03-30T01:40:57.962255+0000","last_unstale":"2024-03-30T01:41:02.782245+0000","last_undegraded":"2024-03-30T01:41:02.782245+0000","last_fullsized":"2024-03-30T01:41:02.782245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:56.936859+0000","last_clean_scrub_stamp":"2024-03-30T01:40:56.936859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:32:33.482081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.965085+0000","last_change":"2024-03-30T01:40:58.441825+0000","last_active":"2024-03-30T01:41:02.965085+0000","last_peered":"2024-03-30T01:41:02.965085+0000","last_clean":"2024-03-30T01:41:02.965085+0000","last_became_active":"2024-03-30T01:40:58.441575+0000","last_became_peered":"2024-03-30T01:40:58.441575+0000","last_unstale":"2024-03-30T01:41:02.965085+0000","last_undegraded":"2024-03-30T01:41:02.965085+0000","last_fullsized":"2024-03-30T01:41:02.965085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:51.791864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872404+0000","last_change":"2024-03-30T01:40:58.425945+0000","last_active":"2024-03-30T01:41:02.872404+0000","last_peered":"2024-03-30T01:41:02.872404+0000","last_clean":"2024-03-30T01:41:02.872404+0000","last_became_active":"2024-03-30T01:40:58.425607+0000","last_became_peered":"2024-03-30T01:40:58.425607+0000","last_unstale":"2024-03-30T01:41:02.872404+0000","last_undegraded":"2024-03-30T01:41:02.872404+0000","last_fullsized":"2024-03-30T01:41:02.872404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868697+0000","last_change":"2024-03-30T01:40:58.427012+0000","last_active":"2024-03-30T01:41:02.868697+0000","last_peered":"2024-03-30T01:41:02.868697+0000","last_clean":"2024-03-30T01:41:02.868697+0000","last_became_active":"2024-03-30T01:40:58.426750+0000","last_became_peered":"2024-03-30T01:40:58.426750+0000","last_unstale":"2024-03-30T01:41:02.868697+0000","last_undegraded":"2024-03-30T01:41:02.868697+0000","last_fullsized":"2024-03-30T01:41:02.868697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:14:37.276326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868317+0000","last_change":"2024-03-30T01:40:58.426120+0000","last_active":"2024-03-30T01:41:02.868317+0000","last_peered":"2024-03-30T01:41:02.868317+0000","last_clean":"2024-03-30T01:41:02.868317+0000","last_became_active":"2024-03-30T01:40:58.425776+0000","last_became_peered":"2024-03-30T01:40:58.425776+0000","last_unstale":"2024-03-30T01:41:02.868317+0000","last_undegraded":"2024-03-30T01:41:02.868317+0000","last_fullsized":"2024-03-30T01:41:02.868317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:42.611753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869771+0000","last_change":"2024-03-30T01:40:58.436516+0000","last_active":"2024-03-30T01:41:02.869771+0000","last_peered":"2024-03-30T01:41:02.869771+0000","last_clean":"2024-03-30T01:41:02.869771+0000","last_became_active":"2024-03-30T01:40:58.436247+0000","last_became_peered":"2024-03-30T01:40:58.436247+0000","last_unstale":"2024-03-30T01:41:02.869771+0000","last_undegraded":"2024-03-30T01:41:02.869771+0000","last_fullsized":"2024-03-30T01:41:02.869771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868868+0000","last_change":"2024-03-30T01:40:58.415507+0000","last_active":"2024-03-30T01:41:02.868868+0000","last_peered":"2024-03-30T01:41:02.868868+0000","last_clean":"2024-03-30T01:41:02.868868+0000","last_became_active":"2024-03-30T01:40:58.415197+0000","last_became_peered":"2024-03-30T01:40:58.415197+0000","last_unstale":"2024-03-30T01:41:02.868868+0000","last_undegraded":"2024-03-30T01:41:02.868868+0000","last_fullsized":"2024-03-30T01:41:02.868868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868506+0000","last_change":"2024-03-30T01:40:58.425962+0000","last_active":"2024-03-30T01:41:02.868506+0000","last_peered":"2024-03-30T01:41:02.868506+0000","last_clean":"2024-03-30T01:41:02.868506+0000","last_became_active":"2024-03-30T01:40:58.425667+0000","last_became_peered":"2024-03-30T01:40:58.425667+0000","last_unstale":"2024-03-30T01:41:02.868506+0000","last_undegraded":"2024-03-30T01:41:02.868506+0000","last_fullsized":"2024-03-30T01:41:02.868506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:32:43.938562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872585+0000","last_change":"2024-03-30T01:40:58.429802+0000","last_active":"2024-03-30T01:41:02.872585+0000","last_peered":"2024-03-30T01:41:02.872585+0000","last_clean":"2024-03-30T01:41:02.872585+0000","last_became_active":"2024-03-30T01:40:58.429296+0000","last_became_peered":"2024-03-30T01:40:58.429296+0000","last_unstale":"2024-03-30T01:41:02.872585+0000","last_undegraded":"2024-03-30T01:41:02.872585+0000","last_fullsized":"2024-03-30T01:41:02.872585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:33:39.397280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869051+0000","last_change":"2024-03-30T01:40:58.420564+0000","last_active":"2024-03-30T01:41:02.869051+0000","last_peered":"2024-03-30T01:41:02.869051+0000","last_clean":"2024-03-30T01:41:02.869051+0000","last_became_active":"2024-03-30T01:40:58.420286+0000","last_became_peered":"2024-03-30T01:40:58.420286+0000","last_unstale":"2024-03-30T01:41:02.869051+0000","last_undegraded":"2024-03-30T01:41:02.869051+0000","last_fullsized":"2024-03-30T01:41:02.869051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:58:16.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868288+0000","last_change":"2024-03-30T01:40:58.418904+0000","last_active":"2024-03-30T01:41:02.868288+0000","last_peered":"2024-03-30T01:41:02.868288+0000","last_clean":"2024-03-30T01:41:02.868288+0000","last_became_active":"2024-03-30T01:40:58.418351+0000","last_became_peered":"2024-03-30T01:40:58.418351+0000","last_unstale":"2024-03-30T01:41:02.868288+0000","last_undegraded":"2024-03-30T01:41:02.868288+0000","last_fullsized":"2024-03-30T01:41:02.868288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:12.554736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868595+0000","last_change":"2024-03-30T01:40:58.416220+0000","last_active":"2024-03-30T01:41:02.868595+0000","last_peered":"2024-03-30T01:41:02.868595+0000","last_clean":"2024-03-30T01:41:02.868595+0000","last_became_active":"2024-03-30T01:40:58.415937+0000","last_became_peered":"2024-03-30T01:40:58.415937+0000","last_unstale":"2024-03-30T01:41:02.868595+0000","last_undegraded":"2024-03-30T01:41:02.868595+0000","last_fullsized":"2024-03-30T01:41:02.868595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:02:10.867212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.868332+0000","last_change":"2024-03-30T01:40:58.433268+0000","last_active":"2024-03-30T01:41:02.868332+0000","last_peered":"2024-03-30T01:41:02.868332+0000","last_clean":"2024-03-30T01:41:02.868332+0000","last_became_active":"2024-03-30T01:40:58.432963+0000","last_became_peered":"2024-03-30T01:40:58.432963+0000","last_unstale":"2024-03-30T01:41:02.868332+0000","last_undegraded":"2024-03-30T01:41:02.868332+0000","last_fullsized":"2024-03-30T01:41:02.868332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:06:50.182379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.869784+0000","last_change":"2024-03-30T01:40:58.440462+0000","last_active":"2024-03-30T01:41:02.869784+0000","last_peered":"2024-03-30T01:41:02.869784+0000","last_clean":"2024-03-30T01:41:02.869784+0000","last_became_active":"2024-03-30T01:40:58.440144+0000","last_became_peered":"2024-03-30T01:40:58.440144+0000","last_unstale":"2024-03-30T01:41:02.869784+0000","last_undegraded":"2024-03-30T01:41:02.869784+0000","last_fullsized":"2024-03-30T01:41:02.869784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:07.980835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-30T01:41:02.872499+0000","last_change":"2024-03-30T01:40:58.422820+0000","last_active":"2024-03-30T01:41:02.872499+0000","last_peered":"2024-03-30T01:41:02.872499+0000","last_clean":"2024-03-30T01:41:02.872499+0000","last_became_active":"2024-03-30T01:40:58.420834+0000","last_became_peered":"2024-03-30T01:40:58.420834+0000","last_unstale":"2024-03-30T01:41:02.872499+0000","last_undegraded":"2024-03-30T01:41:02.872499+0000","last_fullsized":"2024-03-30T01:41:02.872499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:57.939088+0000","last_clean_scrub_stamp":"2024-03-30T01:40:57.939088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:28.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.858000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.99}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.974}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.504}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":856064,"data_stored":689817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.478999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.7400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.3970000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.5919999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7480000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.8410000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.8970000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:41:25.278 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-30T01:41:25.278 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-30T01:41:25.278 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-30T01:41:25.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T01:41:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.443+0000 7f42a4cc2700 1 Processor -- start 2024-03-30T01:41:25.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.443+0000 7f42a4cc2700 1 -- start start 2024-03-30T01:41:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.444+0000 7f42a4cc2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f42a0155680 0x7f42a00c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.444+0000 7f42a4cc2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a00a6d40 0x7f42a00c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.444+0000 7f42a4cc2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a0155a60 con 0x7f42a00c2d80 2024-03-30T01:41:25.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f42a4cc2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a006c450 con 0x7f42a0155680 2024-03-30T01:41:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f42a4cc2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00c32c0 con 0x7f42a00a6d40 2024-03-30T01:41:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429e59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f42a0155680 0x7f42a00c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a00a6d40 0x7f42a00c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429ed9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f42a00c2d80 0x7f42a006c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:32896/0 (socket says 172.21.15.31:32896) 2024-03-30T01:41:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429e59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f42a0155680 0x7f42a00c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:46920/0 (socket says 172.21.15.31:46920) 2024-03-30T01:41:25.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429ed9d700 1 -- 172.21.15.31:0/2671765470 learned_addr learned my addr 172.21.15.31:0/2671765470 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429e59c700 1 -- 172.21.15.31:0/2671765470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a00a6d40 msgr2=0x7f42a00c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429e59c700 1 --2- 172.21.15.31:0/2671765470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a00a6d40 0x7f42a00c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429e59c700 1 -- 172.21.15.31:0/2671765470 >> v1:172.21.15.59:6789/0 conn(0x7f42a00c2d80 legacy=0x7f42a006c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429d59a700 1 -- 172.21.15.31:0/2671765470 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449579541 0 0) 0x7f42a0155a60 con 0x7f42a00c2d80 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.445+0000 7f429e59c700 1 -- 172.21.15.31:0/2671765470 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00c34d0 con 0x7f42a0155680 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.446+0000 7f429e59c700 1 --2- 172.21.15.31:0/2671765470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42a0155680 0x7f42a00c21b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4290009840 tx=0x7f4290009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f7914560e698d09 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.446+0000 7f429d59a700 1 -- 172.21.15.31:0/2671765470 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290011830 con 0x7f42a0155680 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.446+0000 7f429d59a700 1 -- 172.21.15.31:0/2671765470 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4290011990 con 0x7f42a0155680 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.446+0000 7f429d59a700 1 -- 172.21.15.31:0/2671765470 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290011d20 con 0x7f42a0155680 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.446+0000 7f42a4cc2700 1 -- 172.21.15.31:0/2671765470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42a0155680 msgr2=0x7f42a00c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.446+0000 7f42a4cc2700 1 --2- 172.21.15.31:0/2671765470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42a0155680 0x7f42a00c21b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4290009840 tx=0x7f4290009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.447+0000 7f42a4cc2700 1 -- 172.21.15.31:0/2671765470 shutdown_connections 2024-03-30T01:41:25.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.447+0000 7f42a4cc2700 1 --2- 172.21.15.31:0/2671765470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42a0155680 0x7f42a00c21b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.447+0000 7f42a4cc2700 1 --2- 172.21.15.31:0/2671765470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a00a6d40 0x7f42a00c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.447+0000 7f42a4cc2700 1 -- 172.21.15.31:0/2671765470 >> 172.21.15.31:0/2671765470 conn(0x7f42a00a9bf0 msgr2=0x7f42a00aa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.447+0000 7f42a4cc2700 1 -- 172.21.15.31:0/2671765470 shutdown_connections 2024-03-30T01:41:25.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.447+0000 7f42a4cc2700 1 -- 172.21.15.31:0/2671765470 wait complete. 2024-03-30T01:41:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.448+0000 7f42a4cc2700 1 Processor -- start 2024-03-30T01:41:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.448+0000 7f42a4cc2700 1 -- start start 2024-03-30T01:41:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f42a4cc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42a00a6d40 0x7f42a0155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f42a4cc2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a00c2d80 0x7f42a014d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f42a4cc2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a0155680 0x7f42a014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f42a4cc2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a014de10 con 0x7f42a00c2d80 2024-03-30T01:41:25.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f42a4cc2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0156ff0 con 0x7f42a00a6d40 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f42a4cc2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0151e70 con 0x7f42a0155680 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a00c2d80 0x7f42a014d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a00c2d80 0x7f42a014d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51582/0 (socket says 172.21.15.31:51582) 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429dd9b700 1 -- 172.21.15.31:0/500065534 learned_addr learned my addr 172.21.15.31:0/500065534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429e59c700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42a00a6d40 0x7f42a0155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429ed9d700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a0155680 0x7f42a014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429dd9b700 1 -- 172.21.15.31:0/500065534 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a0155680 msgr2=0x7f42a014d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429dd9b700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a0155680 0x7f42a014d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429dd9b700 1 -- 172.21.15.31:0/500065534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42a00a6d40 msgr2=0x7f42a0155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429dd9b700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42a00a6d40 0x7f42a0155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.449+0000 7f429dd9b700 1 -- 172.21.15.31:0/500065534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900094a0 con 0x7f42a00c2d80 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.450+0000 7f429dd9b700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a00c2d80 0x7f42a014d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f429400e980 tx=0x7f429400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.450+0000 7f428effd700 1 -- 172.21.15.31:0/500065534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294009710 con 0x7f42a00c2d80 2024-03-30T01:41:25.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.450+0000 7f428effd700 1 -- 172.21.15.31:0/500065534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429400c690 con 0x7f42a00c2d80 2024-03-30T01:41:25.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.450+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a0152150 con 0x7f42a00c2d80 2024-03-30T01:41:25.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.450+0000 7f428effd700 1 -- 172.21.15.31:0/500065534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400ca20 con 0x7f42a00c2d80 2024-03-30T01:41:25.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.450+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a0157500 con 0x7f42a00c2d80 2024-03-30T01:41:25.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.451+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a0157720 con 0x7f42a00c2d80 2024-03-30T01:41:25.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.451+0000 7f428effd700 1 -- 172.21.15.31:0/500065534 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f429400cb80 con 0x7f42a00c2d80 2024-03-30T01:41:25.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.452+0000 7f428effd700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f428006ac90 0x7f428006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.452+0000 7f428effd700 1 -- 172.21.15.31:0/500065534 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4294089510 con 0x7f42a00c2d80 2024-03-30T01:41:25.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.452+0000 7f429e59c700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f428006ac90 0x7f428006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.452+0000 7f429e59c700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f428006ac90 0x7f428006d150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42900090b0 tx=0x7f429001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:25.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.456+0000 7f428effd700 1 -- 172.21.15.31:0/500065534 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4294054490 con 0x7f42a00c2d80 2024-03-30T01:41:25.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.717+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f42a004ea80 con 0x7f42a00c2d80 2024-03-30T01:41:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.717+0000 7f428effd700 1 -- 172.21.15.31:0/500065534 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4294057ab0 con 0x7f42a00c2d80 2024-03-30T01:41:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:25.724 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-30T01:41:25.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.719+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f428006ac90 msgr2=0x7f428006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f428006ac90 0x7f428006d150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42900090b0 tx=0x7f429001c040 comp rx=0 tx=0).stop 2024-03-30T01:41:25.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a00c2d80 msgr2=0x7f42a014d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a00c2d80 0x7f42a014d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f429400e980 tx=0x7f429400ec90 comp rx=0 tx=0).stop 2024-03-30T01:41:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 shutdown_connections 2024-03-30T01:41:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f428006ac90 0x7f428006d150 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42a0155680 0x7f42a014d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42a00a6d40 0x7f42a0155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 --2- 172.21.15.31:0/500065534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42a00c2d80 0x7f42a014d390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 >> 172.21.15.31:0/500065534 conn(0x7f42a00a9bf0 msgr2=0x7f42a006c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 shutdown_connections 2024-03-30T01:41:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.720+0000 7f42a4cc2700 1 -- 172.21.15.31:0/500065534 wait complete. 2024-03-30T01:41:25.741 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-30T01:41:25.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:25.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.870+0000 7f00d99ec700 1 Processor -- start 2024-03-30T01:41:25.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.870+0000 7f00d99ec700 1 -- start start 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.871+0000 7f00d99ec700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f00d406b170 0x7f00d4155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.871+0000 7f00d99ec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d4154760 0x7f00d406b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.871+0000 7f00d99ec700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d40c8d10 con 0x7f00d40bd7f0 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.871+0000 7f00d99ec700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d406bef0 con 0x7f00d406b170 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.871+0000 7f00d99ec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40c98b0 con 0x7f00d4154760 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.871+0000 7f00d2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f00d406b170 0x7f00d4155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.871+0000 7f00d27fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f00d40bd7f0 0x7f00d40c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:32906/0 (socket says 172.21.15.31:32906) 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.871+0000 7f00d27fc700 1 -- 172.21.15.31:0/1280790938 learned_addr learned my addr 172.21.15.31:0/1280790938 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.871+0000 7f00d37fe700 1 --2- 172.21.15.31:0/1280790938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d4154760 0x7f00d406b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d1ffb700 1 -- 172.21.15.31:0/1280790938 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458071013 0 0) 0x7f00d40c8d10 con 0x7f00d40bd7f0 2024-03-30T01:41:25.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d1ffb700 1 -- 172.21.15.31:0/1280790938 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc0035b0 con 0x7f00d40bd7f0 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d2ffd700 1 -- 172.21.15.31:0/1280790938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d4154760 msgr2=0x7f00d406b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d2ffd700 1 --2- 172.21.15.31:0/1280790938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d4154760 0x7f00d406b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d2ffd700 1 -- 172.21.15.31:0/1280790938 >> v1:172.21.15.59:6789/0 conn(0x7f00d40bd7f0 legacy=0x7f00d40c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d2ffd700 1 -- 172.21.15.31:0/1280790938 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d40c8100 con 0x7f00d406b170 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d37fe700 1 --2- 172.21.15.31:0/1280790938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d4154760 0x7f00d406b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d2ffd700 1 --2- 172.21.15.31:0/1280790938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d406b170 0x7f00d4155c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009840 tx=0x7f00c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3fa476b78e75cf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d1ffb700 1 -- 172.21.15.31:0/1280790938 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4011830 con 0x7f00d406b170 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d1ffb700 1 -- 172.21.15.31:0/1280790938 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c4011990 con 0x7f00d406b170 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.872+0000 7f00d1ffb700 1 -- 172.21.15.31:0/1280790938 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4011d20 con 0x7f00d406b170 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.873+0000 7f00d99ec700 1 -- 172.21.15.31:0/1280790938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d406b170 msgr2=0x7f00d4155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.873+0000 7f00d99ec700 1 --2- 172.21.15.31:0/1280790938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d406b170 0x7f00d4155c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009840 tx=0x7f00c4009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.873+0000 7f00d99ec700 1 -- 172.21.15.31:0/1280790938 shutdown_connections 2024-03-30T01:41:25.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.873+0000 7f00d99ec700 1 --2- 172.21.15.31:0/1280790938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d406b170 0x7f00d4155c50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.873+0000 7f00d99ec700 1 --2- 172.21.15.31:0/1280790938 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d4154760 0x7f00d406b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.873+0000 7f00d99ec700 1 -- 172.21.15.31:0/1280790938 >> 172.21.15.31:0/1280790938 conn(0x7f00d40ca0d0 msgr2=0x7f00d40ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:25.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.873+0000 7f00d99ec700 1 -- 172.21.15.31:0/1280790938 shutdown_connections 2024-03-30T01:41:25.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.873+0000 7f00d99ec700 1 -- 172.21.15.31:0/1280790938 wait complete. 2024-03-30T01:41:25.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.874+0000 7f00d99ec700 1 Processor -- start 2024-03-30T01:41:25.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.874+0000 7f00d99ec700 1 -- start start 2024-03-30T01:41:25.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d99ec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d406b170 0x7f00d41ea320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d99ec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40bd7f0 0x7f00d40c0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d99ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d4154760 0x7f00d41ed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d99ec700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d41e9f00 con 0x7f00d40bd7f0 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d99ec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d4152940 con 0x7f00d4154760 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d99ec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d41edbf0 con 0x7f00d406b170 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40bd7f0 0x7f00d40c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40bd7f0 0x7f00d40c0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51598/0 (socket says 172.21.15.31:51598) 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d27fc700 1 -- 172.21.15.31:0/3899649076 learned_addr learned my addr 172.21.15.31:0/3899649076 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d2ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d406b170 0x7f00d41ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.875+0000 7f00d37fe700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d4154760 0x7f00d41ed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d27fc700 1 -- 172.21.15.31:0/3899649076 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d406b170 msgr2=0x7f00d41ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d27fc700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d406b170 0x7f00d41ea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d27fc700 1 -- 172.21.15.31:0/3899649076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d4154760 msgr2=0x7f00d41ed6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d27fc700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d4154760 0x7f00d41ed6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d27fc700 1 -- 172.21.15.31:0/3899649076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc009a70 con 0x7f00d40bd7f0 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d37fe700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d4154760 0x7f00d41ed6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d2ffd700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d406b170 0x7f00d41ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d27fc700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40bd7f0 0x7f00d40c0170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f00c800c4f0 tx=0x7f00c800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00bbfff700 1 -- 172.21.15.31:0/3899649076 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8005d70 con 0x7f00d40bd7f0 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c40094a0 con 0x7f00d40bd7f0 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00bbfff700 1 -- 172.21.15.31:0/3899649076 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c800ce90 con 0x7f00d40bd7f0 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00bbfff700 1 -- 172.21.15.31:0/3899649076 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c80072d0 con 0x7f00d40bd7f0 2024-03-30T01:41:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.876+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d41e9840 con 0x7f00d40bd7f0 2024-03-30T01:41:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.877+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d41ea040 con 0x7f00d40bd7f0 2024-03-30T01:41:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.878+0000 7f00bbfff700 1 -- 172.21.15.31:0/3899649076 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f00c8007430 con 0x7f00d40bd7f0 2024-03-30T01:41:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.879+0000 7f00bbfff700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f00c006ac90 0x7f00c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.879+0000 7f00d2ffd700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f00c006ac90 0x7f00c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:25.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.879+0000 7f00d2ffd700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f00c006ac90 0x7f00c006d150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f00c4000c00 tx=0x7f00c4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:25.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.879+0000 7f00bbfff700 1 -- 172.21.15.31:0/3899649076 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f00c8057530 con 0x7f00d40bd7f0 2024-03-30T01:41:25.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:25.884+0000 7f00bbfff700 1 -- 172.21.15.31:0/3899649076 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f00c8053870 con 0x7f00d40bd7f0 2024-03-30T01:41:26.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.102+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00d404ea80 con 0x7f00d40bd7f0 2024-03-30T01:41:26.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.103+0000 7f00bbfff700 1 -- 172.21.15.31:0/3899649076 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f00c8056e90 con 0x7f00d40bd7f0 2024-03-30T01:41:26.110 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:26.110 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f00c006ac90 msgr2=0x7f00c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f00c006ac90 0x7f00c006d150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f00c4000c00 tx=0x7f00c4008030 comp rx=0 tx=0).stop 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40bd7f0 msgr2=0x7f00d40c0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40bd7f0 0x7f00d40c0170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f00c800c4f0 tx=0x7f00c800c800 comp rx=0 tx=0).stop 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 shutdown_connections 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f00c006ac90 0x7f00c006d150 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00d4154760 0x7f00d41ed6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00d406b170 0x7f00d41ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 --2- 172.21.15.31:0/3899649076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00d40bd7f0 0x7f00d40c0170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 >> 172.21.15.31:0/3899649076 conn(0x7f00d40ca0d0 msgr2=0x7f00d4153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:26.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 shutdown_connections 2024-03-30T01:41:26.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.106+0000 7f00d99ec700 1 -- 172.21.15.31:0/3899649076 wait complete. 2024-03-30T01:41:26.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:26.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.259+0000 7faf77a6d700 1 Processor -- start 2024-03-30T01:41:26.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.259+0000 7faf77a6d700 1 -- start start 2024-03-30T01:41:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.260+0000 7faf77a6d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 0x7faf7006a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.260+0000 7faf77a6d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faf7006d340 0x7faf70154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.260+0000 7faf77a6d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf70152070 con 0x7faf700c1250 2024-03-30T01:41:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.260+0000 7faf77a6d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf700acaa0 con 0x7faf7006d340 2024-03-30T01:41:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.260+0000 7faf77a6d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf700c3790 con 0x7faf70151c90 2024-03-30T01:41:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.260+0000 7faf75008700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faf700c1250 0x7faf70154680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35276/0 (socket says 172.21.15.31:35276) 2024-03-30T01:41:26.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.260+0000 7faf75008700 1 -- 172.21.15.31:0/285743314 learned_addr learned my addr 172.21.15.31:0/285743314 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf7600a700 1 --2- 172.21.15.31:0/285743314 >> v2:172.21.15.59:3300/0 conn(0x7faf7006d340 0x7faf70154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf75809700 1 --2- 172.21.15.31:0/285743314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 0x7faf7006a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf74807700 1 -- 172.21.15.31:0/285743314 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877762286 0 0) 0x7faf70152070 con 0x7faf700c1250 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf74807700 1 -- 172.21.15.31:0/285743314 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf5c0035b0 con 0x7faf700c1250 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf7600a700 1 -- 172.21.15.31:0/285743314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 msgr2=0x7faf7006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf7600a700 1 --2- 172.21.15.31:0/285743314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 0x7faf7006a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf7600a700 1 -- 172.21.15.31:0/285743314 >> v1:172.21.15.31:6789/0 conn(0x7faf700c1250 legacy=0x7faf70154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf7600a700 1 -- 172.21.15.31:0/285743314 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf700acef0 con 0x7faf7006d340 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf7600a700 1 --2- 172.21.15.31:0/285743314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf7006d340 0x7faf70154d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faf68009d90 tx=0x7faf6800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c715e50240879547 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf74807700 1 -- 172.21.15.31:0/285743314 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6800e070 con 0x7faf7006d340 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf74807700 1 -- 172.21.15.31:0/285743314 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6800bb20 con 0x7faf7006d340 2024-03-30T01:41:26.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.261+0000 7faf74807700 1 -- 172.21.15.31:0/285743314 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68012430 con 0x7faf7006d340 2024-03-30T01:41:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.262+0000 7faf77a6d700 1 -- 172.21.15.31:0/285743314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf7006d340 msgr2=0x7faf70154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.262+0000 7faf77a6d700 1 --2- 172.21.15.31:0/285743314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf7006d340 0x7faf70154d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faf68009d90 tx=0x7faf6800b330 comp rx=0 tx=0).stop 2024-03-30T01:41:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.262+0000 7faf77a6d700 1 -- 172.21.15.31:0/285743314 shutdown_connections 2024-03-30T01:41:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.262+0000 7faf77a6d700 1 --2- 172.21.15.31:0/285743314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf7006d340 0x7faf70154d90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.262+0000 7faf77a6d700 1 --2- 172.21.15.31:0/285743314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 0x7faf7006a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.262+0000 7faf77a6d700 1 -- 172.21.15.31:0/285743314 >> 172.21.15.31:0/285743314 conn(0x7faf700c3910 msgr2=0x7faf700c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.262+0000 7faf77a6d700 1 -- 172.21.15.31:0/285743314 shutdown_connections 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.262+0000 7faf77a6d700 1 -- 172.21.15.31:0/285743314 wait complete. 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf77a6d700 1 Processor -- start 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf77a6d700 1 -- start start 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf77a6d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf7006d340 0x7faf700c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf77a6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf700c1250 0x7faf7014b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf77a6d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 0x7faf7014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf77a6d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7014c340 con 0x7faf7006d340 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf77a6d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf701e9820 con 0x7faf700c1250 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf77a6d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf701e9b00 con 0x7faf70151c90 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf75809700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf7006d340 0x7faf700c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf75809700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf7006d340 0x7faf700c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51612/0 (socket says 172.21.15.31:51612) 2024-03-30T01:41:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf75809700 1 -- 172.21.15.31:0/133186811 learned_addr learned my addr 172.21.15.31:0/133186811 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.264+0000 7faf75008700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf700c1250 0x7faf7014b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf7600a700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 0x7faf7014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf75809700 1 -- 172.21.15.31:0/133186811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 msgr2=0x7faf7014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf75809700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 0x7faf7014be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf75809700 1 -- 172.21.15.31:0/133186811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf700c1250 msgr2=0x7faf7014b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf75809700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf700c1250 0x7faf7014b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf75809700 1 -- 172.21.15.31:0/133186811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf68009a70 con 0x7faf7006d340 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf75809700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf7006d340 0x7faf700c5ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faf6000e9b0 tx=0x7faf6000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf667fc700 1 -- 172.21.15.31:0/133186811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60009710 con 0x7faf7006d340 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf667fc700 1 -- 172.21.15.31:0/133186811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6000ee50 con 0x7faf7006d340 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf667fc700 1 -- 172.21.15.31:0/133186811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000ca50 con 0x7faf7006d340 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf701e9de0 con 0x7faf7006d340 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.265+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf701ea3a0 con 0x7faf7006d340 2024-03-30T01:41:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.266+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf70066f90 con 0x7faf7006d340 2024-03-30T01:41:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.267+0000 7faf667fc700 1 -- 172.21.15.31:0/133186811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faf6000cbb0 con 0x7faf7006d340 2024-03-30T01:41:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.267+0000 7faf667fc700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf5406ac90 0x7faf5406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.267+0000 7faf667fc700 1 -- 172.21.15.31:0/133186811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faf60089670 con 0x7faf7006d340 2024-03-30T01:41:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.267+0000 7faf75008700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf5406ac90 0x7faf5406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.268+0000 7faf75008700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf5406ac90 0x7faf5406d150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf700bfef0 tx=0x7faf6c007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:26.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.272+0000 7faf667fc700 1 -- 172.21.15.31:0/133186811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7faf60054550 con 0x7faf7006d340 2024-03-30T01:41:26.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.485+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf700747d0 con 0x7faf7006d340 2024-03-30T01:41:26.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.485+0000 7faf667fc700 1 -- 172.21.15.31:0/133186811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf6000ce60 con 0x7faf7006d340 2024-03-30T01:41:26.492 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf5406ac90 msgr2=0x7faf5406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf5406ac90 0x7faf5406d150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf700bfef0 tx=0x7faf6c007dc0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf7006d340 msgr2=0x7faf700c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf7006d340 0x7faf700c5ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faf6000e9b0 tx=0x7faf6000ecc0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 shutdown_connections 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf7006d340 0x7faf700c5ef0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf5406ac90 0x7faf5406d150 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf70151c90 0x7faf7014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 --2- 172.21.15.31:0/133186811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf700c1250 0x7faf7014b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 >> 172.21.15.31:0/133186811 conn(0x7faf700c3910 msgr2=0x7faf70071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 shutdown_connections 2024-03-30T01:41:26.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.488+0000 7faf77a6d700 1 -- 172.21.15.31:0/133186811 wait complete. 2024-03-30T01:41:26.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:26.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.642+0000 7f643a102700 1 Processor -- start 2024-03-30T01:41:26.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.642+0000 7f643a102700 1 -- start start 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f643a102700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64340c46c0 0x7f643406ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f643a102700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6434070610 con 0x7f64340c5d10 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f643a102700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64340708f0 con 0x7f64340c4aa0 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f643a102700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64340c3210 con 0x7f64340c46c0 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f6432ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f64340c5d10 0x7f64340c26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35284/0 (socket says 172.21.15.31:35284) 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f6432ffd700 1 -- 172.21.15.31:0/2582636178 learned_addr learned my addr 172.21.15.31:0/2582636178 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f64337fe700 1 --2- 172.21.15.31:0/2582636178 >> v2:172.21.15.59:3300/0 conn(0x7f64340c46c0 0x7f643406ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f64327fc700 1 -- 172.21.15.31:0/2582636178 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731178027 0 0) 0x7f6434070610 con 0x7f64340c5d10 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f64327fc700 1 -- 172.21.15.31:0/2582636178 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64200035b0 con 0x7f64340c5d10 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f64337fe700 1 -- 172.21.15.31:0/2582636178 >> v1:172.21.15.59:6789/0 conn(0x7f64340c4aa0 legacy=0x7f643415eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f64337fe700 1 -- 172.21.15.31:0/2582636178 >> v1:172.21.15.31:6789/0 conn(0x7f64340c5d10 legacy=0x7f64340c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f64337fe700 1 -- 172.21.15.31:0/2582636178 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340c34f0 con 0x7f64340c46c0 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.643+0000 7f64337fe700 1 --2- 172.21.15.31:0/2582636178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c46c0 0x7f643406ce80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f641c009840 tx=0x7f641c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d0b83c049c203c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f64327fc700 1 -- 172.21.15.31:0/2582636178 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c011830 con 0x7f64340c46c0 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f64327fc700 1 -- 172.21.15.31:0/2582636178 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c011990 con 0x7f64340c46c0 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f64327fc700 1 -- 172.21.15.31:0/2582636178 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c011d20 con 0x7f64340c46c0 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f643a102700 1 -- 172.21.15.31:0/2582636178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c46c0 msgr2=0x7f643406ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f643a102700 1 --2- 172.21.15.31:0/2582636178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c46c0 0x7f643406ce80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f641c009840 tx=0x7f641c009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f643a102700 1 -- 172.21.15.31:0/2582636178 shutdown_connections 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f643a102700 1 --2- 172.21.15.31:0/2582636178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c46c0 0x7f643406ce80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f643a102700 1 -- 172.21.15.31:0/2582636178 >> 172.21.15.31:0/2582636178 conn(0x7f64340c7cf0 msgr2=0x7f64340c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f643a102700 1 -- 172.21.15.31:0/2582636178 shutdown_connections 2024-03-30T01:41:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.644+0000 7f643a102700 1 -- 172.21.15.31:0/2582636178 wait complete. 2024-03-30T01:41:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.646+0000 7f643a102700 1 Processor -- start 2024-03-30T01:41:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.646+0000 7f643a102700 1 -- start start 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.646+0000 7f643a102700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64340c46c0 0x7f64341ea320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.646+0000 7f643a102700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64340c4aa0 0x7f64341d91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.646+0000 7f643a102700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c5d10 0x7f64340a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.646+0000 7f643a102700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64341d0b80 con 0x7f64340c4aa0 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.646+0000 7f643a102700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64341ea860 con 0x7f64340c5d10 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.646+0000 7f643a102700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340a9080 con 0x7f64340c46c0 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6432ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64340c4aa0 0x7f64341d91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6432ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64340c4aa0 0x7f64341d91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51618/0 (socket says 172.21.15.31:51618) 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6432ffd700 1 -- 172.21.15.31:0/2792918525 learned_addr learned my addr 172.21.15.31:0/2792918525 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f64337fe700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64340c46c0 0x7f64341ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6433fff700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c5d10 0x7f64340a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6432ffd700 1 -- 172.21.15.31:0/2792918525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64340c46c0 msgr2=0x7f64341ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6432ffd700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64340c46c0 0x7f64341ea320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6432ffd700 1 -- 172.21.15.31:0/2792918525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c5d10 msgr2=0x7f64340a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6432ffd700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c5d10 0x7f64340a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6432ffd700 1 -- 172.21.15.31:0/2792918525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c0094a0 con 0x7f64340c4aa0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6433fff700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c5d10 0x7f64340a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f6432ffd700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64340c4aa0 0x7f64341d91d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f642400c4d0 tx=0x7f642400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f642bfff700 1 -- 172.21.15.31:0/2792918525 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424005d70 con 0x7f64340c4aa0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f642bfff700 1 -- 172.21.15.31:0/2792918525 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64240070d0 con 0x7f64340c4aa0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64340aa880 con 0x7f64340c4aa0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f642bfff700 1 -- 172.21.15.31:0/2792918525 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424007460 con 0x7f64340c4aa0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.647+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64341dca30 con 0x7f64340c4aa0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.648+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64340747d0 con 0x7f64340c4aa0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.649+0000 7f642bfff700 1 -- 172.21.15.31:0/2792918525 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f64240075c0 con 0x7f64340c4aa0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.649+0000 7f642bfff700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f641406ac90 0x7f641406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.649+0000 7f642bfff700 1 -- 172.21.15.31:0/2792918525 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6424088a10 con 0x7f64340c4aa0 2024-03-30T01:41:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.649+0000 7f64337fe700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f641406ac90 0x7f641406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.650+0000 7f64337fe700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f641406ac90 0x7f641406d150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f641c0090b0 tx=0x7f641c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:26.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.655+0000 7f642bfff700 1 -- 172.21.15.31:0/2792918525 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6424053990 con 0x7f64340c4aa0 2024-03-30T01:41:26.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.870+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64340a91c0 con 0x7f64340c4aa0 2024-03-30T01:41:26.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.871+0000 7f642bfff700 1 -- 172.21.15.31:0/2792918525 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6424017020 con 0x7f64340c4aa0 2024-03-30T01:41:26.877 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:26.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f641406ac90 msgr2=0x7f641406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f641406ac90 0x7f641406d150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f641c0090b0 tx=0x7f641c008030 comp rx=0 tx=0).stop 2024-03-30T01:41:26.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64340c4aa0 msgr2=0x7f64341d91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:26.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64340c4aa0 0x7f64341d91d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f642400c4d0 tx=0x7f642400c890 comp rx=0 tx=0).stop 2024-03-30T01:41:26.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 shutdown_connections 2024-03-30T01:41:26.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f641406ac90 0x7f641406d150 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64340c5d10 0x7f64340a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64340c46c0 0x7f64341ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 --2- 172.21.15.31:0/2792918525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64340c4aa0 0x7f64341d91d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:26.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 >> 172.21.15.31:0/2792918525 conn(0x7f64340c7cf0 msgr2=0x7f64340ac200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:26.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 shutdown_connections 2024-03-30T01:41:26.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:26.873+0000 7f643a102700 1 -- 172.21.15.31:0/2792918525 wait complete. 2024-03-30T01:41:26.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.027+0000 7fa698ae0700 1 Processor -- start 2024-03-30T01:41:27.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.027+0000 7fa698ae0700 1 -- start start 2024-03-30T01:41:27.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.028+0000 7fa698ae0700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa69406dcb0 0x7fa694153040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.028+0000 7fa698ae0700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6940c2700 con 0x7fa6940c14a0 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.028+0000 7fa698ae0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa694152bb0 con 0x7fa6940c0f50 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.028+0000 7fa698ae0700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69406f290 con 0x7fa69406dcb0 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.028+0000 7fa69259c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa69406dcb0 0x7fa694153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.028+0000 7fa691d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa6940c14a0 0x7fa69406a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35286/0 (socket says 172.21.15.31:35286) 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.028+0000 7fa691d9b700 1 -- 172.21.15.31:0/739931822 learned_addr learned my addr 172.21.15.31:0/739931822 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.029+0000 7fa69259c700 1 -- 172.21.15.31:0/739931822 >> v1:172.21.15.59:6789/0 conn(0x7fa6940c0f50 legacy=0x7fa69414f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.029+0000 7fa69259c700 1 -- 172.21.15.31:0/739931822 >> v1:172.21.15.31:6789/0 conn(0x7fa6940c14a0 legacy=0x7fa69406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.029+0000 7fa69259c700 1 -- 172.21.15.31:0/739931822 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69406eff0 con 0x7fa69406dcb0 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.029+0000 7fa69159a700 1 -- 172.21.15.31:0/739931822 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686269644 0 0) 0x7fa6940c2700 con 0x7fa6940c14a0 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.029+0000 7fa69259c700 1 --2- 172.21.15.31:0/739931822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 0x7fa694153040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009840 tx=0x7fa67c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e75a66442295083 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:27.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.029+0000 7fa69159a700 1 -- 172.21.15.31:0/739931822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c011830 con 0x7fa69406dcb0 2024-03-30T01:41:27.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.029+0000 7fa69159a700 1 -- 172.21.15.31:0/739931822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c011990 con 0x7fa69406dcb0 2024-03-30T01:41:27.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.029+0000 7fa69159a700 1 -- 172.21.15.31:0/739931822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c011d20 con 0x7fa69406dcb0 2024-03-30T01:41:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.030+0000 7fa698ae0700 1 -- 172.21.15.31:0/739931822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 msgr2=0x7fa694153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.030+0000 7fa698ae0700 1 --2- 172.21.15.31:0/739931822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 0x7fa694153040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009840 tx=0x7fa67c009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.030+0000 7fa698ae0700 1 -- 172.21.15.31:0/739931822 shutdown_connections 2024-03-30T01:41:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.030+0000 7fa698ae0700 1 --2- 172.21.15.31:0/739931822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 0x7fa694153040 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.030+0000 7fa698ae0700 1 -- 172.21.15.31:0/739931822 >> 172.21.15.31:0/739931822 conn(0x7fa6940c6550 msgr2=0x7fa6940c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.030+0000 7fa698ae0700 1 -- 172.21.15.31:0/739931822 shutdown_connections 2024-03-30T01:41:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.030+0000 7fa698ae0700 1 -- 172.21.15.31:0/739931822 wait complete. 2024-03-30T01:41:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.031+0000 7fa698ae0700 1 Processor -- start 2024-03-30T01:41:27.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.031+0000 7fa698ae0700 1 -- start start 2024-03-30T01:41:27.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa698ae0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 0x7fa69414af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa698ae0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6940c0f50 0x7fa6940c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa698ae0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6940c14a0 0x7fa694145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa698ae0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69414bf80 con 0x7fa69406dcb0 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa698ae0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa694144f20 con 0x7fa6940c14a0 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa698ae0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69414bca0 con 0x7fa6940c0f50 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa69259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 0x7fa69414af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa69259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 0x7fa69414af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51634/0 (socket says 172.21.15.31:51634) 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa69259c700 1 -- 172.21.15.31:0/544295474 learned_addr learned my addr 172.21.15.31:0/544295474 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa691d9b700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6940c0f50 0x7fa6940c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.032+0000 7fa692d9d700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6940c14a0 0x7fa694145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa69259c700 1 -- 172.21.15.31:0/544295474 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6940c0f50 msgr2=0x7fa6940c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa69259c700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6940c0f50 0x7fa6940c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa69259c700 1 -- 172.21.15.31:0/544295474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6940c14a0 msgr2=0x7fa694145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa69259c700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6940c14a0 0x7fa694145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa69259c700 1 -- 172.21.15.31:0/544295474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c0094a0 con 0x7fa69406dcb0 2024-03-30T01:41:27.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa69259c700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 0x7fa69414af50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa67c014040 tx=0x7fa67c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:27.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa68affd700 1 -- 172.21.15.31:0/544295474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c01a900 con 0x7fa69406dcb0 2024-03-30T01:41:27.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa68affd700 1 -- 172.21.15.31:0/544295474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c01aa60 con 0x7fa69406dcb0 2024-03-30T01:41:27.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69414a750 con 0x7fa69406dcb0 2024-03-30T01:41:27.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa68affd700 1 -- 172.21.15.31:0/544295474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c01add0 con 0x7fa69406dcb0 2024-03-30T01:41:27.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.033+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6940c5940 con 0x7fa69406dcb0 2024-03-30T01:41:27.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.034+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69414ac70 con 0x7fa69406dcb0 2024-03-30T01:41:27.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.035+0000 7fa68affd700 1 -- 172.21.15.31:0/544295474 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa67c022070 con 0x7fa69406dcb0 2024-03-30T01:41:27.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.035+0000 7fa68affd700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa67406ac90 0x7fa67406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.035+0000 7fa68affd700 1 -- 172.21.15.31:0/544295474 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa67c08a720 con 0x7fa69406dcb0 2024-03-30T01:41:27.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.035+0000 7fa691d9b700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa67406ac90 0x7fa67406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.036+0000 7fa691d9b700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa67406ac90 0x7fa67406d150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa684007fd0 tx=0x7fa684007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:27.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.040+0000 7fa68affd700 1 -- 172.21.15.31:0/544295474 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa69414ac70 con 0x7fa69406dcb0 2024-03-30T01:41:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.253+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa69414ac70 con 0x7fa69406dcb0 2024-03-30T01:41:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.254+0000 7fa68affd700 1 -- 172.21.15.31:0/544295474 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa67c015020 con 0x7fa69406dcb0 2024-03-30T01:41:27.260 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa67406ac90 msgr2=0x7fa67406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa67406ac90 0x7fa67406d150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa684007fd0 tx=0x7fa684007c20 comp rx=0 tx=0).stop 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 msgr2=0x7fa69414af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 0x7fa69414af50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa67c014040 tx=0x7fa67c004750 comp rx=0 tx=0).stop 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 shutdown_connections 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69406dcb0 0x7fa69414af50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa67406ac90 0x7fa67406d150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6940c14a0 0x7fa694145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 --2- 172.21.15.31:0/544295474 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6940c0f50 0x7fa6940c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 >> 172.21.15.31:0/544295474 conn(0x7fa6940c6550 msgr2=0x7fa6940c2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 shutdown_connections 2024-03-30T01:41:27.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.256+0000 7fa698ae0700 1 -- 172.21.15.31:0/544295474 wait complete. 2024-03-30T01:41:27.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.408+0000 7f6534605700 1 Processor -- start 2024-03-30T01:41:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.408+0000 7f6534605700 1 -- start start 2024-03-30T01:41:27.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.409+0000 7f6534605700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f652c0721f0 0x7f652c06a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.409+0000 7f6534605700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652c155000 con 0x7f652c151ea0 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.409+0000 7f6534605700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652c06e650 con 0x7f652c155490 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.409+0000 7f6534605700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652c06e930 con 0x7f652c0721f0 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.409+0000 7f65323a1700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f652c151ea0 0x7f652c0c02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35298/0 (socket says 172.21.15.31:35298) 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.409+0000 7f65323a1700 1 -- 172.21.15.31:0/3848841333 learned_addr learned my addr 172.21.15.31:0/3848841333 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f6531ba0700 1 --2- 172.21.15.31:0/3848841333 >> v2:172.21.15.59:3300/0 conn(0x7f652c0721f0 0x7f652c06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101262031 0 0) 0x7f652c155000 con 0x7f652c151ea0 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65180035b0 con 0x7f652c151ea0 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985328667 0 0) 0x7f652c06e650 con 0x7f652c155490 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f652c155000 con 0x7f652c155490 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2549378667 0 0) 0x7f65180035b0 con 0x7f652c151ea0 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 >> v2:172.21.15.59:3300/0 conn(0x7f652c0721f0 msgr2=0x7f652c06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 --2- 172.21.15.31:0/3848841333 >> v2:172.21.15.59:3300/0 conn(0x7f652c0721f0 0x7f652c06a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 >> v1:172.21.15.59:6789/0 conn(0x7f652c155490 legacy=0x7f652c06ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c1549b0 con 0x7f652c151ea0 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f6520001670 con 0x7f652c151ea0 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 >> v1:172.21.15.31:6789/0 conn(0x7f652c151ea0 legacy=0x7f652c0c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6518003cc0 0x7f65180040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6518004850 0x7f6518004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518005320 0x7f651800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c06e930 con 0x7f6518005320 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651800c280 con 0x7f6518003cc0 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651800c440 con 0x7f6518004850 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f65323a1700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6518003cc0 0x7f65180040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f6532ba2700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518005320 0x7f651800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.410+0000 7f6531ba0700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6518004850 0x7f6518004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f65323a1700 1 -- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6518004850 msgr2=0x7f6518004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f65323a1700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6518004850 0x7f6518004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f65323a1700 1 -- 172.21.15.31:0/3848841333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518005320 msgr2=0x7f651800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f65323a1700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518005320 0x7f651800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f65323a1700 1 -- 172.21.15.31:0/3848841333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65180035b0 con 0x7f6518003cc0 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f6532ba2700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518005320 0x7f651800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f6532ba2700 1 -- 172.21.15.31:0/3848841333 reap_dead start 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f65323a1700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6518003cc0 0x7f65180040c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f652000c110 tx=0x7f6518005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed1fe63ade0c836c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000ccb0 con 0x7f6518003cc0 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652000ce10 con 0x7f6518003cc0 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.411+0000 7f653139f700 1 -- 172.21.15.31:0/3848841333 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520005ba0 con 0x7f6518003cc0 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.412+0000 7f6534605700 1 -- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6518003cc0 msgr2=0x7f65180040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.412+0000 7f6534605700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6518003cc0 0x7f65180040c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f652000c110 tx=0x7f6518005880 comp rx=0 tx=0).stop 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.412+0000 7f6534605700 1 -- 172.21.15.31:0/3848841333 shutdown_connections 2024-03-30T01:41:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.412+0000 7f6534605700 1 --2- 172.21.15.31:0/3848841333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6518003cc0 0x7f65180040c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.412+0000 7f6534605700 1 -- 172.21.15.31:0/3848841333 >> 172.21.15.31:0/3848841333 conn(0x7f652c0c3980 msgr2=0x7f652c0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.412+0000 7f6534605700 1 -- 172.21.15.31:0/3848841333 shutdown_connections 2024-03-30T01:41:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.412+0000 7f6534605700 1 -- 172.21.15.31:0/3848841333 wait complete. 2024-03-30T01:41:27.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.413+0000 7f6534605700 1 Processor -- start 2024-03-30T01:41:27.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.413+0000 7f6534605700 1 -- start start 2024-03-30T01:41:27.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6534605700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518003cc0 0x7f652c14f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6534605700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f652c158a50 0x7f652c148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6534605700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c1487c0 0x7f652c1e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6534605700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c14bac0 con 0x7f6518003cc0 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6534605700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1edf50 con 0x7f652c1487c0 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6534605700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652c1ee3a0 con 0x7f652c158a50 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f65323a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518003cc0 0x7f652c14f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f65323a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518003cc0 0x7f652c14f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51656/0 (socket says 172.21.15.31:51656) 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f65323a1700 1 -- 172.21.15.31:0/2338850013 learned_addr learned my addr 172.21.15.31:0/2338850013 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6531ba0700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f652c158a50 0x7f652c148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6532ba2700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c1487c0 0x7f652c1e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f65323a1700 1 -- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f652c158a50 msgr2=0x7f652c148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f65323a1700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f652c158a50 0x7f652c148280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f65323a1700 1 -- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c1487c0 msgr2=0x7f652c1e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f65323a1700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c1487c0 0x7f652c1e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f65323a1700 1 -- 172.21.15.31:0/2338850013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c158130 con 0x7f6518003cc0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6531ba0700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f652c158a50 0x7f652c148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.414+0000 7f6532ba2700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c1487c0 0x7f652c1e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.415+0000 7f65323a1700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518003cc0 0x7f652c14f980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6520001eb0 tx=0x7f6520001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.415+0000 7f651effd700 1 -- 172.21.15.31:0/2338850013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520007af0 con 0x7f6518003cc0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.415+0000 7f651effd700 1 -- 172.21.15.31:0/2338850013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6520004c80 con 0x7f6518003cc0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.415+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652c14d020 con 0x7f6518003cc0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.415+0000 7f651effd700 1 -- 172.21.15.31:0/2338850013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200070d0 con 0x7f6518003cc0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.415+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c157e40 con 0x7f6518003cc0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.416+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652c1ee1f0 con 0x7f6518003cc0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.416+0000 7f651effd700 1 -- 172.21.15.31:0/2338850013 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6520007230 con 0x7f6518003cc0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.417+0000 7f651effd700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651006ac90 0x7f651006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.417+0000 7f651effd700 1 -- 172.21.15.31:0/2338850013 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f652008a690 con 0x7f6518003cc0 2024-03-30T01:41:27.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.417+0000 7f6531ba0700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651006ac90 0x7f651006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.417+0000 7f6531ba0700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651006ac90 0x7f651006d150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f652c150b20 tx=0x7f6528008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:27.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.422+0000 7f651effd700 1 -- 172.21.15.31:0/2338850013 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6520055590 con 0x7f6518003cc0 2024-03-30T01:41:27.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.634+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f652c0747d0 con 0x7f6518003cc0 2024-03-30T01:41:27.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.634+0000 7f651effd700 1 -- 172.21.15.31:0/2338850013 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6520058bb0 con 0x7f6518003cc0 2024-03-30T01:41:27.642 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:27.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651006ac90 msgr2=0x7f651006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651006ac90 0x7f651006d150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f652c150b20 tx=0x7f6528008040 comp rx=0 tx=0).stop 2024-03-30T01:41:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518003cc0 msgr2=0x7f652c14f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518003cc0 0x7f652c14f980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6520001eb0 tx=0x7f6520001930 comp rx=0 tx=0).stop 2024-03-30T01:41:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 shutdown_connections 2024-03-30T01:41:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6518003cc0 0x7f652c14f980 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651006ac90 0x7f651006d150 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f652c1487c0 0x7f652c1e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 --2- 172.21.15.31:0/2338850013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f652c158a50 0x7f652c148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 >> 172.21.15.31:0/2338850013 conn(0x7f652c0c3980 msgr2=0x7f652c150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 shutdown_connections 2024-03-30T01:41:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.637+0000 7f6534605700 1 -- 172.21.15.31:0/2338850013 wait complete. 2024-03-30T01:41:27.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:27.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c06b63700 1 Processor -- start 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c06b63700 1 -- start start 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c06b63700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3c000ac520 0x7f3c000ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c06b63700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000a74e0 0x7f3c000ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c06b63700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c00153590 con 0x7f3c000c6000 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c06b63700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c000ab680 con 0x7f3c000ac520 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c06b63700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c001500c0 con 0x7f3c000a74e0 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c05100700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3c000c6000 0x7f3c000c6490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35312/0 (socket says 172.21.15.31:35312) 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c05100700 1 -- 172.21.15.31:0/1007927549 learned_addr learned my addr 172.21.15.31:0/1007927549 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3bfffff700 1 --2- 172.21.15.31:0/1007927549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000a74e0 0x7f3c000ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.788+0000 7f3c048ff700 1 --2- 172.21.15.31:0/1007927549 >> v2:172.21.15.59:3300/0 conn(0x7f3c000ac520 0x7f3c000ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3bff7fe700 1 -- 172.21.15.31:0/1007927549 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128006191 0 0) 0x7f3c00153590 con 0x7f3c000c6000 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3bff7fe700 1 -- 172.21.15.31:0/1007927549 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bec0035b0 con 0x7f3c000c6000 2024-03-30T01:41:27.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3c048ff700 1 -- 172.21.15.31:0/1007927549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000a74e0 msgr2=0x7f3c000ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3c048ff700 1 --2- 172.21.15.31:0/1007927549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000a74e0 0x7f3c000ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3c048ff700 1 -- 172.21.15.31:0/1007927549 >> v1:172.21.15.31:6789/0 conn(0x7f3c000c6000 legacy=0x7f3c000c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3c048ff700 1 -- 172.21.15.31:0/1007927549 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c000c7990 con 0x7f3c000ac520 2024-03-30T01:41:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3c048ff700 1 --2- 172.21.15.31:0/1007927549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c000ac520 0x7f3c000ac900 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009840 tx=0x7f3bf0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b92eeca28ab03f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3bff7fe700 1 -- 172.21.15.31:0/1007927549 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0011830 con 0x7f3c000ac520 2024-03-30T01:41:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3bff7fe700 1 -- 172.21.15.31:0/1007927549 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf0011990 con 0x7f3c000ac520 2024-03-30T01:41:27.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.789+0000 7f3bff7fe700 1 -- 172.21.15.31:0/1007927549 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0011d20 con 0x7f3c000ac520 2024-03-30T01:41:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.790+0000 7f3c06b63700 1 -- 172.21.15.31:0/1007927549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c000ac520 msgr2=0x7f3c000ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.790+0000 7f3c06b63700 1 --2- 172.21.15.31:0/1007927549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c000ac520 0x7f3c000ac900 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009840 tx=0x7f3bf0009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.790+0000 7f3c06b63700 1 -- 172.21.15.31:0/1007927549 shutdown_connections 2024-03-30T01:41:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.790+0000 7f3c06b63700 1 --2- 172.21.15.31:0/1007927549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c000ac520 0x7f3c000ac900 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.790+0000 7f3c06b63700 1 --2- 172.21.15.31:0/1007927549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000a74e0 0x7f3c000ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.790+0000 7f3c06b63700 1 -- 172.21.15.31:0/1007927549 >> 172.21.15.31:0/1007927549 conn(0x7f3c0006ad30 msgr2=0x7f3c000a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.790+0000 7f3c06b63700 1 -- 172.21.15.31:0/1007927549 shutdown_connections 2024-03-30T01:41:27.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.790+0000 7f3c06b63700 1 -- 172.21.15.31:0/1007927549 wait complete. 2024-03-30T01:41:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.791+0000 7f3c06b63700 1 Processor -- start 2024-03-30T01:41:27.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.791+0000 7f3c06b63700 1 -- start start 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c06b63700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c000a74e0 0x7f3c001e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c06b63700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000ac520 0x7f3c0014bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c06b63700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c000c6000 0x7f3c00153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c06b63700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c00070010 con 0x7f3c000a74e0 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c06b63700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c00152930 con 0x7f3c000c6000 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c06b63700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0014c760 con 0x7f3c000ac520 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c048ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c000a74e0 0x7f3c001e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c048ff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c000a74e0 0x7f3c001e80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51670/0 (socket says 172.21.15.31:51670) 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c048ff700 1 -- 172.21.15.31:0/1263849872 learned_addr learned my addr 172.21.15.31:0/1263849872 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c05100700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c000c6000 0x7f3c00153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3bfffff700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000ac520 0x7f3c0014bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c048ff700 1 -- 172.21.15.31:0/1263849872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000ac520 msgr2=0x7f3c0014bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c048ff700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000ac520 0x7f3c0014bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c048ff700 1 -- 172.21.15.31:0/1263849872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c000c6000 msgr2=0x7f3c00153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c048ff700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c000c6000 0x7f3c00153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c048ff700 1 -- 172.21.15.31:0/1263849872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00094a0 con 0x7f3c000a74e0 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3c048ff700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c000a74e0 0x7f3c001e80a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00090b0 tx=0x7f3bf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3bfd7fa700 1 -- 172.21.15.31:0/1263849872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00036a0 con 0x7f3c000a74e0 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.793+0000 7f3bfd7fa700 1 -- 172.21.15.31:0/1263849872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf001f070 con 0x7f3c000a74e0 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.794+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0014ca40 con 0x7f3c000a74e0 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.794+0000 7f3bfd7fa700 1 -- 172.21.15.31:0/1263849872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0003a40 con 0x7f3c000a74e0 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.794+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c001514a0 con 0x7f3c000a74e0 2024-03-30T01:41:27.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.794+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0004ea80 con 0x7f3c000a74e0 2024-03-30T01:41:27.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.795+0000 7f3bfd7fa700 1 -- 172.21.15.31:0/1263849872 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3bf0003ba0 con 0x7f3c000a74e0 2024-03-30T01:41:27.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.795+0000 7f3bfd7fa700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3be006ac90 0x7f3be006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:27.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.796+0000 7f3bfd7fa700 1 -- 172.21.15.31:0/1263849872 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3bf0021070 con 0x7f3c000a74e0 2024-03-30T01:41:27.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.796+0000 7f3bfffff700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3be006ac90 0x7f3be006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:27.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.796+0000 7f3bfffff700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3be006ac90 0x7f3be006d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c0006b720 tx=0x7f3be8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:27.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:27.800+0000 7f3bfd7fa700 1 -- 172.21.15.31:0/1263849872 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3bf0055da0 con 0x7f3c000a74e0 2024-03-30T01:41:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.068+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c000c29f0 con 0x7f3c000a74e0 2024-03-30T01:41:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.068+0000 7f3bfd7fa700 1 -- 172.21.15.31:0/1263849872 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f3bf002b090 con 0x7f3c000a74e0 2024-03-30T01:41:28.074 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:28.074 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:16.755934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T01:41:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.070+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3be006ac90 msgr2=0x7f3be006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.070+0000 7f3c06b63700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3be006ac90 0x7f3be006d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c0006b720 tx=0x7f3be8008040 comp rx=0 tx=0).stop 2024-03-30T01:41:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.070+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c000a74e0 msgr2=0x7f3c001e80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.070+0000 7f3c06b63700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c000a74e0 0x7f3c001e80a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00090b0 tx=0x7f3bf0004750 comp rx=0 tx=0).stop 2024-03-30T01:41:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.071+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 shutdown_connections 2024-03-30T01:41:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.071+0000 7f3c06b63700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c000a74e0 0x7f3c001e80a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.071+0000 7f3c06b63700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3be006ac90 0x7f3be006d150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.071+0000 7f3c06b63700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c000c6000 0x7f3c00153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.071+0000 7f3c06b63700 1 --2- 172.21.15.31:0/1263849872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3c000ac520 0x7f3c0014bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.071+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 >> 172.21.15.31:0/1263849872 conn(0x7f3c0006ad30 msgr2=0x7f3c000c35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.071+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 shutdown_connections 2024-03-30T01:41:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.071+0000 7f3c06b63700 1 -- 172.21.15.31:0/1263849872 wait complete. 2024-03-30T01:41:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-03-30T01:41:28.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:28.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:41:28.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:41:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.222+0000 7fcde6637700 1 Processor -- start 2024-03-30T01:41:28.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.222+0000 7fcde6637700 1 -- start start 2024-03-30T01:41:28.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.223+0000 7fcde6637700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcde014fbf0 0x7fcde006a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.223+0000 7fcde6637700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcde006d9e0 0x7fcde01563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.223+0000 7fcde6637700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde006e4a0 con 0x7fcde00a9560 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.223+0000 7fcde6637700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde0155bf0 con 0x7fcde014fbf0 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.223+0000 7fcde6637700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde006dee0 con 0x7fcde006d9e0 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.223+0000 7fcddffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcde014fbf0 0x7fcde006a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.223+0000 7fcde4bd4700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcde00a9560 0x7fcde00ab0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35328/0 (socket says 172.21.15.31:35328) 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.223+0000 7fcddffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcde014fbf0 0x7fcde006a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51684/0 (socket says 172.21.15.31:51684) 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.223+0000 7fcde4bd4700 1 -- 172.21.15.31:0/2914516591 learned_addr learned my addr 172.21.15.31:0/2914516591 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.224+0000 7fcddffff700 1 -- 172.21.15.31:0/2914516591 >> v2:172.21.15.59:3300/0 conn(0x7fcde006d9e0 msgr2=0x7fcde01563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.224+0000 7fcddffff700 1 --2- 172.21.15.31:0/2914516591 >> v2:172.21.15.59:3300/0 conn(0x7fcde006d9e0 0x7fcde01563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.224+0000 7fcddffff700 1 -- 172.21.15.31:0/2914516591 >> v1:172.21.15.31:6789/0 conn(0x7fcde00a9560 legacy=0x7fcde00ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.224+0000 7fcddffff700 1 -- 172.21.15.31:0/2914516591 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde006e1c0 con 0x7fcde014fbf0 2024-03-30T01:41:28.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.224+0000 7fcddeffd700 1 -- 172.21.15.31:0/2914516591 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449673267 0 0) 0x7fcde006e4a0 con 0x7fcde00a9560 2024-03-30T01:41:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.224+0000 7fcddffff700 1 --2- 172.21.15.31:0/2914516591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde014fbf0 0x7fcde006a700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcde006eac0 tx=0x7fcdd0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9636690543a1090 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.224+0000 7fcddeffd700 1 -- 172.21.15.31:0/2914516591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0011830 con 0x7fcde014fbf0 2024-03-30T01:41:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.224+0000 7fcddeffd700 1 -- 172.21.15.31:0/2914516591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd0011990 con 0x7fcde014fbf0 2024-03-30T01:41:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.224+0000 7fcddeffd700 1 -- 172.21.15.31:0/2914516591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0011d20 con 0x7fcde014fbf0 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.225+0000 7fcde6637700 1 -- 172.21.15.31:0/2914516591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde014fbf0 msgr2=0x7fcde006a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.225+0000 7fcde6637700 1 --2- 172.21.15.31:0/2914516591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde014fbf0 0x7fcde006a700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcde006eac0 tx=0x7fcdd0009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.225+0000 7fcde6637700 1 -- 172.21.15.31:0/2914516591 shutdown_connections 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.225+0000 7fcde6637700 1 --2- 172.21.15.31:0/2914516591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde014fbf0 0x7fcde006a700 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.225+0000 7fcde6637700 1 --2- 172.21.15.31:0/2914516591 >> v2:172.21.15.59:3300/0 conn(0x7fcde006d9e0 0x7fcde01563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.225+0000 7fcde6637700 1 -- 172.21.15.31:0/2914516591 >> 172.21.15.31:0/2914516591 conn(0x7fcde00c3190 msgr2=0x7fcde00c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.225+0000 7fcde6637700 1 -- 172.21.15.31:0/2914516591 shutdown_connections 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.225+0000 7fcde6637700 1 -- 172.21.15.31:0/2914516591 wait complete. 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.226+0000 7fcde6637700 1 Processor -- start 2024-03-30T01:41:28.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.226+0000 7fcde6637700 1 -- start start 2024-03-30T01:41:28.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcde6637700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde006d9e0 0x7fcde014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcde6637700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcde00a9560 0x7fcde014b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcde6637700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde014fbf0 0x7fcde01eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcde6637700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde00bb4d0 con 0x7fcde006d9e0 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcde6637700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde014bd40 con 0x7fcde014fbf0 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcde6637700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde00bb720 con 0x7fcde00a9560 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde006d9e0 0x7fcde014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde006d9e0 0x7fcde014b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51694/0 (socket says 172.21.15.31:51694) 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcddffff700 1 -- 172.21.15.31:0/3260973059 learned_addr learned my addr 172.21.15.31:0/3260973059 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcde4bd4700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde014fbf0 0x7fcde01eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcde00a9560 0x7fcde014b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcddffff700 1 -- 172.21.15.31:0/3260973059 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcde00a9560 msgr2=0x7fcde014b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcddffff700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcde00a9560 0x7fcde014b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcddffff700 1 -- 172.21.15.31:0/3260973059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde014fbf0 msgr2=0x7fcde01eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcddffff700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde014fbf0 0x7fcde01eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcddffff700 1 -- 172.21.15.31:0/3260973059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd00094a0 con 0x7fcde006d9e0 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.227+0000 7fcde4bd4700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde014fbf0 0x7fcde01eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.228+0000 7fcddffff700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde006d9e0 0x7fcde014b2c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0014040 tx=0x7fcdd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.228+0000 7fcddcff9700 1 -- 172.21.15.31:0/3260973059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00038a0 con 0x7fcde006d9e0 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.228+0000 7fcddcff9700 1 -- 172.21.15.31:0/3260973059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd001e070 con 0x7fcde006d9e0 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.228+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde00bba00 con 0x7fcde006d9e0 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.228+0000 7fcddcff9700 1 -- 172.21.15.31:0/3260973059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0003c60 con 0x7fcde006d9e0 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.228+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde0150ee0 con 0x7fcde006d9e0 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.229+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde004ea80 con 0x7fcde006d9e0 2024-03-30T01:41:28.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.229+0000 7fcddcff9700 1 -- 172.21.15.31:0/3260973059 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcdd002d450 con 0x7fcde006d9e0 2024-03-30T01:41:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.230+0000 7fcddcff9700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcdc006ac90 0x7fcdc006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.230+0000 7fcddcff9700 1 -- 172.21.15.31:0/3260973059 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcdd008a780 con 0x7fcde006d9e0 2024-03-30T01:41:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.230+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcdc006ac90 0x7fcdc006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.230+0000 7fcddf7fe700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcdc006ac90 0x7fcdc006d150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcde00a9e90 tx=0x7fcdc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:28.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.235+0000 7fcddcff9700 1 -- 172.21.15.31:0/3260973059 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcdd0055700 con 0x7fcde006d9e0 2024-03-30T01:41:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.449+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcde00c08a0 con 0x7fcde006d9e0 2024-03-30T01:41:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.450+0000 7fcddcff9700 1 -- 172.21.15.31:0/3260973059 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7fcdd0058d20 con 0x7fcde006d9e0 2024-03-30T01:41:28.456 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:28.456 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcdc006ac90 msgr2=0x7fcdc006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcdc006ac90 0x7fcdc006d150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcde00a9e90 tx=0x7fcdc8008040 comp rx=0 tx=0).stop 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde006d9e0 msgr2=0x7fcde014b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde006d9e0 0x7fcde014b2c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0014040 tx=0x7fcdd0004750 comp rx=0 tx=0).stop 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 shutdown_connections 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde006d9e0 0x7fcde014b2c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcdc006ac90 0x7fcdc006d150 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcde014fbf0 0x7fcde01eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 --2- 172.21.15.31:0/3260973059 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcde00a9560 0x7fcde014b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 >> 172.21.15.31:0/3260973059 conn(0x7fcde00c3190 msgr2=0x7fcde00c24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 shutdown_connections 2024-03-30T01:41:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.453+0000 7fcde6637700 1 -- 172.21.15.31:0/3260973059 wait complete. 2024-03-30T01:41:28.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:28.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.607+0000 7fcb82e6c700 1 Processor -- start 2024-03-30T01:41:28.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.607+0000 7fcb82e6c700 1 -- start start 2024-03-30T01:41:28.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb82e6c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb7c06d040 0x7fcb7c072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb82e6c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c06d900 0x7fcb7c06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb82e6c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7c14ee60 con 0x7fcb7c1549a0 2024-03-30T01:41:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb82e6c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7c0c2290 con 0x7fcb7c06d040 2024-03-30T01:41:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb82e6c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c06e2b0 con 0x7fcb7c06d900 2024-03-30T01:41:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb7bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb7c06d040 0x7fcb7c072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb7bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb7c06d040 0x7fcb7c072410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51706/0 (socket says 172.21.15.31:51706) 2024-03-30T01:41:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb7bfff700 1 -- 172.21.15.31:0/4290467795 learned_addr learned my addr 172.21.15.31:0/4290467795 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb81409700 1 --2- 172.21.15.31:0/4290467795 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c06d900 0x7fcb7c06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb7bfff700 1 -- 172.21.15.31:0/4290467795 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c06d900 msgr2=0x7fcb7c06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb7bfff700 1 --2- 172.21.15.31:0/4290467795 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c06d900 0x7fcb7c06dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb7bfff700 1 -- 172.21.15.31:0/4290467795 >> v1:172.21.15.59:6789/0 conn(0x7fcb7c1549a0 legacy=0x7fcb7c154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.608+0000 7fcb7bfff700 1 -- 172.21.15.31:0/4290467795 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c155490 con 0x7fcb7c06d040 2024-03-30T01:41:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb7bfff700 1 --2- 172.21.15.31:0/4290467795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 0x7fcb7c072410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800bb90 tx=0x7fcb6800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc2162dc17776a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb7b7fe700 1 -- 172.21.15.31:0/4290467795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800dd50 con 0x7fcb7c06d040 2024-03-30T01:41:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb7b7fe700 1 -- 172.21.15.31:0/4290467795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb68010470 con 0x7fcb7c06d040 2024-03-30T01:41:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb7b7fe700 1 -- 172.21.15.31:0/4290467795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68010800 con 0x7fcb7c06d040 2024-03-30T01:41:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb82e6c700 1 -- 172.21.15.31:0/4290467795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 msgr2=0x7fcb7c072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb82e6c700 1 --2- 172.21.15.31:0/4290467795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 0x7fcb7c072410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800bb90 tx=0x7fcb6800bea0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb82e6c700 1 -- 172.21.15.31:0/4290467795 shutdown_connections 2024-03-30T01:41:28.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb82e6c700 1 --2- 172.21.15.31:0/4290467795 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c06d900 0x7fcb7c06dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb82e6c700 1 --2- 172.21.15.31:0/4290467795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 0x7fcb7c072410 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb82e6c700 1 -- 172.21.15.31:0/4290467795 >> 172.21.15.31:0/4290467795 conn(0x7fcb7c0c3980 msgr2=0x7fcb7c0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:28.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb82e6c700 1 -- 172.21.15.31:0/4290467795 shutdown_connections 2024-03-30T01:41:28.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.609+0000 7fcb82e6c700 1 -- 172.21.15.31:0/4290467795 wait complete. 2024-03-30T01:41:28.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.611+0000 7fcb82e6c700 1 Processor -- start 2024-03-30T01:41:28.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.611+0000 7fcb82e6c700 1 -- start start 2024-03-30T01:41:28.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb82e6c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 0x7fcb7c149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb82e6c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb7c06d900 0x7fcb7c1e8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb82e6c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c1549a0 0x7fcb7c1e92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb82e6c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c14a190 con 0x7fcb7c06d040 2024-03-30T01:41:28.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb82e6c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1e9df0 con 0x7fcb7c06d900 2024-03-30T01:41:28.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb82e6c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1dbbe0 con 0x7fcb7c1549a0 2024-03-30T01:41:28.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb80c08700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 0x7fcb7c149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb80c08700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 0x7fcb7c149c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51716/0 (socket says 172.21.15.31:51716) 2024-03-30T01:41:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb80c08700 1 -- 172.21.15.31:0/1210206236 learned_addr learned my addr 172.21.15.31:0/1210206236 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb81409700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c1549a0 0x7fcb7c1e92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb7bfff700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb7c06d900 0x7fcb7c1e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb80c08700 1 -- 172.21.15.31:0/1210206236 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c1549a0 msgr2=0x7fcb7c1e92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb80c08700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c1549a0 0x7fcb7c1e92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb80c08700 1 -- 172.21.15.31:0/1210206236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb7c06d900 msgr2=0x7fcb7c1e8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb80c08700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb7c06d900 0x7fcb7c1e8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb80c08700 1 -- 172.21.15.31:0/1210206236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6800b7f0 con 0x7fcb7c06d040 2024-03-30T01:41:28.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb80c08700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 0x7fcb7c149c50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000c450 tx=0x7fcb7000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:28.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.612+0000 7fcb797fa700 1 -- 172.21.15.31:0/1210206236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb700052d0 con 0x7fcb7c06d040 2024-03-30T01:41:28.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.613+0000 7fcb797fa700 1 -- 172.21.15.31:0/1210206236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb70005430 con 0x7fcb7c06d040 2024-03-30T01:41:28.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.613+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c1dbec0 con 0x7fcb7c06d040 2024-03-30T01:41:28.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.613+0000 7fcb797fa700 1 -- 172.21.15.31:0/1210206236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb700038c0 con 0x7fcb7c06d040 2024-03-30T01:41:28.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.613+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c149100 con 0x7fcb7c06d040 2024-03-30T01:41:28.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.614+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c0c4870 con 0x7fcb7c06d040 2024-03-30T01:41:28.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.614+0000 7fcb797fa700 1 -- 172.21.15.31:0/1210206236 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcb70003a20 con 0x7fcb7c06d040 2024-03-30T01:41:28.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.614+0000 7fcb797fa700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb6006ac90 0x7fcb6006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:28.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.615+0000 7fcb797fa700 1 -- 172.21.15.31:0/1210206236 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcb700889c0 con 0x7fcb7c06d040 2024-03-30T01:41:28.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.615+0000 7fcb7bfff700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb6006ac90 0x7fcb6006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:28.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.615+0000 7fcb7bfff700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb6006ac90 0x7fcb6006d150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c06d460 tx=0x7fcb68013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:28.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.619+0000 7fcb797fa700 1 -- 172.21.15.31:0/1210206236 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcb700538b0 con 0x7fcb7c06d040 2024-03-30T01:41:28.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.835+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb7c1dc3e0 con 0x7fcb7c06d040 2024-03-30T01:41:28.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.835+0000 7fcb797fa700 1 -- 172.21.15.31:0/1210206236 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb7c1dc3e0 con 0x7fcb7c06d040 2024-03-30T01:41:28.841 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb6006ac90 msgr2=0x7fcb6006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb6006ac90 0x7fcb6006d150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c06d460 tx=0x7fcb68013000 comp rx=0 tx=0).stop 2024-03-30T01:41:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 msgr2=0x7fcb7c149c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 0x7fcb7c149c50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000c450 tx=0x7fcb7000c810 comp rx=0 tx=0).stop 2024-03-30T01:41:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 shutdown_connections 2024-03-30T01:41:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb7c06d040 0x7fcb7c149c50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb6006ac90 0x7fcb6006d150 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb7c1549a0 0x7fcb7c1e92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 --2- 172.21.15.31:0/1210206236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb7c06d900 0x7fcb7c1e8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:28.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 >> 172.21.15.31:0/1210206236 conn(0x7fcb7c0c3980 msgr2=0x7fcb7c154d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:28.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 shutdown_connections 2024-03-30T01:41:28.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.838+0000 7fcb82e6c700 1 -- 172.21.15.31:0/1210206236 wait complete. 2024-03-30T01:41:28.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.991+0000 7f8a6bebb700 1 Processor -- start 2024-03-30T01:41:28.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.991+0000 7f8a6bebb700 1 -- start start 2024-03-30T01:41:28.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6bebb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c3aa0 0x7f8a64158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6bebb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a6406d510 0x7f8a640c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6bebb700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a64159b00 con 0x7f8a640c4a60 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6bebb700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6415c5e0 con 0x7f8a6406d510 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6bebb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640c45d0 con 0x7f8a640c3aa0 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6a458700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a6406d510 0x7f8a640c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6a458700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a6406d510 0x7f8a640c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51730/0 (socket says 172.21.15.31:51730) 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a69c57700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8a640c4a60 0x7f8a6415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35344/0 (socket says 172.21.15.31:35344) 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6a458700 1 -- 172.21.15.31:0/3169074870 learned_addr learned my addr 172.21.15.31:0/3169074870 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a69456700 1 --2- 172.21.15.31:0/3169074870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c3aa0 0x7f8a64158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6a458700 1 -- 172.21.15.31:0/3169074870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c3aa0 msgr2=0x7f8a64158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:29.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6a458700 1 --2- 172.21.15.31:0/3169074870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c3aa0 0x7f8a64158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6a458700 1 -- 172.21.15.31:0/3169074870 >> v1:172.21.15.31:6789/0 conn(0x7f8a640c4a60 legacy=0x7f8a6415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a6a458700 1 -- 172.21.15.31:0/3169074870 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a640c4010 con 0x7f8a6406d510 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.992+0000 7f8a68c55700 1 -- 172.21.15.31:0/3169074870 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042564163 0 0) 0x7f8a60001110 con 0x7f8a640c4a60 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a6a458700 1 --2- 172.21.15.31:0/3169074870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 0x7f8a640c6c70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009e50 tx=0x7f8a5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91dc3802da541b07 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a68c55700 1 -- 172.21.15.31:0/3169074870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00e070 con 0x7f8a6406d510 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a68c55700 1 -- 172.21.15.31:0/3169074870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c00b9c0 con 0x7f8a6406d510 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a68c55700 1 -- 172.21.15.31:0/3169074870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00bd70 con 0x7f8a6406d510 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a6bebb700 1 -- 172.21.15.31:0/3169074870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 msgr2=0x7f8a640c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a6bebb700 1 --2- 172.21.15.31:0/3169074870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 0x7f8a640c6c70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009e50 tx=0x7f8a5c00b330 comp rx=0 tx=0).stop 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a6bebb700 1 -- 172.21.15.31:0/3169074870 shutdown_connections 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a6bebb700 1 --2- 172.21.15.31:0/3169074870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 0x7f8a640c6c70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a6bebb700 1 --2- 172.21.15.31:0/3169074870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c3aa0 0x7f8a64158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a6bebb700 1 -- 172.21.15.31:0/3169074870 >> 172.21.15.31:0/3169074870 conn(0x7f8a640c7d60 msgr2=0x7f8a640c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:29.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a6bebb700 1 -- 172.21.15.31:0/3169074870 shutdown_connections 2024-03-30T01:41:29.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.993+0000 7f8a6bebb700 1 -- 172.21.15.31:0/3169074870 wait complete. 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.995+0000 7f8a6bebb700 1 Processor -- start 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.995+0000 7f8a6bebb700 1 -- start start 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.995+0000 7f8a6bebb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 0x7f8a640a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.995+0000 7f8a6bebb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a640c3aa0 0x7f8a640c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.995+0000 7f8a6bebb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c4a60 0x7f8a640c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.995+0000 7f8a6bebb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640abc80 con 0x7f8a6406d510 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.995+0000 7f8a6bebb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640ac490 con 0x7f8a640c3aa0 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.995+0000 7f8a6bebb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640ad060 con 0x7f8a640c4a60 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69c57700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 0x7f8a640a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69c57700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 0x7f8a640a8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51736/0 (socket says 172.21.15.31:51736) 2024-03-30T01:41:29.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69c57700 1 -- 172.21.15.31:0/1921728346 learned_addr learned my addr 172.21.15.31:0/1921728346 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69456700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a640c3aa0 0x7f8a640c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a6a458700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c4a60 0x7f8a640c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69c57700 1 -- 172.21.15.31:0/1921728346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c4a60 msgr2=0x7f8a640c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69c57700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c4a60 0x7f8a640c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69c57700 1 -- 172.21.15.31:0/1921728346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a640c3aa0 msgr2=0x7f8a640c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69c57700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a640c3aa0 0x7f8a640c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69c57700 1 -- 172.21.15.31:0/1921728346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c009b30 con 0x7f8a6406d510 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a69c57700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 0x7f8a640a8b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c550 tx=0x7f8a6000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a5a7fc700 1 -- 172.21.15.31:0/1921728346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60004c80 con 0x7f8a6406d510 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a5a7fc700 1 -- 172.21.15.31:0/1921728346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60004e00 con 0x7f8a6406d510 2024-03-30T01:41:29.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a640abed0 con 0x7f8a6406d510 2024-03-30T01:41:29.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a5a7fc700 1 -- 172.21.15.31:0/1921728346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60007310 con 0x7f8a6406d510 2024-03-30T01:41:29.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.996+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a640a90b0 con 0x7f8a6406d510 2024-03-30T01:41:29.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.997+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a641f1290 con 0x7f8a6406d510 2024-03-30T01:41:29.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.998+0000 7f8a5a7fc700 1 -- 172.21.15.31:0/1921728346 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8a60007470 con 0x7f8a6406d510 2024-03-30T01:41:29.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.998+0000 7f8a5a7fc700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8a4806ac90 0x7f8a4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.998+0000 7f8a5a7fc700 1 -- 172.21.15.31:0/1921728346 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8a60088fa0 con 0x7f8a6406d510 2024-03-30T01:41:29.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.998+0000 7f8a69456700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8a4806ac90 0x7f8a4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:28.999+0000 7f8a69456700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8a4806ac90 0x7f8a4806d150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8a640a77f0 tx=0x7f8a54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:29.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.003+0000 7f8a5a7fc700 1 -- 172.21.15.31:0/1921728346 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8a600bc580 con 0x7f8a6406d510 2024-03-30T01:41:29.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.218+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a64159b00 con 0x7f8a6406d510 2024-03-30T01:41:29.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.219+0000 7f8a5a7fc700 1 -- 172.21.15.31:0/1921728346 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a64159b00 con 0x7f8a6406d510 2024-03-30T01:41:29.225 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:29.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8a4806ac90 msgr2=0x7f8a4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8a4806ac90 0x7f8a4806d150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8a640a77f0 tx=0x7f8a54008040 comp rx=0 tx=0).stop 2024-03-30T01:41:29.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 msgr2=0x7f8a640a8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 0x7f8a640a8b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c550 tx=0x7f8a6000c860 comp rx=0 tx=0).stop 2024-03-30T01:41:29.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 shutdown_connections 2024-03-30T01:41:29.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a6406d510 0x7f8a640a8b70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8a4806ac90 0x7f8a4806d150 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a640c4a60 0x7f8a640c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 --2- 172.21.15.31:0/1921728346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a640c3aa0 0x7f8a640c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 >> 172.21.15.31:0/1921728346 conn(0x7f8a640c7d60 msgr2=0x7f8a640684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:29.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 shutdown_connections 2024-03-30T01:41:29.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.221+0000 7f8a6bebb700 1 -- 172.21.15.31:0/1921728346 wait complete. 2024-03-30T01:41:29.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:29.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.373+0000 7fc79ca7c700 1 Processor -- start 2024-03-30T01:41:29.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.373+0000 7fc79ca7c700 1 -- start start 2024-03-30T01:41:29.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.374+0000 7fc79ca7c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc798151ea0 0x7fc7980c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.374+0000 7fc79ca7c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980c07f0 0x7fc79806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.374+0000 7fc79ca7c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc798155000 con 0x7fc7980721f0 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.374+0000 7fc79ca7c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79806e930 con 0x7fc798151ea0 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.374+0000 7fc79ca7c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79806df40 con 0x7fc7980c07f0 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc798151ea0 0x7fc7980c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc798151ea0 0x7fc7980c02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51750/0 (socket says 172.21.15.31:51750) 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79659c700 1 -- 172.21.15.31:0/3036174798 learned_addr learned my addr 172.21.15.31:0/3036174798 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc796d9d700 1 --2- 172.21.15.31:0/3036174798 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980c07f0 0x7fc79806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79659c700 1 -- 172.21.15.31:0/3036174798 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980c07f0 msgr2=0x7fc79806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79659c700 1 --2- 172.21.15.31:0/3036174798 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980c07f0 0x7fc79806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79659c700 1 -- 172.21.15.31:0/3036174798 >> v1:172.21.15.59:6789/0 conn(0x7fc7980721f0 legacy=0x7fc79806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79659c700 1 -- 172.21.15.31:0/3036174798 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79806e650 con 0x7fc798151ea0 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79659c700 1 --2- 172.21.15.31:0/3036174798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 0x7fc7980c02b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc7800098f0 tx=0x7fc780009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beda5018e21c4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79559a700 1 -- 172.21.15.31:0/3036174798 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780011830 con 0x7fc798151ea0 2024-03-30T01:41:29.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79559a700 1 -- 172.21.15.31:0/3036174798 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780011990 con 0x7fc798151ea0 2024-03-30T01:41:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.375+0000 7fc79559a700 1 -- 172.21.15.31:0/3036174798 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780011d20 con 0x7fc798151ea0 2024-03-30T01:41:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.376+0000 7fc79ca7c700 1 -- 172.21.15.31:0/3036174798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 msgr2=0x7fc7980c02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.376+0000 7fc79ca7c700 1 --2- 172.21.15.31:0/3036174798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 0x7fc7980c02b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc7800098f0 tx=0x7fc780009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.376+0000 7fc79ca7c700 1 -- 172.21.15.31:0/3036174798 shutdown_connections 2024-03-30T01:41:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.376+0000 7fc79ca7c700 1 --2- 172.21.15.31:0/3036174798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 0x7fc7980c02b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.376+0000 7fc79ca7c700 1 --2- 172.21.15.31:0/3036174798 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980c07f0 0x7fc79806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.376+0000 7fc79ca7c700 1 -- 172.21.15.31:0/3036174798 >> 172.21.15.31:0/3036174798 conn(0x7fc7980c3980 msgr2=0x7fc7980c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.376+0000 7fc79ca7c700 1 -- 172.21.15.31:0/3036174798 shutdown_connections 2024-03-30T01:41:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.376+0000 7fc79ca7c700 1 -- 172.21.15.31:0/3036174798 wait complete. 2024-03-30T01:41:29.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.377+0000 7fc79ca7c700 1 Processor -- start 2024-03-30T01:41:29.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.377+0000 7fc79ca7c700 1 -- start start 2024-03-30T01:41:29.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc79ca7c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980721f0 0x7fc798157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc79ca7c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7980c07f0 0x7fc798071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc79ca7c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 0x7fc79814c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc79ca7c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7981eacd0 con 0x7fc798151ea0 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc79ca7c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7981ea150 con 0x7fc7980c07f0 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc79ca7c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7981ea430 con 0x7fc7980721f0 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc796d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 0x7fc79814c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc796d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 0x7fc79814c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51762/0 (socket says 172.21.15.31:51762) 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc796d9d700 1 -- 172.21.15.31:0/4138580470 learned_addr learned my addr 172.21.15.31:0/4138580470 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc795d9b700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7980c07f0 0x7fc798071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.378+0000 7fc79659c700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980721f0 0x7fc798157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc796d9d700 1 -- 172.21.15.31:0/4138580470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980721f0 msgr2=0x7fc798157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc796d9d700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980721f0 0x7fc798157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc796d9d700 1 -- 172.21.15.31:0/4138580470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7980c07f0 msgr2=0x7fc798071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc796d9d700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7980c07f0 0x7fc798071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc796d9d700 1 -- 172.21.15.31:0/4138580470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780009550 con 0x7fc798151ea0 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc796d9d700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 0x7fc79814c1c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc79000efc0 tx=0x7fc79000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc78effd700 1 -- 172.21.15.31:0/4138580470 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79000cc90 con 0x7fc798151ea0 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc78effd700 1 -- 172.21.15.31:0/4138580470 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79000f040 con 0x7fc798151ea0 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7981ea710 con 0x7fc798151ea0 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc78effd700 1 -- 172.21.15.31:0/4138580470 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7900146c0 con 0x7fc798151ea0 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.379+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc798071990 con 0x7fc798151ea0 2024-03-30T01:41:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.380+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7980747d0 con 0x7fc798151ea0 2024-03-30T01:41:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.381+0000 7fc78effd700 1 -- 172.21.15.31:0/4138580470 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc790014820 con 0x7fc798151ea0 2024-03-30T01:41:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.381+0000 7fc78effd700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc77806ac90 0x7fc77806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.381+0000 7fc78effd700 1 -- 172.21.15.31:0/4138580470 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc79008a140 con 0x7fc798151ea0 2024-03-30T01:41:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.381+0000 7fc79659c700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc77806ac90 0x7fc77806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.382+0000 7fc79659c700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc77806ac90 0x7fc77806d150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc780014040 tx=0x7fc780008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:29.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.386+0000 7fc78effd700 1 -- 172.21.15.31:0/4138580470 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc7900550c0 con 0x7fc798151ea0 2024-03-30T01:41:29.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.599+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc7980c2d60 con 0x7fc798151ea0 2024-03-30T01:41:29.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.599+0000 7fc78effd700 1 -- 172.21.15.31:0/4138580470 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc790009ca0 con 0x7fc798151ea0 2024-03-30T01:41:29.606 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc77806ac90 msgr2=0x7fc77806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc77806ac90 0x7fc77806d150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc780014040 tx=0x7fc780008030 comp rx=0 tx=0).stop 2024-03-30T01:41:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 msgr2=0x7fc79814c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 0x7fc79814c1c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc79000efc0 tx=0x7fc79000c3a0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 shutdown_connections 2024-03-30T01:41:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc798151ea0 0x7fc79814c1c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc77806ac90 0x7fc77806d150 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7980721f0 0x7fc798157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 --2- 172.21.15.31:0/4138580470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7980c07f0 0x7fc798071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 >> 172.21.15.31:0/4138580470 conn(0x7fc7980c3980 msgr2=0x7fc7980c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 shutdown_connections 2024-03-30T01:41:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.602+0000 7fc79ca7c700 1 -- 172.21.15.31:0/4138580470 wait complete. 2024-03-30T01:41:29.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:29.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.756+0000 7f2c8110a700 1 Processor -- start 2024-03-30T01:41:29.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.756+0000 7f2c8110a700 1 -- start start 2024-03-30T01:41:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c8110a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c06ae10 0x7f2c7c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c8110a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c7c070d40 0x7f2c7c0717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c8110a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c14ee10 con 0x7f2c7c14ea30 2024-03-30T01:41:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c8110a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7c0c1b10 con 0x7f2c7c070d40 2024-03-30T01:41:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c8110a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0c2ab0 con 0x7f2c7c06ae10 2024-03-30T01:41:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c7ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c7c14ea30 0x7f2c7c06a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35352/0 (socket says 172.21.15.31:35352) 2024-03-30T01:41:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c7ad9d700 1 -- 172.21.15.31:0/4124914073 learned_addr learned my addr 172.21.15.31:0/4124914073 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:29.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c7b59e700 1 --2- 172.21.15.31:0/4124914073 >> v2:172.21.15.59:3300/0 conn(0x7f2c7c070d40 0x7f2c7c0717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c7a59c700 1 --2- 172.21.15.31:0/4124914073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c06ae10 0x7f2c7c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c7b59e700 1 -- 172.21.15.31:0/4124914073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c06ae10 msgr2=0x7f2c7c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c7b59e700 1 --2- 172.21.15.31:0/4124914073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c06ae10 0x7f2c7c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c7b59e700 1 -- 172.21.15.31:0/4124914073 >> v1:172.21.15.31:6789/0 conn(0x7f2c7c14ea30 legacy=0x7f2c7c06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c7b59e700 1 -- 172.21.15.31:0/4124914073 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c0c2d90 con 0x7f2c7c070d40 2024-03-30T01:41:29.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.757+0000 7f2c7b59e700 1 --2- 172.21.15.31:0/4124914073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c7c070d40 0x7f2c7c0717d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009d90 tx=0x7f2c7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d137bd16011dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.759+0000 7f2c79d9b700 1 -- 172.21.15.31:0/4124914073 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400e070 con 0x7f2c7c070d40 2024-03-30T01:41:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.759+0000 7f2c79d9b700 1 -- 172.21.15.31:0/4124914073 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7400bb20 con 0x7f2c7c070d40 2024-03-30T01:41:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.759+0000 7f2c79d9b700 1 -- 172.21.15.31:0/4124914073 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c74012430 con 0x7f2c7c070d40 2024-03-30T01:41:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.760+0000 7f2c8110a700 1 -- 172.21.15.31:0/4124914073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c7c070d40 msgr2=0x7f2c7c0717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.760+0000 7f2c8110a700 1 --2- 172.21.15.31:0/4124914073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c7c070d40 0x7f2c7c0717d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009d90 tx=0x7f2c7400b330 comp rx=0 tx=0).stop 2024-03-30T01:41:29.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.760+0000 7f2c8110a700 1 -- 172.21.15.31:0/4124914073 shutdown_connections 2024-03-30T01:41:29.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.760+0000 7f2c8110a700 1 --2- 172.21.15.31:0/4124914073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c7c070d40 0x7f2c7c0717d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.760+0000 7f2c8110a700 1 --2- 172.21.15.31:0/4124914073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c06ae10 0x7f2c7c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.760+0000 7f2c8110a700 1 -- 172.21.15.31:0/4124914073 >> 172.21.15.31:0/4124914073 conn(0x7f2c7c0c65c0 msgr2=0x7f2c7c0c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:29.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.760+0000 7f2c8110a700 1 -- 172.21.15.31:0/4124914073 shutdown_connections 2024-03-30T01:41:29.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.760+0000 7f2c8110a700 1 -- 172.21.15.31:0/4124914073 wait complete. 2024-03-30T01:41:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.761+0000 7f2c8110a700 1 Processor -- start 2024-03-30T01:41:29.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.761+0000 7f2c8110a700 1 -- start start 2024-03-30T01:41:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c8110a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c06ae10 0x7f2c7c154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c8110a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c7c070d40 0x7f2c7c1e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c8110a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c14ea30 0x7f2c7c1e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c8110a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c1e7770 con 0x7f2c7c06ae10 2024-03-30T01:41:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c8110a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c1555d0 con 0x7f2c7c070d40 2024-03-30T01:41:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c8110a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c154ed0 con 0x7f2c7c14ea30 2024-03-30T01:41:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c7ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c06ae10 0x7f2c7c154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c7ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c06ae10 0x7f2c7c154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51774/0 (socket says 172.21.15.31:51774) 2024-03-30T01:41:29.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c7ad9d700 1 -- 172.21.15.31:0/238691168 learned_addr learned my addr 172.21.15.31:0/238691168 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c7b59e700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c14ea30 0x7f2c7c1e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.762+0000 7f2c7a59c700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c7c070d40 0x7f2c7c1e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c7ad9d700 1 -- 172.21.15.31:0/238691168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c14ea30 msgr2=0x7f2c7c1e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c7ad9d700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c14ea30 0x7f2c7c1e8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c7ad9d700 1 -- 172.21.15.31:0/238691168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c7c070d40 msgr2=0x7f2c7c1e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c7ad9d700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c7c070d40 0x7f2c7c1e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c7ad9d700 1 -- 172.21.15.31:0/238691168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c74009a70 con 0x7f2c7c06ae10 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c7ad9d700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c06ae10 0x7f2c7c154600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00c520 tx=0x7f2c6c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c637fe700 1 -- 172.21.15.31:0/238691168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c005d70 con 0x7f2c7c06ae10 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c637fe700 1 -- 172.21.15.31:0/238691168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c0070d0 con 0x7f2c7c06ae10 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c1eb6f0 con 0x7f2c7c06ae10 2024-03-30T01:41:29.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c637fe700 1 -- 172.21.15.31:0/238691168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c007440 con 0x7f2c7c06ae10 2024-03-30T01:41:29.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.763+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c0bd7f0 con 0x7f2c7c06ae10 2024-03-30T01:41:29.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.764+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c066d90 con 0x7f2c7c06ae10 2024-03-30T01:41:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.765+0000 7f2c637fe700 1 -- 172.21.15.31:0/238691168 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2c6c0075a0 con 0x7f2c7c06ae10 2024-03-30T01:41:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.765+0000 7f2c637fe700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c6806ac90 0x7f2c6806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.765+0000 7f2c637fe700 1 -- 172.21.15.31:0/238691168 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c6c088810 con 0x7f2c7c06ae10 2024-03-30T01:41:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.765+0000 7f2c7a59c700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c6806ac90 0x7f2c6806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:29.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.766+0000 7f2c7a59c700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c6806ac90 0x7f2c6806d150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0bdc60 tx=0x7f2c70009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:29.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.770+0000 7f2c637fe700 1 -- 172.21.15.31:0/238691168 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2c6c053840 con 0x7f2c7c06ae10 2024-03-30T01:41:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.984+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2c7c1eb550 con 0x7f2c7c06ae10 2024-03-30T01:41:29.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.984+0000 7f2c637fe700 1 -- 172.21.15.31:0/238691168 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2c6c017020 con 0x7f2c7c06ae10 2024-03-30T01:41:29.990 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c6806ac90 msgr2=0x7f2c6806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c6806ac90 0x7f2c6806d150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0bdc60 tx=0x7f2c70009210 comp rx=0 tx=0).stop 2024-03-30T01:41:29.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c06ae10 msgr2=0x7f2c7c154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c06ae10 0x7f2c7c154600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00c520 tx=0x7f2c6c00c830 comp rx=0 tx=0).stop 2024-03-30T01:41:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 shutdown_connections 2024-03-30T01:41:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c7c06ae10 0x7f2c7c154600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2c6806ac90 0x7f2c6806d150 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c7c14ea30 0x7f2c7c1e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 --2- 172.21.15.31:0/238691168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c7c070d40 0x7f2c7c1e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 >> 172.21.15.31:0/238691168 conn(0x7f2c7c0c65c0 msgr2=0x7f2c7c071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 shutdown_connections 2024-03-30T01:41:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:29.987+0000 7f2c8110a700 1 -- 172.21.15.31:0/238691168 wait complete. 2024-03-30T01:41:30.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:30.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.136+0000 7fe579ba7700 1 Processor -- start 2024-03-30T01:41:30.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.136+0000 7fe579ba7700 1 -- start start 2024-03-30T01:41:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe579ba7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe57406d9e0 0x7fe57406a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe579ba7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 0x7fe5740ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe579ba7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57406e4a0 con 0x7fe57414fbf0 2024-03-30T01:41:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe579ba7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe574156ac0 con 0x7fe57406d9e0 2024-03-30T01:41:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe579ba7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe574155bf0 con 0x7fe5740a9560 2024-03-30T01:41:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe572ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe57406d9e0 0x7fe57406a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe572ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe57406d9e0 0x7fe57406a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51780/0 (socket says 172.21.15.31:51780) 2024-03-30T01:41:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe572ffd700 1 -- 172.21.15.31:0/4147247392 learned_addr learned my addr 172.21.15.31:0/4147247392 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:30.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe573fff700 1 --2- 172.21.15.31:0/4147247392 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 0x7fe5740ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe572ffd700 1 -- 172.21.15.31:0/4147247392 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 msgr2=0x7fe5740ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe572ffd700 1 --2- 172.21.15.31:0/4147247392 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 0x7fe5740ab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe572ffd700 1 -- 172.21.15.31:0/4147247392 >> v1:172.21.15.59:6789/0 conn(0x7fe57414fbf0 legacy=0x7fe5741563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe572ffd700 1 -- 172.21.15.31:0/4147247392 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57406dee0 con 0x7fe57406d9e0 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.137+0000 7fe573fff700 1 --2- 172.21.15.31:0/4147247392 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 0x7fe5740ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe572ffd700 1 --2- 172.21.15.31:0/4147247392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 0x7fe57406a700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe568006fd0 tx=0x7fe568009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d510b1a721d2b7a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe5727fc700 1 -- 172.21.15.31:0/4147247392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568005e30 con 0x7fe57406d9e0 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe5727fc700 1 -- 172.21.15.31:0/4147247392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568004030 con 0x7fe57406d9e0 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe5727fc700 1 -- 172.21.15.31:0/4147247392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680043e0 con 0x7fe57406d9e0 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe579ba7700 1 -- 172.21.15.31:0/4147247392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 msgr2=0x7fe57406a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe579ba7700 1 --2- 172.21.15.31:0/4147247392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 0x7fe57406a700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe568006fd0 tx=0x7fe568009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe579ba7700 1 -- 172.21.15.31:0/4147247392 shutdown_connections 2024-03-30T01:41:30.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe579ba7700 1 --2- 172.21.15.31:0/4147247392 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 0x7fe5740ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe579ba7700 1 --2- 172.21.15.31:0/4147247392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 0x7fe57406a700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.138+0000 7fe579ba7700 1 -- 172.21.15.31:0/4147247392 >> 172.21.15.31:0/4147247392 conn(0x7fe5740c3190 msgr2=0x7fe5740c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:30.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.139+0000 7fe579ba7700 1 -- 172.21.15.31:0/4147247392 shutdown_connections 2024-03-30T01:41:30.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.139+0000 7fe579ba7700 1 -- 172.21.15.31:0/4147247392 wait complete. 2024-03-30T01:41:30.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.140+0000 7fe579ba7700 1 Processor -- start 2024-03-30T01:41:30.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.140+0000 7fe579ba7700 1 -- start start 2024-03-30T01:41:30.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.141+0000 7fe579ba7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 0x7fe5741e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.141+0000 7fe579ba7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 0x7fe57406b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.141+0000 7fe579ba7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57414fbf0 0x7fe5740a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.141+0000 7fe579ba7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5741e8330 con 0x7fe57406d9e0 2024-03-30T01:41:30.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.141+0000 7fe579ba7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740c70d0 con 0x7fe57414fbf0 2024-03-30T01:41:30.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.141+0000 7fe579ba7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740c6ac0 con 0x7fe5740a9560 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.141+0000 7fe5737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 0x7fe5741e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.141+0000 7fe5737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 0x7fe5741e7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51790/0 (socket says 172.21.15.31:51790) 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.141+0000 7fe5737fe700 1 -- 172.21.15.31:0/3361743947 learned_addr learned my addr 172.21.15.31:0/3361743947 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe572ffd700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 0x7fe57406b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe573fff700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57414fbf0 0x7fe5740a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe5737fe700 1 -- 172.21.15.31:0/3361743947 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 msgr2=0x7fe57406b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe5737fe700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 0x7fe57406b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe5737fe700 1 -- 172.21.15.31:0/3361743947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57414fbf0 msgr2=0x7fe5740a74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe5737fe700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57414fbf0 0x7fe5740a74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe5737fe700 1 -- 172.21.15.31:0/3361743947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568009500 con 0x7fe57406d9e0 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe5737fe700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 0x7fe5741e7df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe56400c450 tx=0x7fe56400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe55bfff700 1 -- 172.21.15.31:0/3361743947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640052d0 con 0x7fe57406d9e0 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe55bfff700 1 -- 172.21.15.31:0/3361743947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe564005430 con 0x7fe57406d9e0 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5740c46a0 con 0x7fe57406d9e0 2024-03-30T01:41:30.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe55bfff700 1 -- 172.21.15.31:0/3361743947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640038c0 con 0x7fe57406d9e0 2024-03-30T01:41:30.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.142+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5740c6e10 con 0x7fe57406d9e0 2024-03-30T01:41:30.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.143+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5740a7340 con 0x7fe57406d9e0 2024-03-30T01:41:30.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.144+0000 7fe55bfff700 1 -- 172.21.15.31:0/3361743947 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe564003a20 con 0x7fe57406d9e0 2024-03-30T01:41:30.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.144+0000 7fe55bfff700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe56006ac90 0x7fe56006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.144+0000 7fe55bfff700 1 -- 172.21.15.31:0/3361743947 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe5640888a0 con 0x7fe57406d9e0 2024-03-30T01:41:30.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.144+0000 7fe572ffd700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe56006ac90 0x7fe56006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.145+0000 7fe572ffd700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe56006ac90 0x7fe56006d150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe57406de10 tx=0x7fe568008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:30.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.149+0000 7fe55bfff700 1 -- 172.21.15.31:0/3361743947 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe564053820 con 0x7fe57406d9e0 2024-03-30T01:41:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.411+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe57404ea80 con 0x7fe57406d9e0 2024-03-30T01:41:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.411+0000 7fe55bfff700 1 -- 172.21.15.31:0/3361743947 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4727 (secure 0 0 0) 0x7fe56400ce60 con 0x7fe57406d9e0 2024-03-30T01:41:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:30.417 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:16.755934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T01:41:30.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe56006ac90 msgr2=0x7fe56006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe56006ac90 0x7fe56006d150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe57406de10 tx=0x7fe568008070 comp rx=0 tx=0).stop 2024-03-30T01:41:30.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 msgr2=0x7fe5741e7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 0x7fe5741e7df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe56400c450 tx=0x7fe56400c760 comp rx=0 tx=0).stop 2024-03-30T01:41:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 shutdown_connections 2024-03-30T01:41:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe57406d9e0 0x7fe5741e7df0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe56006ac90 0x7fe56006d150 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57414fbf0 0x7fe5740a74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 --2- 172.21.15.31:0/3361743947 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5740a9560 0x7fe57406b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 >> 172.21.15.31:0/3361743947 conn(0x7fe5740c3190 msgr2=0x7fe5741540a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 shutdown_connections 2024-03-30T01:41:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.414+0000 7fe579ba7700 1 -- 172.21.15.31:0/3361743947 wait complete. 2024-03-30T01:41:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-03-30T01:41:30.434 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:30.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.565+0000 7f8d75e2f700 1 Processor -- start 2024-03-30T01:41:30.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.565+0000 7f8d75e2f700 1 -- start start 2024-03-30T01:41:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.566+0000 7f8d75e2f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8d700bc2a0 0x7f8d700c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.566+0000 7f8d75e2f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7006d0c0 con 0x7f8d7006d930 2024-03-30T01:41:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.566+0000 7f8d75e2f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7006d3a0 con 0x7f8d7006f680 2024-03-30T01:41:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.566+0000 7f8d75e2f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7006d680 con 0x7f8d700bc2a0 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.566+0000 7f8d6ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8d700bc2a0 0x7f8d700c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.566+0000 7f8d6effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8d7006d930 0x7f8d70152160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35360/0 (socket says 172.21.15.31:35360) 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.566+0000 7f8d6ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8d700bc2a0 0x7f8d700c2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51794/0 (socket says 172.21.15.31:51794) 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.566+0000 7f8d6effd700 1 -- 172.21.15.31:0/2985997117 learned_addr learned my addr 172.21.15.31:0/2985997117 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.567+0000 7f8d6ffff700 1 -- 172.21.15.31:0/2985997117 >> v1:172.21.15.59:6789/0 conn(0x7f8d7006f680 legacy=0x7f8d700bce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.567+0000 7f8d6ffff700 1 -- 172.21.15.31:0/2985997117 >> v1:172.21.15.31:6789/0 conn(0x7f8d7006d930 legacy=0x7f8d70152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.567+0000 7f8d6ffff700 1 -- 172.21.15.31:0/2985997117 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d70151980 con 0x7f8d700bc2a0 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.567+0000 7f8d6ffff700 1 --2- 172.21.15.31:0/2985997117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 0x7f8d700c2ac0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009d90 tx=0x7f8d6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b85b61840d74598 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.567+0000 7f8d6e7fc700 1 -- 172.21.15.31:0/2985997117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800e070 con 0x7f8d700bc2a0 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.567+0000 7f8d6e7fc700 1 -- 172.21.15.31:0/2985997117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6800bbd0 con 0x7f8d700bc2a0 2024-03-30T01:41:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.567+0000 7f8d6e7fc700 1 -- 172.21.15.31:0/2985997117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68012430 con 0x7f8d700bc2a0 2024-03-30T01:41:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.567+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2985997117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 msgr2=0x7f8d700c2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.567+0000 7f8d75e2f700 1 --2- 172.21.15.31:0/2985997117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 0x7f8d700c2ac0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009d90 tx=0x7f8d6800b330 comp rx=0 tx=0).stop 2024-03-30T01:41:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.568+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2985997117 shutdown_connections 2024-03-30T01:41:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.568+0000 7f8d75e2f700 1 --2- 172.21.15.31:0/2985997117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 0x7f8d700c2ac0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.568+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2985997117 >> 172.21.15.31:0/2985997117 conn(0x7f8d700c3980 msgr2=0x7f8d700c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.568+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2985997117 shutdown_connections 2024-03-30T01:41:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.568+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2985997117 wait complete. 2024-03-30T01:41:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.569+0000 7f8d75e2f700 1 Processor -- start 2024-03-30T01:41:30.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.569+0000 7f8d75e2f700 1 -- start start 2024-03-30T01:41:30.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d75e2f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d7006d930 0x7f8d7014bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d75e2f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d7006f680 0x7f8d701e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d75e2f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 0x7f8d701ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d75e2f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7014a1b0 con 0x7f8d700bc2a0 2024-03-30T01:41:30.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d75e2f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700c2060 con 0x7f8d7006d930 2024-03-30T01:41:30.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d75e2f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701ea700 con 0x7f8d7006f680 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 0x7f8d701ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 0x7f8d701ea100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51798/0 (socket says 172.21.15.31:51798) 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6ffff700 1 -- 172.21.15.31:0/2476080188 learned_addr learned my addr 172.21.15.31:0/2476080188 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6effd700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d7006f680 0x7f8d701e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6f7fe700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d7006d930 0x7f8d7014bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6ffff700 1 -- 172.21.15.31:0/2476080188 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d7006f680 msgr2=0x7f8d701e9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6ffff700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d7006f680 0x7f8d701e9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6ffff700 1 -- 172.21.15.31:0/2476080188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d7006d930 msgr2=0x7f8d7014bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6ffff700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d7006d930 0x7f8d7014bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6ffff700 1 -- 172.21.15.31:0/2476080188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68009a70 con 0x7f8d700bc2a0 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.570+0000 7f8d6f7fe700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d7006d930 0x7f8d7014bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.571+0000 7f8d6ffff700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 0x7f8d701ea100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d68014040 tx=0x7f8d68004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:30.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.571+0000 7f8d67fff700 1 -- 172.21.15.31:0/2476080188 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68012d20 con 0x7f8d700bc2a0 2024-03-30T01:41:30.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.571+0000 7f8d67fff700 1 -- 172.21.15.31:0/2476080188 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68020070 con 0x7f8d700bc2a0 2024-03-30T01:41:30.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.571+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d70152ae0 con 0x7f8d700bc2a0 2024-03-30T01:41:30.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.571+0000 7f8d67fff700 1 -- 172.21.15.31:0/2476080188 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68012ea0 con 0x7f8d700bc2a0 2024-03-30T01:41:30.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.571+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d701530a0 con 0x7f8d700bc2a0 2024-03-30T01:41:30.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.572+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d700c2200 con 0x7f8d700bc2a0 2024-03-30T01:41:30.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.572+0000 7f8d67fff700 1 -- 172.21.15.31:0/2476080188 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d68004280 con 0x7f8d700bc2a0 2024-03-30T01:41:30.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.573+0000 7f8d67fff700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d5006ac90 0x7f8d5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.573+0000 7f8d67fff700 1 -- 172.21.15.31:0/2476080188 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8d68022070 con 0x7f8d700bc2a0 2024-03-30T01:41:30.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.573+0000 7f8d6f7fe700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d5006ac90 0x7f8d5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.573+0000 7f8d6f7fe700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d5006ac90 0x7f8d5006d150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8d60007fd0 tx=0x7f8d60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:30.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.577+0000 7f8d67fff700 1 -- 172.21.15.31:0/2476080188 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d680562e0 con 0x7f8d700bc2a0 2024-03-30T01:41:30.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.792+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d7004ea80 con 0x7f8d700bc2a0 2024-03-30T01:41:30.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.793+0000 7f8d67fff700 1 -- 172.21.15.31:0/2476080188 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f8d68059900 con 0x7f8d700bc2a0 2024-03-30T01:41:30.799 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:30.799 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:30.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d5006ac90 msgr2=0x7f8d5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d5006ac90 0x7f8d5006d150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8d60007fd0 tx=0x7f8d60009040 comp rx=0 tx=0).stop 2024-03-30T01:41:30.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 msgr2=0x7f8d701ea100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 0x7f8d701ea100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d68014040 tx=0x7f8d68004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 shutdown_connections 2024-03-30T01:41:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d700bc2a0 0x7f8d701ea100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d5006ac90 0x7f8d5006d150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d7006d930 0x7f8d7014bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 --2- 172.21.15.31:0/2476080188 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d7006f680 0x7f8d701e9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.796+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 >> 172.21.15.31:0/2476080188 conn(0x7f8d700c3980 msgr2=0x7f8d700c4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.797+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 shutdown_connections 2024-03-30T01:41:30.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.797+0000 7f8d75e2f700 1 -- 172.21.15.31:0/2476080188 wait complete. 2024-03-30T01:41:30.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:30.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.947+0000 7f9b718d3700 1 Processor -- start 2024-03-30T01:41:30.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.947+0000 7f9b718d3700 1 -- start start 2024-03-30T01:41:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b718d3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b6c0c7270 0x7f9b6c0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b718d3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c1521d0 0x7f9b6c157da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b718d3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c06bf10 con 0x7f9b6c0c6330 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b718d3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6c0c5990 con 0x7f9b6c0c7270 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b718d3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c06b300 con 0x7f9b6c1521d0 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b6b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c1521d0 0x7f9b6c157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b6affd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b6c0c7270 0x7f9b6c0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b6b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c1521d0 0x7f9b6c157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:47078/0 (socket says 172.21.15.31:47078) 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b6affd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b6c0c7270 0x7f9b6c0c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:47112/0 (socket says 172.21.15.31:47112) 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b6b7fe700 1 -- 172.21.15.31:0/4054248024 learned_addr learned my addr 172.21.15.31:0/4054248024 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.948+0000 7f9b6a7fc700 1 --1- 172.21.15.31:0/4054248024 >> v1:172.21.15.59:6789/0 conn(0x7f9b6c0c6330 0x7f9b6c151ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:32976/0 (socket says 172.21.15.31:32976) 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b69ffb700 1 -- 172.21.15.31:0/4054248024 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393768715 0 0) 0x7f9b6c06bf10 con 0x7f9b6c0c6330 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b69ffb700 1 -- 172.21.15.31:0/4054248024 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b580035b0 con 0x7f9b6c0c6330 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b6b7fe700 1 -- 172.21.15.31:0/4054248024 >> v2:172.21.15.59:3300/0 conn(0x7f9b6c0c7270 msgr2=0x7f9b6c0c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b6b7fe700 1 --2- 172.21.15.31:0/4054248024 >> v2:172.21.15.59:3300/0 conn(0x7f9b6c0c7270 0x7f9b6c0c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b6b7fe700 1 -- 172.21.15.31:0/4054248024 >> v1:172.21.15.59:6789/0 conn(0x7f9b6c0c6330 legacy=0x7f9b6c151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b6b7fe700 1 -- 172.21.15.31:0/4054248024 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c06b5e0 con 0x7f9b6c1521d0 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b6b7fe700 1 --2- 172.21.15.31:0/4054248024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c1521d0 0x7f9b6c157da0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009e50 tx=0x7f9b6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21781f6224cb6e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b69ffb700 1 -- 172.21.15.31:0/4054248024 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6400e070 con 0x7f9b6c1521d0 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b69ffb700 1 -- 172.21.15.31:0/4054248024 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6400b910 con 0x7f9b6c1521d0 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.949+0000 7f9b69ffb700 1 -- 172.21.15.31:0/4054248024 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6400bcc0 con 0x7f9b6c1521d0 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.950+0000 7f9b718d3700 1 -- 172.21.15.31:0/4054248024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c1521d0 msgr2=0x7f9b6c157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.950+0000 7f9b718d3700 1 --2- 172.21.15.31:0/4054248024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c1521d0 0x7f9b6c157da0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009e50 tx=0x7f9b6400b330 comp rx=0 tx=0).stop 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.950+0000 7f9b718d3700 1 -- 172.21.15.31:0/4054248024 shutdown_connections 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.950+0000 7f9b718d3700 1 --2- 172.21.15.31:0/4054248024 >> v2:172.21.15.59:3300/0 conn(0x7f9b6c0c7270 0x7f9b6c0c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.950+0000 7f9b718d3700 1 --2- 172.21.15.31:0/4054248024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c1521d0 0x7f9b6c157da0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.950+0000 7f9b718d3700 1 -- 172.21.15.31:0/4054248024 >> 172.21.15.31:0/4054248024 conn(0x7f9b6c0abf60 msgr2=0x7f9b6c0ac370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.950+0000 7f9b718d3700 1 -- 172.21.15.31:0/4054248024 shutdown_connections 2024-03-30T01:41:30.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.950+0000 7f9b718d3700 1 -- 172.21.15.31:0/4054248024 wait complete. 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.951+0000 7f9b718d3700 1 Processor -- start 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.951+0000 7f9b718d3700 1 -- start start 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b718d3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c0c6330 0x7f9b6c14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b718d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b6c0c7270 0x7f9b6c14b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b718d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b6c1521d0 0x7f9b6c14b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b718d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c154070 con 0x7f9b6c0c7270 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b718d3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c1ecaa0 con 0x7f9b6c1521d0 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b718d3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0aeaa0 con 0x7f9b6c0c6330 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b6c0c7270 0x7f9b6c14b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b6c0c7270 0x7f9b6c14b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51806/0 (socket says 172.21.15.31:51806) 2024-03-30T01:41:30.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6a7fc700 1 -- 172.21.15.31:0/3585838838 learned_addr learned my addr 172.21.15.31:0/3585838838 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6b7fe700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b6c1521d0 0x7f9b6c14b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6affd700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c0c6330 0x7f9b6c14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6a7fc700 1 -- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c0c6330 msgr2=0x7f9b6c14e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6a7fc700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c0c6330 0x7f9b6c14e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6a7fc700 1 -- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b6c1521d0 msgr2=0x7f9b6c14b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6a7fc700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b6c1521d0 0x7f9b6c14b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6a7fc700 1 -- 172.21.15.31:0/3585838838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b64009b30 con 0x7f9b6c0c7270 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6affd700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c0c6330 0x7f9b6c14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.952+0000 7f9b6b7fe700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b6c1521d0 0x7f9b6c14b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.953+0000 7f9b6a7fc700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b6c0c7270 0x7f9b6c14b1c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c700 tx=0x7f9b5400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.953+0000 7f9b63fff700 1 -- 172.21.15.31:0/3585838838 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54005cb0 con 0x7f9b6c0c7270 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.953+0000 7f9b63fff700 1 -- 172.21.15.31:0/3585838838 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b54005e10 con 0x7f9b6c0c7270 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.953+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c0adf50 con 0x7f9b6c0c7270 2024-03-30T01:41:30.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.953+0000 7f9b63fff700 1 -- 172.21.15.31:0/3585838838 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b540072d0 con 0x7f9b6c0c7270 2024-03-30T01:41:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.953+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c0ae510 con 0x7f9b6c0c7270 2024-03-30T01:41:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.954+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c04ea80 con 0x7f9b6c0c7270 2024-03-30T01:41:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.954+0000 7f9b63fff700 1 -- 172.21.15.31:0/3585838838 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9b54007430 con 0x7f9b6c0c7270 2024-03-30T01:41:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.955+0000 7f9b63fff700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9b4c06ac90 0x7f9b4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.955+0000 7f9b63fff700 1 -- 172.21.15.31:0/3585838838 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9b54088a90 con 0x7f9b6c0c7270 2024-03-30T01:41:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.955+0000 7f9b6affd700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9b4c06ac90 0x7f9b4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:30.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.955+0000 7f9b6affd700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9b4c06ac90 0x7f9b4c06d150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009c30 tx=0x7f9b5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:30.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:30.959+0000 7f9b63fff700 1 -- 172.21.15.31:0/3585838838 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9b54053a40 con 0x7f9b6c0c7270 2024-03-30T01:41:31.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.221+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b6c14bc40 con 0x7f9b6c0c7270 2024-03-30T01:41:31.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.221+0000 7f9b63fff700 1 -- 172.21.15.31:0/3585838838 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f9b54057060 con 0x7f9b6c0c7270 2024-03-30T01:41:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:31.227 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:16.755934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T01:41:31.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9b4c06ac90 msgr2=0x7f9b4c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9b4c06ac90 0x7f9b4c06d150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009c30 tx=0x7f9b5c008040 comp rx=0 tx=0).stop 2024-03-30T01:41:31.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b6c0c7270 msgr2=0x7f9b6c14b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b6c0c7270 0x7f9b6c14b1c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c700 tx=0x7f9b5400cac0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 shutdown_connections 2024-03-30T01:41:31.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9b4c06ac90 0x7f9b4c06d150 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b6c1521d0 0x7f9b6c14b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b6c0c6330 0x7f9b6c14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 --2- 172.21.15.31:0/3585838838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b6c0c7270 0x7f9b6c14b1c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 >> 172.21.15.31:0/3585838838 conn(0x7f9b6c0abf60 msgr2=0x7f9b6c14c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:31.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 shutdown_connections 2024-03-30T01:41:31.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.224+0000 7f9b718d3700 1 -- 172.21.15.31:0/3585838838 wait complete. 2024-03-30T01:41:31.231 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-03-30T01:41:31.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T01:40:58.401733+0000', 'modified': '2024-03-30T01:41:16.755934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4269}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4269': {'gid': 4269, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6837/3724657244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6835', 'nonce': 3724657244}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 3724657244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T01:41:31.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T01:41:31.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T01:41:31.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.786+0000 7f0ce688d700 1 Processor -- start 2024-03-30T01:41:31.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.786+0000 7f0ce688d700 1 -- start start 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.787+0000 7f0ce688d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0151d20 0x7f0ce00705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.787+0000 7f0ce688d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ce00c5dc0 0x7f0ce0154880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.787+0000 7f0ce688d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce0070ae0 con 0x7f0ce0152100 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.787+0000 7f0ce688d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce0070d60 con 0x7f0ce00c5dc0 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.787+0000 7f0ce688d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0157e90 con 0x7f0ce0151d20 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.787+0000 7f0ce4e2a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ce00c5dc0 0x7f0ce0154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.787+0000 7f0ce4e2a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ce00c5dc0 0x7f0ce0154880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51808/0 (socket says 172.21.15.31:51808) 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.787+0000 7f0cdf7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0ce0152100 0x7f0ce00c4850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35372/0 (socket says 172.21.15.31:35372) 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.787+0000 7f0ce4e2a700 1 -- 172.21.15.31:0/2473115920 learned_addr learned my addr 172.21.15.31:0/2473115920 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:31.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2473115920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0151d20 0x7f0ce00705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0ce4e2a700 1 -- 172.21.15.31:0/2473115920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0151d20 msgr2=0x7f0ce00705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/2473115920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0151d20 0x7f0ce00705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0ce4e2a700 1 -- 172.21.15.31:0/2473115920 >> v1:172.21.15.31:6789/0 conn(0x7f0ce0152100 legacy=0x7f0ce00c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0ce4e2a700 1 -- 172.21.15.31:0/2473115920 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0158170 con 0x7f0ce00c5dc0 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0cdeffd700 1 -- 172.21.15.31:0/2473115920 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801900619 0 0) 0x7f0ce0070ae0 con 0x7f0ce0152100 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/2473115920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce00c5dc0 0x7f0ce0154880 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009e50 tx=0x7f0cd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a744852c69f6fd21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0cdeffd700 1 -- 172.21.15.31:0/2473115920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800e070 con 0x7f0ce00c5dc0 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0cdeffd700 1 -- 172.21.15.31:0/2473115920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd800b9c0 con 0x7f0ce00c5dc0 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.788+0000 7f0cdeffd700 1 -- 172.21.15.31:0/2473115920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800bd70 con 0x7f0ce00c5dc0 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.789+0000 7f0ce688d700 1 -- 172.21.15.31:0/2473115920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce00c5dc0 msgr2=0x7f0ce0154880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.789+0000 7f0ce688d700 1 --2- 172.21.15.31:0/2473115920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce00c5dc0 0x7f0ce0154880 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009e50 tx=0x7f0cd800b330 comp rx=0 tx=0).stop 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.789+0000 7f0ce688d700 1 -- 172.21.15.31:0/2473115920 shutdown_connections 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.789+0000 7f0ce688d700 1 --2- 172.21.15.31:0/2473115920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce00c5dc0 0x7f0ce0154880 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.789+0000 7f0ce688d700 1 --2- 172.21.15.31:0/2473115920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0151d20 0x7f0ce00705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.789+0000 7f0ce688d700 1 -- 172.21.15.31:0/2473115920 >> 172.21.15.31:0/2473115920 conn(0x7f0ce006ad30 msgr2=0x7f0ce00a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.789+0000 7f0ce688d700 1 -- 172.21.15.31:0/2473115920 shutdown_connections 2024-03-30T01:41:31.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.789+0000 7f0ce688d700 1 -- 172.21.15.31:0/2473115920 wait complete. 2024-03-30T01:41:31.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.790+0000 7f0ce688d700 1 Processor -- start 2024-03-30T01:41:31.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.790+0000 7f0ce688d700 1 -- start start 2024-03-30T01:41:31.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0ce688d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce00c5dc0 0x7f0ce014cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0ce688d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce0151d20 0x7f0ce00c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0ce688d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0152100 0x7f0ce01ed660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0ce688d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01ea7a0 con 0x7f0ce0151d20 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0ce688d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01eac90 con 0x7f0ce00c5dc0 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0ce688d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00acde0 con 0x7f0ce0152100 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce0151d20 0x7f0ce00c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce0151d20 0x7f0ce00c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51816/0 (socket says 172.21.15.31:51816) 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/2604467 learned_addr learned my addr 172.21.15.31:0/2604467 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce00c5dc0 0x7f0ce014cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0152100 0x7f0ce01ed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/2604467 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0152100 msgr2=0x7f0ce01ed660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0152100 0x7f0ce01ed660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/2604467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce00c5dc0 msgr2=0x7f0ce014cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce00c5dc0 0x7f0ce014cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.791+0000 7f0cdf7fe700 1 -- 172.21.15.31:0/2604467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd8009b30 con 0x7f0ce0151d20 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.792+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce0151d20 0x7f0ce00c7170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40011e0 tx=0x7f0cd400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.792+0000 7f0cdcff9700 1 -- 172.21.15.31:0/2604467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4004c80 con 0x7f0ce0151d20 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.792+0000 7f0cdcff9700 1 -- 172.21.15.31:0/2604467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd4004e00 con 0x7f0ce0151d20 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.792+0000 7f0ce688d700 1 -- 172.21.15.31:0/2604467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce01ea9b0 con 0x7f0ce0151d20 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.792+0000 7f0cdcff9700 1 -- 172.21.15.31:0/2604467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4007310 con 0x7f0ce0151d20 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.792+0000 7f0ce688d700 1 -- 172.21.15.31:0/2604467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce00abfd0 con 0x7f0ce0151d20 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.793+0000 7f0cdcff9700 1 -- 172.21.15.31:0/2604467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0cd4007470 con 0x7f0ce0151d20 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.794+0000 7f0cdcff9700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0ccc06ac90 0x7f0ccc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.794+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0ccc06ac90 0x7f0ccc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.794+0000 7f0cdcff9700 1 -- 172.21.15.31:0/2604467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0cd40893b0 con 0x7f0ce0151d20 2024-03-30T01:41:31.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.795+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0ccc06ac90 0x7f0ccc06d150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0ce01e4660 tx=0x7f0cd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:31.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.796+0000 7f0cf2a23b80 1 Processor -- start 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.796+0000 7f0cf2a23b80 1 -- start start 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f73768690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376e090 0x562f737aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 0x562f7378ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f736c4940 con 0x562f7376cff0 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f736c2300 con 0x562f7376e090 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f736bdbf0 con 0x562f7376d3d0 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cdffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f73768690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cdffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f73768690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51820/0 (socket says 172.21.15.31:51820) 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cdffff700 1 -- 172.21.15.31:0/3351315781 learned_addr learned my addr 172.21.15.31:0/3351315781 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/3351315781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376e090 0x562f737aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.797+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/3351315781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 0x562f7378ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.798+0000 7f0cdffff700 1 -- 172.21.15.31:0/3351315781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 msgr2=0x562f7378ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.798+0000 7f0cdffff700 1 --2- 172.21.15.31:0/3351315781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 0x562f7378ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.798+0000 7f0cdffff700 1 -- 172.21.15.31:0/3351315781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376e090 msgr2=0x562f737aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.798+0000 7f0cdffff700 1 --2- 172.21.15.31:0/3351315781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376e090 0x562f737aaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.798+0000 7f0cdffff700 1 -- 172.21.15.31:0/3351315781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f734ebb10 con 0x562f7376cff0 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.798+0000 7f0cdffff700 1 --2- 172.21.15.31:0/3351315781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f73768690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000bfd0 tx=0x7f0cd0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8f15482d450df server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.798+0000 7f0cc5ffb700 1 -- 172.21.15.31:0/3351315781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0014660 con 0x562f7376cff0 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.798+0000 7f0cc5ffb700 1 -- 172.21.15.31:0/3351315781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0019070 con 0x562f7376cff0 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.798+0000 7f0cc5ffb700 1 -- 172.21.15.31:0/3351315781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00149c0 con 0x562f7376cff0 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/3351315781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 msgr2=0x562f73768690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/3351315781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f73768690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000bfd0 tx=0x7f0cd0012000 comp rx=0 tx=0).stop 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/3351315781 shutdown_connections 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/3351315781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f73768690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/3351315781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 0x562f7378ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/3351315781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376e090 0x562f737aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/3351315781 >> 172.21.15.31:0/3351315781 conn(0x562f736cb650 msgr2=0x562f7376ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/3351315781 shutdown_connections 2024-03-30T01:41:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/3351315781 wait complete. 2024-03-30T01:41:31.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.799+0000 7f0cf2a23b80 1 -- start start 2024-03-30T01:41:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cf2a23b80 1 Processor -- start 2024-03-30T01:41:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 0x562f73788e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f73789b90 0x562f737a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f73659d30 con 0x562f7376cff0 2024-03-30T01:41:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f736599d0 con 0x562f7376d3d0 2024-03-30T01:41:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f73541650 con 0x562f73789b90 2024-03-30T01:41:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cf2a23b80 10 client.0 _ll_register_callbacks cb 0x562f73663420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T01:41:31.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cdffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cdffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51836/0 (socket says 172.21.15.31:51836) 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.800+0000 7f0cdffff700 1 -- 172.21.15.31:0/1983740590 learned_addr learned my addr 172.21.15.31:0/1983740590 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 0x562f73788e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f73789b90 0x562f737a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cdffff700 1 -- 172.21.15.31:0/1983740590 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f73789b90 msgr2=0x562f737a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cdffff700 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f73789b90 0x562f737a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cdffff700 1 -- 172.21.15.31:0/1983740590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 msgr2=0x562f73788e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cdffff700 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 0x562f73788e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cdffff700 1 -- 172.21.15.31:0/1983740590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd000e040 con 0x562f7376cff0 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cdffff700 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737a0df0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0022040 tx=0x7f0cd00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac0ad949487f72d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cbeffd700 1 -- 172.21.15.31:0/1983740590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0014e00 con 0x562f7376cff0 2024-03-30T01:41:31.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cbeffd700 1 -- 172.21.15.31:0/1983740590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0cd0034470 con 0x562f7376cff0 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/1983740590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f7367e330 con 0x562f7376cff0 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cbeffd700 1 -- 172.21.15.31:0/1983740590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd0034630 con 0x562f7376cff0 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cf2a23b80 10 client.4827 fetch_fsmap learned FSMap version 9 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.801+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/1983740590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x562f73683e50 con 0x562f7376cff0 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cbeffd700 1 -- 172.21.15.31:0/1983740590 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f7367e330 con 0x562f7376cff0 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cbeffd700 1 -- 172.21.15.31:0/1983740590 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f0cd0034790 con 0x562f7376cff0 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cf2a23b80 10 client.4827 fetch_fsmap finished waiting for FSMap version 9 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cf2a23b80 10 client.4827 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cf2a23b80 20 client.4827 populate_metadata read hostname 'smithi031' 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x562f737a1330 0x562f737bc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cf2a23b80 4 client.4827 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/1983740590 --> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f736b66b0 con 0x562f737a1330 2024-03-30T01:41:31.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x562f737a1330 0x562f737bc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cbeffd700 10 client.4827 ms_handle_connect on v2:172.21.15.31:6835/3724657244 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.802+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x562f737a1330 0x562f737bc3e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b60d8859904019 server_cookie=9ecdd5d7bbefc6e0 in_seq=0 out_seq=0 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cbeffd700 1 -- 172.21.15.31:0/1983740590 <== mds.0 v2:172.21.15.31:6835/3724657244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562f736b66b0 con 0x562f737a1330 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cbeffd700 10 client.4827 handle_command_reply: tid=0 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 client.4827 shutdown 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/1983740590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 msgr2=0x562f737a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737a0df0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0022040 tx=0x7f0cd00047c0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/1983740590 shutdown_connections 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x562f737a1330 0x562f737bc3e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737a0df0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f73789b90 0x562f737a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cbeffd700 0 client.4827 ms_handle_reset on v2:172.21.15.31:6835/3724657244 2024-03-30T01:41:31.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/1983740590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 0x562f73788e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/1983740590 >> 172.21.15.31:0/1983740590 conn(0x562f736cb650 msgr2=0x562f7376ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/1983740590 shutdown_connections 2024-03-30T01:41:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/1983740590 wait complete. 2024-03-30T01:41:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.803+0000 7f0cf2a23b80 20 client.4827 trim_cache size 0 max 16384 2024-03-30T01:41:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.822+0000 7f0cf2a23b80 1 Processor -- start 2024-03-30T01:41:31.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.822+0000 7f0cf2a23b80 1 -- start start 2024-03-30T01:41:31.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 0x562f737660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f73789b90 0x562f737a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f73767710 con 0x562f7376cff0 2024-03-30T01:41:31.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f73767500 con 0x562f73789b90 2024-03-30T01:41:31.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f7379fef0 con 0x562f7376d3d0 2024-03-30T01:41:31.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cdffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cdffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737ab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51838/0 (socket says 172.21.15.31:51838) 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cdffff700 1 -- 172.21.15.31:0/2891923391 learned_addr learned my addr 172.21.15.31:0/2891923391 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 0x562f737660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f73789b90 0x562f737a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cdffff700 1 -- 172.21.15.31:0/2891923391 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 msgr2=0x562f737660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 0x562f737660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cdffff700 1 -- 172.21.15.31:0/2891923391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f73789b90 msgr2=0x562f737a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f73789b90 0x562f737a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0cdffff700 1 -- 172.21.15.31:0/2891923391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f73543c80 con 0x562f7376cff0 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.823+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f73789b90 0x562f737a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737ab1d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0010010 tx=0x7f0cd002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d478b9b631f17cb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cc67fc700 1 -- 172.21.15.31:0/2891923391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0005c20 con 0x562f7376cff0 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cc67fc700 1 -- 172.21.15.31:0/2891923391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd002f070 con 0x562f7376cff0 2024-03-30T01:41:31.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cc67fc700 1 -- 172.21.15.31:0/2891923391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0034ca0 con 0x562f7376cff0 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2891923391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 msgr2=0x562f737ab1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737ab1d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0010010 tx=0x7f0cd002d070 comp rx=0 tx=0).stop 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2891923391 shutdown_connections 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f7376cff0 0x562f737ab1d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f73789b90 0x562f737a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2891923391 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376d3d0 0x562f737660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2891923391 >> 172.21.15.31:0/2891923391 conn(0x562f736cb650 msgr2=0x562f737ac550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2891923391 shutdown_connections 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.824+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2891923391 wait complete. 2024-03-30T01:41:31.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.825+0000 7f0cf2a23b80 1 -- start start 2024-03-30T01:41:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.825+0000 7f0cf2a23b80 1 Processor -- start 2024-03-30T01:41:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376cff0 0x562f7378e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 0x562f737a7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0cf2a23b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f73789b90 0x562f737b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f737b6380 con 0x562f73789b90 2024-03-30T01:41:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7376e7c0 con 0x562f7376d3d0 2024-03-30T01:41:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0cf2a23b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f737a7780 con 0x562f7376cff0 2024-03-30T01:41:31.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0cf2a23b80 10 client.0 _ll_register_callbacks cb 0x7f0cd4007720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0ce4e2a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f73789b90 0x562f737b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0ce4e2a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f73789b90 0x562f737b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51844/0 (socket says 172.21.15.31:51844) 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0ce4e2a700 1 -- 172.21.15.31:0/2284610856 learned_addr learned my addr 172.21.15.31:0/2284610856 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376cff0 0x562f7378e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.826+0000 7f0cdf7fe700 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 0x562f737a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0ce4e2a700 1 -- 172.21.15.31:0/2284610856 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376cff0 msgr2=0x562f7378e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376cff0 0x562f7378e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0ce4e2a700 1 -- 172.21.15.31:0/2284610856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 msgr2=0x562f737a7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 0x562f737a7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0ce4e2a700 1 -- 172.21.15.31:0/2284610856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f73683e50 con 0x562f73789b90 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0ce4e2a700 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f73789b90 0x562f737b5a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00c2540 tx=0x7f0cd8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ac779f731239e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cbeffd700 1 -- 172.21.15.31:0/2284610856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8004450 con 0x562f73789b90 2024-03-30T01:41:31.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cbeffd700 1 -- 172.21.15.31:0/2284610856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0cd80045b0 con 0x562f73789b90 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2284610856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cd0019070 con 0x562f73789b90 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cbeffd700 1 -- 172.21.15.31:0/2284610856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd8003e60 con 0x562f73789b90 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cf2a23b80 10 client.4833 fetch_fsmap learned FSMap version 9 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2284610856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x562f734e9fe0 con 0x562f73789b90 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cbeffd700 1 -- 172.21.15.31:0/2284610856 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0019070 con 0x562f73789b90 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cbeffd700 1 -- 172.21.15.31:0/2284610856 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f0cd800b050 con 0x562f73789b90 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cf2a23b80 10 client.4833 fetch_fsmap finished waiting for FSMap version 9 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cf2a23b80 10 client.4833 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cf2a23b80 20 client.4833 populate_metadata read hostname 'smithi031' 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x562f7380d930 0x562f737a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cf2a23b80 4 client.4833 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.827+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2284610856 --> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562f7350cca0 con 0x562f7380d930 2024-03-30T01:41:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x562f7380d930 0x562f737a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cdffff700 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x562f7380d930 0x562f737a9fd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ace63ee7ee6b server_cookie=da050a049998e492 in_seq=0 out_seq=0 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cbeffd700 10 client.4833 ms_handle_connect on v2:172.21.15.31:6835/3724657244 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cbeffd700 1 -- 172.21.15.31:0/2284610856 <== mds.0 v2:172.21.15.31:6835/3724657244 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x562f7350cca0 con 0x562f7380d930 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cbeffd700 10 client.4833 handle_command_reply: tid=0 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 client.4833 shutdown 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2284610856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f73789b90 msgr2=0x562f737b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f73789b90 0x562f737b5a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00c2540 tx=0x7f0cd8004b40 comp rx=0 tx=0).stop 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2284610856 shutdown_connections 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.31:6835/3724657244,v1:172.21.15.31:6837/3724657244] conn(0x562f7380d930 0x562f737a9fd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f73789b90 0x562f737b5a80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x562f7376cff0 0x562f7378e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cbeffd700 0 client.4833 ms_handle_reset on v2:172.21.15.31:6835/3724657244 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 --2- 172.21.15.31:0/2284610856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f7376d3d0 0x562f737a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2284610856 >> 172.21.15.31:0/2284610856 conn(0x562f736cb650 msgr2=0x562f737662d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2284610856 shutdown_connections 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.828+0000 7f0cf2a23b80 1 -- 172.21.15.31:0/2284610856 wait complete. 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0cf2a23b80 20 client.4833 trim_cache size 0 max 16384 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9739890c-57ec-4503-9fd1-d4b0145023f4", 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4269, 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-03-30T01:41:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 33.425275227999997, 2024-03-30T01:41:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 9, 2024-03-30T01:41:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 24, 2024-03-30T01:41:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 24, 2024-03-30T01:41:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 38.169585644999998 2024-03-30T01:41:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-03-30T01:41:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 -- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0ccc06ac90 msgr2=0x7f0ccc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0ccc06ac90 0x7f0ccc06d150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0ce01e4660 tx=0x7f0cd0008040 comp rx=0 tx=0).stop 2024-03-30T01:41:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 -- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce0151d20 msgr2=0x7f0ce00c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce0151d20 0x7f0ce00c7170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40011e0 tx=0x7f0cd400cae0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 -- 172.21.15.31:0/2604467 shutdown_connections 2024-03-30T01:41:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0ccc06ac90 0x7f0ccc06d150 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ce0152100 0x7f0ce01ed660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce00c5dc0 0x7f0ce014cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 --2- 172.21.15.31:0/2604467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ce0151d20 0x7f0ce00c7170 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:31.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 -- 172.21.15.31:0/2604467 >> 172.21.15.31:0/2604467 conn(0x7f0ce006ad30 msgr2=0x7f0ce00c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:31.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 -- 172.21.15.31:0/2604467 shutdown_connections 2024-03-30T01:41:31.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:31.829+0000 7f0ce688d700 1 -- 172.21.15.31:0/2604467 wait complete. 2024-03-30T01:41:31.848 INFO:teuthology.run_tasks:Running task kclient... 2024-03-30T01:41:31.862 INFO:tasks.kclient:Mounting kernel clients... 2024-03-30T01:41:31.862 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-30T01:41:31.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:31.863 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-03-30T01:41:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-30T01:41:31.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:31.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T01:41:31.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:31.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-03-30T01:41:32.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:32.056 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-03-30T01:41:32.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:32.078 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-03-30T01:41:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2024-03-30T01:41:32.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T01:41:32.159 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-30T01:41:32.159 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T01:41:32.159 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T01:41:32.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:41:32.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.302+0000 7f0785d34700 1 Processor -- start 2024-03-30T01:41:32.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.302+0000 7f0785d34700 1 -- start start 2024-03-30T01:41:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f0785d34700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800ad010 0x7f07800c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f0785d34700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07800a74e0 0x7f07800c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f0785d34700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0780153590 con 0x7f07800ac520 2024-03-30T01:41:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f0785d34700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07800ab680 con 0x7f07800a74e0 2024-03-30T01:41:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f0785d34700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07801500c0 con 0x7f07800ad010 2024-03-30T01:41:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f077ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07800a74e0 0x7f07800c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f077ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07800a74e0 0x7f07800c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51854/0 (socket says 172.21.15.31:51854) 2024-03-30T01:41:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f077ffff700 1 -- 172.21.15.31:0/2710113909 learned_addr learned my addr 172.21.15.31:0/2710113909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f077effd700 1 --2- 172.21.15.31:0/2710113909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800ad010 0x7f07800c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f077ffff700 1 -- 172.21.15.31:0/2710113909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800ad010 msgr2=0x7f07800c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f077ffff700 1 --2- 172.21.15.31:0/2710113909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800ad010 0x7f07800c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f077ffff700 1 -- 172.21.15.31:0/2710113909 >> v1:172.21.15.59:6789/0 conn(0x7f07800ac520 legacy=0x7f07800ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f077ffff700 1 -- 172.21.15.31:0/2710113909 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07800c7990 con 0x7f07800a74e0 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.303+0000 7f077ffff700 1 --2- 172.21.15.31:0/2710113909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800a74e0 0x7f07800c6540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0778009e80 tx=0x7f077800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c656234268f494a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f077e7fc700 1 -- 172.21.15.31:0/2710113909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077800e070 con 0x7f07800a74e0 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f077e7fc700 1 -- 172.21.15.31:0/2710113909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0778003a20 con 0x7f07800a74e0 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f077e7fc700 1 -- 172.21.15.31:0/2710113909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778003dd0 con 0x7f07800a74e0 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f0785d34700 1 -- 172.21.15.31:0/2710113909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800a74e0 msgr2=0x7f07800c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f0785d34700 1 --2- 172.21.15.31:0/2710113909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800a74e0 0x7f07800c6540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0778009e80 tx=0x7f077800b330 comp rx=0 tx=0).stop 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f0785d34700 1 -- 172.21.15.31:0/2710113909 shutdown_connections 2024-03-30T01:41:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f0785d34700 1 --2- 172.21.15.31:0/2710113909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800a74e0 0x7f07800c6540 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f0785d34700 1 --2- 172.21.15.31:0/2710113909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800ad010 0x7f07800c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f0785d34700 1 -- 172.21.15.31:0/2710113909 >> 172.21.15.31:0/2710113909 conn(0x7f078006ad30 msgr2=0x7f07800a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f0785d34700 1 -- 172.21.15.31:0/2710113909 shutdown_connections 2024-03-30T01:41:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.304+0000 7f0785d34700 1 -- 172.21.15.31:0/2710113909 wait complete. 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.306+0000 7f0785d34700 1 Processor -- start 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.306+0000 7f0785d34700 1 -- start start 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f0785d34700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800a74e0 0x7f07801eb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f0785d34700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07800ac520 0x7f07801dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f0785d34700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800ad010 0x7f07801dc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f0785d34700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07801563f0 con 0x7f07800ad010 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f0785d34700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0780156920 con 0x7f07800ac520 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f0785d34700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07801dd880 con 0x7f07800a74e0 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800ad010 0x7f07801dc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800ad010 0x7f07801dc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51862/0 (socket says 172.21.15.31:51862) 2024-03-30T01:41:32.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077ffff700 1 -- 172.21.15.31:0/2375006772 learned_addr learned my addr 172.21.15.31:0/2375006772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077effd700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07800ac520 0x7f07801dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077f7fe700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800a74e0 0x7f07801eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077ffff700 1 -- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800a74e0 msgr2=0x7f07801eb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077ffff700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800a74e0 0x7f07801eb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077ffff700 1 -- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07800ac520 msgr2=0x7f07801dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077ffff700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07800ac520 0x7f07801dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077ffff700 1 -- 172.21.15.31:0/2375006772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778009b30 con 0x7f07800ad010 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077f7fe700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800a74e0 0x7f07801eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077effd700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07800ac520 0x7f07801dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f077ffff700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800ad010 0x7f07801dc4b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0778015040 tx=0x7f0778004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f0767fff700 1 -- 172.21.15.31:0/2375006772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07780044c0 con 0x7f07800ad010 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f0767fff700 1 -- 172.21.15.31:0/2375006772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0778004660 con 0x7f07800ad010 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.307+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07801ebf60 con 0x7f07800ad010 2024-03-30T01:41:32.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.308+0000 7f0767fff700 1 -- 172.21.15.31:0/2375006772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778012430 con 0x7f07800ad010 2024-03-30T01:41:32.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.308+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07801557e0 con 0x7f07800ad010 2024-03-30T01:41:32.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.308+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07800c7320 con 0x7f07800ad010 2024-03-30T01:41:32.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.309+0000 7f0767fff700 1 -- 172.21.15.31:0/2375006772 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0778012590 con 0x7f07800ad010 2024-03-30T01:41:32.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.309+0000 7f0767fff700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f076c06ac90 0x7f076c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:32.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.309+0000 7f0767fff700 1 -- 172.21.15.31:0/2375006772 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f077808aa90 con 0x7f07800ad010 2024-03-30T01:41:32.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.310+0000 7f077f7fe700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f076c06ac90 0x7f076c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:32.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.310+0000 7f077f7fe700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f076c06ac90 0x7f076c06d150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f07800c46e0 tx=0x7f0770009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:32.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.314+0000 7f0767fff700 1 -- 172.21.15.31:0/2375006772 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f07800c7320 con 0x7f07800ad010 2024-03-30T01:41:32.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.578+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f07800c7320 con 0x7f07800ad010 2024-03-30T01:41:32.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.579+0000 7f0767fff700 1 -- 172.21.15.31:0/2375006772 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f07800c7320 con 0x7f07800ad010 2024-03-30T01:41:32.584 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T01:41:32.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f076c06ac90 msgr2=0x7f076c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:32.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f076c06ac90 0x7f076c06d150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f07800c46e0 tx=0x7f0770009040 comp rx=0 tx=0).stop 2024-03-30T01:41:32.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800ad010 msgr2=0x7f07801dc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:32.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800ad010 0x7f07801dc4b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0778015040 tx=0x7f0778004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 shutdown_connections 2024-03-30T01:41:32.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07800ad010 0x7f07801dc4b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f076c06ac90 0x7f076c06d150 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07800a74e0 0x7f07801eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 --2- 172.21.15.31:0/2375006772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07800ac520 0x7f07801dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:32.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 >> 172.21.15.31:0/2375006772 conn(0x7f078006ad30 msgr2=0x7f07800c31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:32.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 shutdown_connections 2024-03-30T01:41:32.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:32.581+0000 7f0785d34700 1 -- 172.21.15.31:0/2375006772 wait complete. 2024-03-30T01:41:32.600 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T01:41:32.600 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:41:32.600 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-03-30T01:41:32.600 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T01:41:32.600 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T01:41:32.601 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T01:41:32.601 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T01:41:32.601 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T01:41:32.601 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:41:32.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:32.601 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-03-30T01:41:32.620 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:41:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:41:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:41:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:41:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-03-30T01:41:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:41:32.620 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:41:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42232sec preferred_lft 42232sec 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-03-30T01:41:32.622 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T01:41:32.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:32.622 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:32.622 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:32.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-03-30T01:41:32.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-03-30T01:41:32.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-03-30T01:41:32.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T01:41:32.622 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:32.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:32.756 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:41:32.786 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-03-30T01:41:32.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:32.788 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-03-30T01:41:32.851 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-03-30T01:41:32.851 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:41:32.851 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:41:32.851 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:41:32.851 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:41:32.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:32.852 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:32.852 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:32.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:41:32.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:41:32.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:41:32.852 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:32.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:32.977 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-03-30T01:41:32.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:32.998 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-03-30T01:41:33.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:33.060 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:33.060 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:33.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T01:41:33.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-30T01:41:33.060 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:33.170 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-30T01:41:33.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:33.170 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:33.170 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:33.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-30T01:41:33.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T01:41:33.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-30T01:41:33.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-30T01:41:33.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-30T01:41:33.171 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:33.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:33.379 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:41:33.379 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:41:33.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-03-30T01:41:33.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T01:41:33.380 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:41:33.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:41:33.491 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T01:41:33.510 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:41:33.510 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:41:33.510 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T01:41:33.569 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-30T01:41:33.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:33.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:41:33.667 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:41:33.667 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-30T01:41:33.667 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:41:33.754 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:41:33.754 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-30T01:41:33.754 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:41:33.754 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:41:33.754 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:41:33.774 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:41:33.774 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-30T01:41:33.774 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:41:33.774 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:41:33.774 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:41:33.775 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790" 2024-03-30T01:41:33.775 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790:/ 2024-03-30T01:41:33.775 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9739890c-57ec-4503-9fd1-d4b0145023f4" will pass to kernel 2024-03-30T01:41:33.777 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T01:41:33.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:33.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T01:41:33.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:33.808 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:41:33.888 INFO:teuthology.orchestra.run.smithi031.stdout:client4622 2024-03-30T01:41:33.888 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:41:33.888 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:33.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:33.951 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:41:34.034 INFO:teuthology.orchestra.run.smithi031.stdout:client4622 2024-03-30T01:41:34.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:34.034 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:41:34.116 INFO:teuthology.orchestra.run.smithi031.stdout:client4622 2024-03-30T01:41:34.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:34.116 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9739890c-57ec-4503-9fd1-d4b0145023f4.client4622/status') 2024-03-30T01:41:34.210 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.4622 (3)172.21.15.31:0/3045696839 2024-03-30T01:41:34.210 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2024-03-30T01:41:34.210 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2024-03-30T01:41:34.210 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2024-03-30T01:41:34.210 INFO:teuthology.orchestra.run.smithi031.stderr:70 bytes copied, 2.9716e-05 s, 2.4 MB/s 2024-03-30T01:41:34.212 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-30T01:41:34.212 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T01:41:34.212 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T01:41:34.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:41:34.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.347+0000 7fc6cbd6c700 1 Processor -- start 2024-03-30T01:41:34.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.347+0000 7fc6cbd6c700 1 -- start start 2024-03-30T01:41:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6cbd6c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6c40c1300 0x7fc6c406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6cbd6c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c40c2f60 con 0x7fc6c414fc30 2024-03-30T01:41:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6cbd6c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c40c2a40 con 0x7fc6c4152650 2024-03-30T01:41:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6cbd6c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c406eca0 con 0x7fc6c40c1300 2024-03-30T01:41:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6c9307700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6c40c1300 0x7fc6c406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:34.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6c9307700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6c40c1300 0x7fc6c406ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51878/0 (socket says 172.21.15.31:51878) 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6c9307700 1 -- 172.21.15.31:0/1224893333 learned_addr learned my addr 172.21.15.31:0/1224893333 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6c9b08700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc6c414fc30 0x7fc6c406a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35380/0 (socket says 172.21.15.31:35380) 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6c9307700 1 -- 172.21.15.31:0/1224893333 >> v1:172.21.15.59:6789/0 conn(0x7fc6c4152650 legacy=0x7fc6c4152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6c9307700 1 -- 172.21.15.31:0/1224893333 >> v1:172.21.15.31:6789/0 conn(0x7fc6c414fc30 legacy=0x7fc6c406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6c9307700 1 -- 172.21.15.31:0/1224893333 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c406b350 con 0x7fc6c40c1300 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.348+0000 7fc6c8b06700 1 -- 172.21.15.31:0/1224893333 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180322527 0 0) 0x7fc6c40c2f60 con 0x7fc6c414fc30 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.349+0000 7fc6c9307700 1 --2- 172.21.15.31:0/1224893333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c40c1300 0x7fc6c406ae10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40098f0 tx=0x7fc6b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175c11105d58b9e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.349+0000 7fc6c8b06700 1 -- 172.21.15.31:0/1224893333 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4004230 con 0x7fc6c40c1300 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.349+0000 7fc6c8b06700 1 -- 172.21.15.31:0/1224893333 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b4004390 con 0x7fc6c40c1300 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.349+0000 7fc6c8b06700 1 -- 172.21.15.31:0/1224893333 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40036a0 con 0x7fc6c40c1300 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.349+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/1224893333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c40c1300 msgr2=0x7fc6c406ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.349+0000 7fc6cbd6c700 1 --2- 172.21.15.31:0/1224893333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c40c1300 0x7fc6c406ae10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40098f0 tx=0x7fc6b4009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:34.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.349+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/1224893333 shutdown_connections 2024-03-30T01:41:34.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.349+0000 7fc6cbd6c700 1 --2- 172.21.15.31:0/1224893333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c40c1300 0x7fc6c406ae10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:34.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.349+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/1224893333 >> 172.21.15.31:0/1224893333 conn(0x7fc6c40c3a60 msgr2=0x7fc6c40c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:34.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.350+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/1224893333 shutdown_connections 2024-03-30T01:41:34.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.350+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/1224893333 wait complete. 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.351+0000 7fc6cbd6c700 1 Processor -- start 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.351+0000 7fc6cbd6c700 1 -- start start 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.351+0000 7fc6cbd6c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc6c40c1300 0x7fc6c40c0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6cbd6c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c414fc30 0x7fc6c414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6cbd6c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6c4152650 0x7fc6c414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6cbd6c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c414dcf0 con 0x7fc6c414fc30 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6cbd6c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c414a970 con 0x7fc6c4152650 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6cbd6c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c414ac30 con 0x7fc6c40c1300 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9307700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c414fc30 0x7fc6c414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9307700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c414fc30 0x7fc6c414e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51880/0 (socket says 172.21.15.31:51880) 2024-03-30T01:41:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9307700 1 -- 172.21.15.31:0/3727097945 learned_addr learned my addr 172.21.15.31:0/3727097945 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9b08700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc6c40c1300 0x7fc6c40c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6ca309700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6c4152650 0x7fc6c414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9307700 1 -- 172.21.15.31:0/3727097945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc6c40c1300 msgr2=0x7fc6c40c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9307700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc6c40c1300 0x7fc6c40c0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9307700 1 -- 172.21.15.31:0/3727097945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6c4152650 msgr2=0x7fc6c414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9307700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6c4152650 0x7fc6c414e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9307700 1 -- 172.21.15.31:0/3727097945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b4009550 con 0x7fc6c414fc30 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6c9307700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c414fc30 0x7fc6c414e480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009130 tx=0x7fc6b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6ba7fc700 1 -- 172.21.15.31:0/3727097945 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400b660 con 0x7fc6c414fc30 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6ba7fc700 1 -- 172.21.15.31:0/3727097945 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b400b7e0 con 0x7fc6c414fc30 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c414af10 con 0x7fc6c414fc30 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.352+0000 7fc6ba7fc700 1 -- 172.21.15.31:0/3727097945 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400bb90 con 0x7fc6c414fc30 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.353+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c40aa3a0 con 0x7fc6c414fc30 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.354+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c404ea80 con 0x7fc6c414fc30 2024-03-30T01:41:34.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.354+0000 7fc6ba7fc700 1 -- 172.21.15.31:0/3727097945 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc6b4021070 con 0x7fc6c414fc30 2024-03-30T01:41:34.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.354+0000 7fc6ba7fc700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc6a806ac90 0x7fc6a806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:34.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.354+0000 7fc6ba7fc700 1 -- 172.21.15.31:0/3727097945 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc6b408ad90 con 0x7fc6c414fc30 2024-03-30T01:41:34.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.354+0000 7fc6c9b08700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc6a806ac90 0x7fc6a806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:34.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.355+0000 7fc6c9b08700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc6a806ac90 0x7fc6a806d150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0007fd0 tx=0x7fc6c0007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:34.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.359+0000 7fc6ba7fc700 1 -- 172.21.15.31:0/3727097945 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc6b4055dc0 con 0x7fc6c414fc30 2024-03-30T01:41:34.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.621+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc6c40c2a40 con 0x7fc6c414fc30 2024-03-30T01:41:34.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.622+0000 7fc6ba7fc700 1 -- 172.21.15.31:0/3727097945 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc6c40c2a40 con 0x7fc6c414fc30 2024-03-30T01:41:34.627 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc6a806ac90 msgr2=0x7fc6a806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc6a806ac90 0x7fc6a806d150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0007fd0 tx=0x7fc6c0007c20 comp rx=0 tx=0).stop 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c414fc30 msgr2=0x7fc6c414e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c414fc30 0x7fc6c414e480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009130 tx=0x7fc6b4004750 comp rx=0 tx=0).stop 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 shutdown_connections 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc6a806ac90 0x7fc6a806d150 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6c4152650 0x7fc6c414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc6c40c1300 0x7fc6c40c0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 --2- 172.21.15.31:0/3727097945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6c414fc30 0x7fc6c414e480 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 >> 172.21.15.31:0/3727097945 conn(0x7fc6c40c3a60 msgr2=0x7fc6c406d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:34.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 shutdown_connections 2024-03-30T01:41:34.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:34.624+0000 7fc6cbd6c700 1 -- 172.21.15.31:0/3727097945 wait complete. 2024-03-30T01:41:34.643 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T01:41:34.643 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:41:34.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-03-30T01:41:34.643 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T01:41:34.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T01:41:34.643 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T01:41:34.643 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T01:41:34.643 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T01:41:34.643 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:41:34.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:34.644 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-03-30T01:41:34.667 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:41:34.667 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:41:34.667 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:41:34.667 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:41:34.667 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-03-30T01:41:34.667 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:41:34.667 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42247sec preferred_lft 42247sec 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-03-30T01:41:34.668 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-03-30T01:41:34.669 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T01:41:34.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:34.669 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2024-03-30T01:41:34.669 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-03-30T01:41:34.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2024-03-30T01:41:34.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2024-03-30T01:41:34.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2024-03-30T01:41:34.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T01:41:34.669 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-03-30T01:41:34.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:34.860 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:41:34.893 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-03-30T01:41:34.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:34.895 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-03-30T01:41:34.977 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-03-30T01:41:34.978 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:41:34.978 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:41:34.978 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:41:34.978 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:41:34.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:34.978 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2024-03-30T01:41:34.978 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-03-30T01:41:34.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:41:34.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:41:34.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:41:34.979 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-03-30T01:41:35.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:35.230 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-03-30T01:41:35.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:35.251 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-03-30T01:41:35.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:35.313 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2024-03-30T01:41:35.313 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-03-30T01:41:35.313 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T01:41:35.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-30T01:41:35.314 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-03-30T01:41:35.423 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-30T01:41:35.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:35.423 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2024-03-30T01:41:35.423 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-03-30T01:41:35.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-30T01:41:35.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T01:41:35.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-30T01:41:35.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-30T01:41:35.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-30T01:41:35.424 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-03-30T01:41:35.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:35.687 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2024-03-30T01:41:35.687 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-03-30T01:41:35.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-03-30T01:41:35.688 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T01:41:35.688 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-03-30T01:41:35.800 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:41:35.800 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T01:41:35.820 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:41:35.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:41:35.820 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:41:35.879 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-30T01:41:35.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:35.880 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:41:35.978 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:41:35.979 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-30T01:41:35.979 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:41:36.059 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:41:36.059 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-30T01:41:36.059 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:41:36.059 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:41:36.059 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:41:36.078 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:41:36.079 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-30T01:41:36.079 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:41:36.079 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:41:36.079 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:41:36.079 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790" 2024-03-30T01:41:36.079 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790:/ 2024-03-30T01:41:36.079 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9739890c-57ec-4503-9fd1-d4b0145023f4" will pass to kernel 2024-03-30T01:41:36.081 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T01:41:36.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:36.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:41:36.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:36.113 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:41:36.192 INFO:teuthology.orchestra.run.smithi059.stdout:client4822 2024-03-30T01:41:36.192 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:41:36.192 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:36.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:36.255 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:41:36.338 INFO:teuthology.orchestra.run.smithi059.stdout:client4822 2024-03-30T01:41:36.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:36.338 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:41:36.421 INFO:teuthology.orchestra.run.smithi059.stdout:client4822 2024-03-30T01:41:36.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:36.422 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9739890c-57ec-4503-9fd1-d4b0145023f4.client4822/status') 2024-03-30T01:41:36.514 INFO:teuthology.orchestra.run.smithi059.stdout:instance: client.4822 (3)192.168.0.1:0/2791881716 2024-03-30T01:41:36.514 INFO:teuthology.orchestra.run.smithi059.stdout:blocklisted: false 2024-03-30T01:41:36.515 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records in 2024-03-30T01:41:36.515 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records out 2024-03-30T01:41:36.515 INFO:teuthology.orchestra.run.smithi059.stderr:69 bytes copied, 2.923e-05 s, 2.4 MB/s 2024-03-30T01:41:36.516 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-30T01:41:36.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.670+0000 7ff11ce9b700 1 Processor -- start 2024-03-30T01:41:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.670+0000 7ff11ce9b700 1 -- start start 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.671+0000 7ff11ce9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff1180c3180 0x7ff11806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.671+0000 7ff11ce9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11806a700 0x7ff11806f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.671+0000 7ff11ce9b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1181547d0 con 0x7ff1180c4710 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.671+0000 7ff11ce9b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff118154a90 con 0x7ff1180c3180 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.671+0000 7ff11ce9b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff118154d70 con 0x7ff11806a700 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff1180c3180 0x7ff11806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff116d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff1180c4710 0x7ff1180c0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53028/0 (socket says 172.21.15.31:53028) 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff116d9d700 1 -- 172.21.15.31:0/3784938749 learned_addr learned my addr 172.21.15.31:0/3784938749 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff115d9b700 1 --2- 172.21.15.31:0/3784938749 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11806a700 0x7ff11806f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11659c700 1 -- 172.21.15.31:0/3784938749 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11806a700 msgr2=0x7ff11806f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11659c700 1 --2- 172.21.15.31:0/3784938749 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11806a700 0x7ff11806f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11659c700 1 -- 172.21.15.31:0/3784938749 >> v1:172.21.15.31:6789/0 conn(0x7ff1180c4710 legacy=0x7ff1180c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11659c700 1 -- 172.21.15.31:0/3784938749 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118155050 con 0x7ff1180c3180 2024-03-30T01:41:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11559a700 1 -- 172.21.15.31:0/3784938749 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114864157 0 0) 0x7ff1181547d0 con 0x7ff1180c4710 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff115d9b700 1 --2- 172.21.15.31:0/3784938749 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11806a700 0x7ff11806f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11659c700 1 --2- 172.21.15.31:0/3784938749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1180c3180 0x7ff11806eb00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff108009840 tx=0x7ff108009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4060644d6f31ade3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11559a700 1 -- 172.21.15.31:0/3784938749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108011830 con 0x7ff1180c3180 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11559a700 1 -- 172.21.15.31:0/3784938749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff108011990 con 0x7ff1180c3180 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.672+0000 7ff11559a700 1 -- 172.21.15.31:0/3784938749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108011d20 con 0x7ff1180c3180 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.673+0000 7ff11ce9b700 1 -- 172.21.15.31:0/3784938749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1180c3180 msgr2=0x7ff11806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.673+0000 7ff11ce9b700 1 --2- 172.21.15.31:0/3784938749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1180c3180 0x7ff11806eb00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff108009840 tx=0x7ff108009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.673+0000 7ff11ce9b700 1 -- 172.21.15.31:0/3784938749 shutdown_connections 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.673+0000 7ff11ce9b700 1 --2- 172.21.15.31:0/3784938749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1180c3180 0x7ff11806eb00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.673+0000 7ff11ce9b700 1 --2- 172.21.15.31:0/3784938749 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11806a700 0x7ff11806f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.673+0000 7ff11ce9b700 1 -- 172.21.15.31:0/3784938749 >> 172.21.15.31:0/3784938749 conn(0x7ff1180c3a60 msgr2=0x7ff1180c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.673+0000 7ff11ce9b700 1 -- 172.21.15.31:0/3784938749 shutdown_connections 2024-03-30T01:41:36.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.673+0000 7ff11ce9b700 1 -- 172.21.15.31:0/3784938749 wait complete. 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.674+0000 7ff11ce9b700 1 Processor -- start 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11ce9b700 1 -- start start 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11ce9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11806a700 0x7ff118157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11ce9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1180c3180 0x7ff118152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11ce9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1180c4710 0x7ff118158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11ce9b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff118156a00 con 0x7ff11806a700 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11ce9b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1181572a0 con 0x7ff1180c3180 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11ce9b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff118157c70 con 0x7ff1180c4710 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11806a700 0x7ff118157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11806a700 0x7ff118157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46350/0 (socket says 172.21.15.31:46350) 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff11659c700 1 -- 172.21.15.31:0/1331999165 learned_addr learned my addr 172.21.15.31:0/1331999165 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.675+0000 7ff116d9d700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1180c4710 0x7ff118158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff115d9b700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1180c3180 0x7ff118152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff11659c700 1 -- 172.21.15.31:0/1331999165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1180c4710 msgr2=0x7ff118158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff11659c700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1180c4710 0x7ff118158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff11659c700 1 -- 172.21.15.31:0/1331999165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1180c3180 msgr2=0x7ff118152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff11659c700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1180c3180 0x7ff118152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff11659c700 1 -- 172.21.15.31:0/1331999165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1080094a0 con 0x7ff11806a700 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff11659c700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11806a700 0x7ff118157730 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff108000c00 tx=0x7ff108004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff10effd700 1 -- 172.21.15.31:0/1331999165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1080036a0 con 0x7ff11806a700 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff10effd700 1 -- 172.21.15.31:0/1331999165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10801f070 con 0x7ff11806a700 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1181ede60 con 0x7ff11806a700 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff10effd700 1 -- 172.21.15.31:0/1331999165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108003a40 con 0x7ff11806a700 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.676+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff118156f30 con 0x7ff11806a700 2024-03-30T01:41:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.677+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff118157580 con 0x7ff11806a700 2024-03-30T01:41:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.678+0000 7ff10effd700 1 -- 172.21.15.31:0/1331999165 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff108003ba0 con 0x7ff11806a700 2024-03-30T01:41:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.678+0000 7ff10effd700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff0f806ac90 0x7ff0f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.678+0000 7ff10effd700 1 -- 172.21.15.31:0/1331999165 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff108021070 con 0x7ff11806a700 2024-03-30T01:41:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.678+0000 7ff115d9b700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff0f806ac90 0x7ff0f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.679+0000 7ff115d9b700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff0f806ac90 0x7ff0f806d150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff1181e7940 tx=0x7ff100008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.683+0000 7ff10effd700 1 -- 172.21.15.31:0/1331999165 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff108055ee0 con 0x7ff11806a700 2024-03-30T01:41:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.900+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1181570c0 con 0x7ff11806a700 2024-03-30T01:41:36.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.901+0000 7ff10effd700 1 -- 172.21.15.31:0/1331999165 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7ff108015070 con 0x7ff11806a700 2024-03-30T01:41:36.907 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:36.907 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff0f806ac90 msgr2=0x7ff0f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff0f806ac90 0x7ff0f806d150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff1181e7940 tx=0x7ff100008040 comp rx=0 tx=0).stop 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11806a700 msgr2=0x7ff118157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11806a700 0x7ff118157730 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff108000c00 tx=0x7ff108004750 comp rx=0 tx=0).stop 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 shutdown_connections 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff11806a700 0x7ff118157730 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff0f806ac90 0x7ff0f806d150 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1180c4710 0x7ff118158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 --2- 172.21.15.31:0/1331999165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1180c3180 0x7ff118152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 >> 172.21.15.31:0/1331999165 conn(0x7ff1180c3a60 msgr2=0x7ff1180c4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 shutdown_connections 2024-03-30T01:41:36.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:36.904+0000 7ff11ce9b700 1 -- 172.21.15.31:0/1331999165 wait complete. 2024-03-30T01:41:36.924 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:36.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.059+0000 7efe261d6700 1 Processor -- start 2024-03-30T01:41:37.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.059+0000 7efe261d6700 1 -- start start 2024-03-30T01:41:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe261d6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efe200c1560 0x7efe2006c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe261d6700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe200721b0 0x7efe200c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe261d6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe2006cad0 con 0x7efe200c2430 2024-03-30T01:41:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe261d6700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe200bafa0 con 0x7efe200721b0 2024-03-30T01:41:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe261d6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe2006d2e0 con 0x7efe200c1560 2024-03-30T01:41:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe1effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe200721b0 0x7efe200c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe1effd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe200721b0 0x7efe200c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46358/0 (socket says 172.21.15.31:46358) 2024-03-30T01:41:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe1effd700 1 -- 172.21.15.31:0/942385186 learned_addr learned my addr 172.21.15.31:0/942385186 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe1f7fe700 1 --2- 172.21.15.31:0/942385186 >> v2:172.21.15.59:3300/0 conn(0x7efe200c1560 0x7efe2006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe1effd700 1 -- 172.21.15.31:0/942385186 >> v2:172.21.15.59:3300/0 conn(0x7efe200c1560 msgr2=0x7efe2006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.060+0000 7efe1effd700 1 --2- 172.21.15.31:0/942385186 >> v2:172.21.15.59:3300/0 conn(0x7efe200c1560 0x7efe2006c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.061+0000 7efe1effd700 1 -- 172.21.15.31:0/942385186 >> v1:172.21.15.59:6789/0 conn(0x7efe200c2430 legacy=0x7efe200bb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.061+0000 7efe1effd700 1 -- 172.21.15.31:0/942385186 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe200c41b0 con 0x7efe200721b0 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.061+0000 7efe1effd700 1 --2- 172.21.15.31:0/942385186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200721b0 0x7efe200c1940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efe200c35f0 tx=0x7efe08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08841f2c1252b83 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.061+0000 7efe1e7fc700 1 -- 172.21.15.31:0/942385186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08004230 con 0x7efe200721b0 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.061+0000 7efe1e7fc700 1 -- 172.21.15.31:0/942385186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08004390 con 0x7efe200721b0 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.061+0000 7efe1e7fc700 1 -- 172.21.15.31:0/942385186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe080036a0 con 0x7efe200721b0 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.061+0000 7efe261d6700 1 -- 172.21.15.31:0/942385186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200721b0 msgr2=0x7efe200c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.061+0000 7efe261d6700 1 --2- 172.21.15.31:0/942385186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200721b0 0x7efe200c1940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efe200c35f0 tx=0x7efe08009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.062+0000 7efe261d6700 1 -- 172.21.15.31:0/942385186 shutdown_connections 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.062+0000 7efe261d6700 1 --2- 172.21.15.31:0/942385186 >> v2:172.21.15.59:3300/0 conn(0x7efe200c1560 0x7efe2006c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.062+0000 7efe261d6700 1 --2- 172.21.15.31:0/942385186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200721b0 0x7efe200c1940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.062+0000 7efe261d6700 1 -- 172.21.15.31:0/942385186 >> 172.21.15.31:0/942385186 conn(0x7efe2006b110 msgr2=0x7efe200a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.062+0000 7efe261d6700 1 -- 172.21.15.31:0/942385186 shutdown_connections 2024-03-30T01:41:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.062+0000 7efe261d6700 1 -- 172.21.15.31:0/942385186 wait complete. 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.063+0000 7efe261d6700 1 Processor -- start 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.063+0000 7efe261d6700 1 -- start start 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe261d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe200721b0 0x7efe20071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe261d6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200c1560 0x7efe200aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe261d6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe200c2430 0x7efe2014af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe261d6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe200a7c50 con 0x7efe200c1560 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe261d6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe20071680 con 0x7efe200c2430 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe261d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe20070ab0 con 0x7efe200721b0 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200c1560 0x7efe200aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200c1560 0x7efe200aa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46362/0 (socket says 172.21.15.31:46362) 2024-03-30T01:41:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1effd700 1 -- 172.21.15.31:0/1118585166 learned_addr learned my addr 172.21.15.31:0/1118585166 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1ffff700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe200c2430 0x7efe2014af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1f7fe700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe200721b0 0x7efe20071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1effd700 1 -- 172.21.15.31:0/1118585166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe200721b0 msgr2=0x7efe20071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1effd700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe200721b0 0x7efe20071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1effd700 1 -- 172.21.15.31:0/1118585166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe200c2430 msgr2=0x7efe2014af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1effd700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe200c2430 0x7efe2014af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1effd700 1 -- 172.21.15.31:0/1118585166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe08009550 con 0x7efe200c1560 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1f7fe700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe200721b0 0x7efe20071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.064+0000 7efe1effd700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200c1560 0x7efe200aa330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efe08009130 tx=0x7efe08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.065+0000 7efe17fff700 1 -- 172.21.15.31:0/1118585166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08011630 con 0x7efe200c1560 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.065+0000 7efe17fff700 1 -- 172.21.15.31:0/1118585166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08011790 con 0x7efe200c1560 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.065+0000 7efe17fff700 1 -- 172.21.15.31:0/1118585166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08011b40 con 0x7efe200c1560 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.065+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe20070d50 con 0x7efe200c1560 2024-03-30T01:41:37.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.065+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe20071310 con 0x7efe200c1560 2024-03-30T01:41:37.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.066+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe200717f0 con 0x7efe200c1560 2024-03-30T01:41:37.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.066+0000 7efe17fff700 1 -- 172.21.15.31:0/1118585166 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efe08011ca0 con 0x7efe200c1560 2024-03-30T01:41:37.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.067+0000 7efe17fff700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe0006ac90 0x7efe0006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.067+0000 7efe17fff700 1 -- 172.21.15.31:0/1118585166 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efe0808ab50 con 0x7efe200c1560 2024-03-30T01:41:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.067+0000 7efe1f7fe700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe0006ac90 0x7efe0006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.067+0000 7efe1f7fe700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe0006ac90 0x7efe0006d150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efe10009820 tx=0x7efe10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:37.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.071+0000 7efe17fff700 1 -- 172.21.15.31:0/1118585166 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efe08055b80 con 0x7efe200c1560 2024-03-30T01:41:37.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.284+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe200747d0 con 0x7efe200c1560 2024-03-30T01:41:37.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.285+0000 7efe17fff700 1 -- 172.21.15.31:0/1118585166 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe080591a0 con 0x7efe200c1560 2024-03-30T01:41:37.290 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe0006ac90 msgr2=0x7efe0006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe0006ac90 0x7efe0006d150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efe10009820 tx=0x7efe10008040 comp rx=0 tx=0).stop 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200c1560 msgr2=0x7efe200aa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200c1560 0x7efe200aa330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efe08009130 tx=0x7efe08004750 comp rx=0 tx=0).stop 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 shutdown_connections 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe0006ac90 0x7efe0006d150 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe200c2430 0x7efe2014af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe200721b0 0x7efe20071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 --2- 172.21.15.31:0/1118585166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe200c1560 0x7efe200aa330 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 >> 172.21.15.31:0/1118585166 conn(0x7efe2006b110 msgr2=0x7efe200c2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 shutdown_connections 2024-03-30T01:41:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.287+0000 7efe261d6700 1 -- 172.21.15.31:0/1118585166 wait complete. 2024-03-30T01:41:37.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:37.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.439+0000 7f58ae6cd700 1 Processor -- start 2024-03-30T01:41:37.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.440+0000 7f58ae6cd700 1 -- start start 2024-03-30T01:41:37.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.440+0000 7f58ae6cd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f58a81525e0 0x7f58a80c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.440+0000 7f58ae6cd700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a8156930 con 0x7f58a814fce0 2024-03-30T01:41:37.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.440+0000 7f58ae6cd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a8156370 con 0x7f58a80a91c0 2024-03-30T01:41:37.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.440+0000 7f58ae6cd700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a80aa330 con 0x7f58a81525e0 2024-03-30T01:41:37.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.440+0000 7f58a77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f58a81525e0 0x7f58a80c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.440+0000 7f58a7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f58a814fce0 0x7f58a8155a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53030/0 (socket says 172.21.15.31:53030) 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.440+0000 7f58a7fff700 1 -- 172.21.15.31:0/2448894997 learned_addr learned my addr 172.21.15.31:0/2448894997 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.441+0000 7f58a77fe700 1 -- 172.21.15.31:0/2448894997 >> v1:172.21.15.59:6789/0 conn(0x7f58a80a91c0 legacy=0x7f58a806f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.441+0000 7f58a77fe700 1 -- 172.21.15.31:0/2448894997 >> v1:172.21.15.31:6789/0 conn(0x7f58a814fce0 legacy=0x7f58a8155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.441+0000 7f58a77fe700 1 -- 172.21.15.31:0/2448894997 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a8156650 con 0x7f58a81525e0 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.441+0000 7f58a6ffd700 1 -- 172.21.15.31:0/2448894997 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224693206 0 0) 0x7f58a8156930 con 0x7f58a814fce0 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.441+0000 7f58a77fe700 1 --2- 172.21.15.31:0/2448894997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a81525e0 0x7f58a80c1f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5890009840 tx=0x7f5890009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ecd95389ace4c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.441+0000 7f58a6ffd700 1 -- 172.21.15.31:0/2448894997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890004230 con 0x7f58a81525e0 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.441+0000 7f58a6ffd700 1 -- 172.21.15.31:0/2448894997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890004390 con 0x7f58a81525e0 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.441+0000 7f58a6ffd700 1 -- 172.21.15.31:0/2448894997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58900036a0 con 0x7f58a81525e0 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.442+0000 7f58ae6cd700 1 -- 172.21.15.31:0/2448894997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a81525e0 msgr2=0x7f58a80c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.442+0000 7f58ae6cd700 1 --2- 172.21.15.31:0/2448894997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a81525e0 0x7f58a80c1f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5890009840 tx=0x7f5890009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.442+0000 7f58ae6cd700 1 -- 172.21.15.31:0/2448894997 shutdown_connections 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.442+0000 7f58ae6cd700 1 --2- 172.21.15.31:0/2448894997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a81525e0 0x7f58a80c1f80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.442+0000 7f58ae6cd700 1 -- 172.21.15.31:0/2448894997 >> 172.21.15.31:0/2448894997 conn(0x7f58a80c3910 msgr2=0x7f58a80c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.442+0000 7f58ae6cd700 1 -- 172.21.15.31:0/2448894997 shutdown_connections 2024-03-30T01:41:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.442+0000 7f58ae6cd700 1 -- 172.21.15.31:0/2448894997 wait complete. 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.443+0000 7f58ae6cd700 1 Processor -- start 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.443+0000 7f58ae6cd700 1 -- start start 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58ae6cd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58a80a91c0 0x7f58a81534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58ae6cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a814fce0 0x7f58a81e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58ae6cd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a81525e0 0x7f58a81e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58ae6cd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a8146000 con 0x7f58a814fce0 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58ae6cd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a81465c0 con 0x7f58a81525e0 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58ae6cd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a81e57e0 con 0x7f58a80a91c0 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58a77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a814fce0 0x7f58a81e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58a77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a814fce0 0x7f58a81e4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46378/0 (socket says 172.21.15.31:46378) 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58a77fe700 1 -- 172.21.15.31:0/3546602363 learned_addr learned my addr 172.21.15.31:0/3546602363 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58a7fff700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58a80a91c0 0x7f58a81534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.444+0000 7f58acc6a700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a81525e0 0x7f58a81e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58a77fe700 1 -- 172.21.15.31:0/3546602363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58a80a91c0 msgr2=0x7f58a81534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58a77fe700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58a80a91c0 0x7f58a81534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58a77fe700 1 -- 172.21.15.31:0/3546602363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a81525e0 msgr2=0x7f58a81e52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58a77fe700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a81525e0 0x7f58a81e52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58a77fe700 1 -- 172.21.15.31:0/3546602363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900094a0 con 0x7f58a814fce0 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58a77fe700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a814fce0 0x7f58a81e4d60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5890009f50 tx=0x7f58900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58a4ff9700 1 -- 172.21.15.31:0/3546602363 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890011630 con 0x7f58a814fce0 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58a4ff9700 1 -- 172.21.15.31:0/3546602363 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890011790 con 0x7f58a814fce0 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a81e86d0 con 0x7f58a814fce0 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58a4ff9700 1 -- 172.21.15.31:0/3546602363 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890011b40 con 0x7f58a814fce0 2024-03-30T01:41:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.445+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a8146250 con 0x7f58a814fce0 2024-03-30T01:41:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.446+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a8146410 con 0x7f58a814fce0 2024-03-30T01:41:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.447+0000 7f58a4ff9700 1 -- 172.21.15.31:0/3546602363 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5890011ca0 con 0x7f58a814fce0 2024-03-30T01:41:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.448+0000 7f58a4ff9700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f588806ac90 0x7f588806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.448+0000 7f58a4ff9700 1 -- 172.21.15.31:0/3546602363 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f589008aff0 con 0x7f58a814fce0 2024-03-30T01:41:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.448+0000 7f58a7fff700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f588806ac90 0x7f588806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.448+0000 7f58a7fff700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f588806ac90 0x7f588806d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5898000f60 tx=0x7f5898007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:37.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.452+0000 7f58a4ff9700 1 -- 172.21.15.31:0/3546602363 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5890055f90 con 0x7f58a814fce0 2024-03-30T01:41:37.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.664+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58a8066f90 con 0x7f58a814fce0 2024-03-30T01:41:37.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.664+0000 7f58a4ff9700 1 -- 172.21.15.31:0/3546602363 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58900595b0 con 0x7f58a814fce0 2024-03-30T01:41:37.670 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f588806ac90 msgr2=0x7f588806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f588806ac90 0x7f588806d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5898000f60 tx=0x7f5898007be0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a814fce0 msgr2=0x7f58a81e4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a814fce0 0x7f58a81e4d60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5890009f50 tx=0x7f58900047c0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 shutdown_connections 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f588806ac90 0x7f588806d150 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a81525e0 0x7f58a81e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58a80a91c0 0x7f58a81534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 --2- 172.21.15.31:0/3546602363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a814fce0 0x7f58a81e4d60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 >> 172.21.15.31:0/3546602363 conn(0x7f58a80c3910 msgr2=0x7f58a8150370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 shutdown_connections 2024-03-30T01:41:37.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.667+0000 7f58ae6cd700 1 -- 172.21.15.31:0/3546602363 wait complete. 2024-03-30T01:41:37.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.819+0000 7f18f9922700 1 Processor -- start 2024-03-30T01:41:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.820+0000 7f18f9922700 1 -- start start 2024-03-30T01:41:37.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.820+0000 7f18f9922700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18f40a7600 0x7f18f4152f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.820+0000 7f18f9922700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f18f414ff20 0x7f18f40c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.820+0000 7f18f9922700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f40c4620 con 0x7f18f414fb40 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.820+0000 7f18f9922700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f4152570 con 0x7f18f40a7600 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.820+0000 7f18f9922700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f4152ad0 con 0x7f18f414ff20 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.820+0000 7f18f27fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18f40a7600 0x7f18f4152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.820+0000 7f18f27fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f18f40a7600 0x7f18f4152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46380/0 (socket says 172.21.15.31:46380) 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f27fc700 1 -- 172.21.15.31:0/2493833934 learned_addr learned my addr 172.21.15.31:0/2493833934 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f37fe700 1 --2- 172.21.15.31:0/2493833934 >> v2:172.21.15.59:3300/0 conn(0x7f18f414ff20 0x7f18f40c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f27fc700 1 -- 172.21.15.31:0/2493833934 >> v2:172.21.15.59:3300/0 conn(0x7f18f414ff20 msgr2=0x7f18f40c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f27fc700 1 --2- 172.21.15.31:0/2493833934 >> v2:172.21.15.59:3300/0 conn(0x7f18f414ff20 0x7f18f40c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f27fc700 1 -- 172.21.15.31:0/2493833934 >> v1:172.21.15.59:6789/0 conn(0x7f18f414fb40 legacy=0x7f18f40c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f27fc700 1 -- 172.21.15.31:0/2493833934 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f41534a0 con 0x7f18f40a7600 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f37fe700 1 --2- 172.21.15.31:0/2493833934 >> v2:172.21.15.59:3300/0 conn(0x7f18f414ff20 0x7f18f40c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f27fc700 1 --2- 172.21.15.31:0/2493833934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 0x7f18f4152f60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f18f40c8e00 tx=0x7f18dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9932895f265ccd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f1ffb700 1 -- 172.21.15.31:0/2493833934 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00b870 con 0x7f18f40a7600 2024-03-30T01:41:37.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f1ffb700 1 -- 172.21.15.31:0/2493833934 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc00b9d0 con 0x7f18f40a7600 2024-03-30T01:41:37.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.821+0000 7f18f1ffb700 1 -- 172.21.15.31:0/2493833934 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00bd60 con 0x7f18f40a7600 2024-03-30T01:41:37.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.822+0000 7f18f9922700 1 -- 172.21.15.31:0/2493833934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 msgr2=0x7f18f4152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.822+0000 7f18f9922700 1 --2- 172.21.15.31:0/2493833934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 0x7f18f4152f60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f18f40c8e00 tx=0x7f18dc009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.822+0000 7f18f9922700 1 -- 172.21.15.31:0/2493833934 shutdown_connections 2024-03-30T01:41:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.822+0000 7f18f9922700 1 --2- 172.21.15.31:0/2493833934 >> v2:172.21.15.59:3300/0 conn(0x7f18f414ff20 0x7f18f40c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.822+0000 7f18f9922700 1 --2- 172.21.15.31:0/2493833934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 0x7f18f4152f60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.822+0000 7f18f9922700 1 -- 172.21.15.31:0/2493833934 >> 172.21.15.31:0/2493833934 conn(0x7f18f40c7d60 msgr2=0x7f18f40c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.822+0000 7f18f9922700 1 -- 172.21.15.31:0/2493833934 shutdown_connections 2024-03-30T01:41:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.822+0000 7f18f9922700 1 -- 172.21.15.31:0/2493833934 wait complete. 2024-03-30T01:41:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.823+0000 7f18f9922700 1 Processor -- start 2024-03-30T01:41:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.823+0000 7f18f9922700 1 -- start start 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f9922700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 0x7f18f406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f9922700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18f414fb40 0x7f18f40a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f9922700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18f414ff20 0x7f18f40a8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f9922700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f41eab80 con 0x7f18f40a7600 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f9922700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f41e9f70 con 0x7f18f414ff20 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f9922700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f41ea250 con 0x7f18f414fb40 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 0x7f18f406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 0x7f18f406f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46396/0 (socket says 172.21.15.31:46396) 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f2ffd700 1 -- 172.21.15.31:0/4009131634 learned_addr learned my addr 172.21.15.31:0/4009131634 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f27fc700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18f414fb40 0x7f18f40a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f37fe700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18f414ff20 0x7f18f40a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f2ffd700 1 -- 172.21.15.31:0/4009131634 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18f414fb40 msgr2=0x7f18f40a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.824+0000 7f18f2ffd700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18f414fb40 0x7f18f40a7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.825+0000 7f18f2ffd700 1 -- 172.21.15.31:0/4009131634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18f414ff20 msgr2=0x7f18f40a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.825+0000 7f18f2ffd700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18f414ff20 0x7f18f40a8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.825+0000 7f18f2ffd700 1 -- 172.21.15.31:0/4009131634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009550 con 0x7f18f40a7600 2024-03-30T01:41:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.825+0000 7f18f2ffd700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 0x7f18f406f590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18e400c4e0 tx=0x7f18e400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.825+0000 7f18ebfff700 1 -- 172.21.15.31:0/4009131634 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e40052d0 con 0x7f18f40a7600 2024-03-30T01:41:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.825+0000 7f18ebfff700 1 -- 172.21.15.31:0/4009131634 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e4005430 con 0x7f18f40a7600 2024-03-30T01:41:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.825+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f41ea530 con 0x7f18f40a7600 2024-03-30T01:41:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.825+0000 7f18ebfff700 1 -- 172.21.15.31:0/4009131634 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e40038c0 con 0x7f18f40a7600 2024-03-30T01:41:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.825+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f406fad0 con 0x7f18f40a7600 2024-03-30T01:41:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.826+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f40a7b30 con 0x7f18f40a7600 2024-03-30T01:41:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.827+0000 7f18ebfff700 1 -- 172.21.15.31:0/4009131634 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f18e4003a20 con 0x7f18f40a7600 2024-03-30T01:41:37.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.827+0000 7f18ebfff700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d406ac90 0x7f18d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:37.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.827+0000 7f18f27fc700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d406ac90 0x7f18d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:37.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.827+0000 7f18ebfff700 1 -- 172.21.15.31:0/4009131634 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f18e40887a0 con 0x7f18f40a7600 2024-03-30T01:41:37.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.828+0000 7f18f27fc700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d406ac90 0x7f18d406d150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f18dc014040 tx=0x7f18dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:37.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:37.832+0000 7f18ebfff700 1 -- 172.21.15.31:0/4009131634 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f18e4053720 con 0x7f18f40a7600 2024-03-30T01:41:38.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.044+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f18f40c4620 con 0x7f18f40a7600 2024-03-30T01:41:38.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.045+0000 7f18ebfff700 1 -- 172.21.15.31:0/4009131634 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f18f40c4620 con 0x7f18f40a7600 2024-03-30T01:41:38.050 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d406ac90 msgr2=0x7f18d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d406ac90 0x7f18d406d150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f18dc014040 tx=0x7f18dc008030 comp rx=0 tx=0).stop 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 msgr2=0x7f18f406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 0x7f18f406f590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18e400c4e0 tx=0x7f18e400c7f0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 shutdown_connections 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f40a7600 0x7f18f406f590 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d406ac90 0x7f18d406d150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18f414ff20 0x7f18f40a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 --2- 172.21.15.31:0/4009131634 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18f414fb40 0x7f18f40a7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 >> 172.21.15.31:0/4009131634 conn(0x7f18f40c7d60 msgr2=0x7f18f40684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 shutdown_connections 2024-03-30T01:41:38.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.047+0000 7f18f9922700 1 -- 172.21.15.31:0/4009131634 wait complete. 2024-03-30T01:41:38.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.201+0000 7f8d4f684700 1 Processor -- start 2024-03-30T01:41:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.201+0000 7f8d4f684700 1 -- start start 2024-03-30T01:41:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d4f684700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4806f310 0x7f8d4806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d4f684700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8d480c18f0 0x7f8d48156620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d4f684700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d480a8460 con 0x7f8d480c2670 2024-03-30T01:41:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d4f684700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d48156b60 con 0x7f8d480c18f0 2024-03-30T01:41:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d4f684700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d481526a0 con 0x7f8d4806f310 2024-03-30T01:41:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d4dc21700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8d480c2670 0x7f8d48152dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53034/0 (socket says 172.21.15.31:53034) 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d4dc21700 1 -- 172.21.15.31:0/2334987005 learned_addr learned my addr 172.21.15.31:0/2334987005 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d4d420700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4806f310 0x7f8d4806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d4cc1f700 1 --2- 172.21.15.31:0/2334987005 >> v2:172.21.15.59:3300/0 conn(0x7f8d480c18f0 0x7f8d48156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138300487 0 0) 0x7f8d480a8460 con 0x7f8d480c2670 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.202+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d340035b0 con 0x7f8d480c2670 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1154208004 0 0) 0x7f8d340035b0 con 0x7f8d480c2670 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4806f310 msgr2=0x7f8d4806f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4806f310 0x7f8d4806f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 >> v2:172.21.15.59:3300/0 conn(0x7f8d480c18f0 msgr2=0x7f8d48156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 --2- 172.21.15.31:0/2334987005 >> v2:172.21.15.59:3300/0 conn(0x7f8d480c18f0 0x7f8d48156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d480a86b0 con 0x7f8d480c2670 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f8d400019f0 con 0x7f8d480c2670 2024-03-30T01:41:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 >> v1:172.21.15.31:6789/0 conn(0x7f8d480c2670 legacy=0x7f8d48152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4806f310 0x7f8d34003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d34004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d34004d90 0x7f8d34005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d481526a0 con 0x7f8d34004060 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3400ba10 con 0x7f8d34004d90 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3400bbd0 con 0x7f8d4806f310 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4cc1f700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d34004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4d420700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4806f310 0x7f8d34003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4dc21700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d34004d90 0x7f8d34005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4cc1f700 1 -- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4806f310 msgr2=0x7f8d34003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4cc1f700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4806f310 0x7f8d34003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4cc1f700 1 -- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d34004d90 msgr2=0x7f8d34005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4cc1f700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d34004d90 0x7f8d34005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4cc1f700 1 -- 172.21.15.31:0/2334987005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340035b0 con 0x7f8d34004060 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4dc21700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d34004d90 0x7f8d34005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.203+0000 7f8d4cc1f700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d34004850 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009b30 tx=0x7f8d34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2def0f469bec720 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400e6c0 con 0x7f8d34004060 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4400e820 con 0x7f8d34004060 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d3bfff700 1 -- 172.21.15.31:0/2334987005 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400eb90 con 0x7f8d34004060 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d4f684700 1 -- 172.21.15.31:0/2334987005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 msgr2=0x7f8d34004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d4f684700 1 --2- 172.21.15.31:0/2334987005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d34004850 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009b30 tx=0x7f8d34005880 comp rx=0 tx=0).stop 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d4dc21700 1 -- 172.21.15.31:0/2334987005 reap_dead start 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d4f684700 1 -- 172.21.15.31:0/2334987005 shutdown_connections 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d4f684700 1 -- 172.21.15.31:0/2334987005 >> 172.21.15.31:0/2334987005 conn(0x7f8d480c3980 msgr2=0x7f8d480c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d4f684700 1 -- 172.21.15.31:0/2334987005 shutdown_connections 2024-03-30T01:41:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.204+0000 7f8d4f684700 1 -- 172.21.15.31:0/2334987005 wait complete. 2024-03-30T01:41:38.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.205+0000 7f8d4f684700 1 Processor -- start 2024-03-30T01:41:38.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4f684700 1 -- start start 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4f684700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d4814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4f684700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d48151c50 0x7f8d48131110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4f684700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4814f580 0x7f8d48131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4f684700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4814a460 con 0x7f8d34004060 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4f684700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d481ee800 con 0x7f8d48151c50 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4f684700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d481ee370 con 0x7f8d4814f580 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4d420700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d4814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4d420700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d4814f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46410/0 (socket says 172.21.15.31:46410) 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.206+0000 7f8d4d420700 1 -- 172.21.15.31:0/963926497 learned_addr learned my addr 172.21.15.31:0/963926497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4cc1f700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d48151c50 0x7f8d48131110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4dc21700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4814f580 0x7f8d48131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4d420700 1 -- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4814f580 msgr2=0x7f8d48131650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4d420700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4814f580 0x7f8d48131650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4d420700 1 -- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d48151c50 msgr2=0x7f8d48131110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4d420700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d48151c50 0x7f8d48131110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4d420700 1 -- 172.21.15.31:0/963926497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48131b90 con 0x7f8d34004060 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4dc21700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4814f580 0x7f8d48131650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4cc1f700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d48151c50 0x7f8d48131110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4d420700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d4814f040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c011aa0 tx=0x7f8d3c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d39ffb700 1 -- 172.21.15.31:0/963926497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00f980 con 0x7f8d34004060 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d39ffb700 1 -- 172.21.15.31:0/963926497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c00fae0 con 0x7f8d34004060 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d481eda40 con 0x7f8d34004060 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d39ffb700 1 -- 172.21.15.31:0/963926497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00fe90 con 0x7f8d34004060 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.207+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d481ee000 con 0x7f8d34004060 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.208+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d480747d0 con 0x7f8d34004060 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.209+0000 7f8d39ffb700 1 -- 172.21.15.31:0/963926497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d3c014440 con 0x7f8d34004060 2024-03-30T01:41:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.209+0000 7f8d39ffb700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d2c06ac90 0x7f8d2c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.209+0000 7f8d39ffb700 1 -- 172.21.15.31:0/963926497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8d3c089d00 con 0x7f8d34004060 2024-03-30T01:41:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.209+0000 7f8d4cc1f700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d2c06ac90 0x7f8d2c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.210+0000 7f8d4cc1f700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d2c06ac90 0x7f8d2c06d150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d440090b0 tx=0x7f8d44008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.214+0000 7f8d39ffb700 1 -- 172.21.15.31:0/963926497 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d3c054ca0 con 0x7f8d34004060 2024-03-30T01:41:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.426+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8d480c2220 con 0x7f8d34004060 2024-03-30T01:41:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.427+0000 7f8d39ffb700 1 -- 172.21.15.31:0/963926497 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8d3c022020 con 0x7f8d34004060 2024-03-30T01:41:38.432 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.429+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d2c06ac90 msgr2=0x7f8d2c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.429+0000 7f8d4f684700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d2c06ac90 0x7f8d2c06d150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d440090b0 tx=0x7f8d44008030 comp rx=0 tx=0).stop 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.429+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 msgr2=0x7f8d4814f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.429+0000 7f8d4f684700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d4814f040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c011aa0 tx=0x7f8d3c011e60 comp rx=0 tx=0).stop 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.430+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 shutdown_connections 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.430+0000 7f8d4f684700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d34004060 0x7f8d4814f040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.430+0000 7f8d4f684700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d2c06ac90 0x7f8d2c06d150 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.430+0000 7f8d4f684700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d4814f580 0x7f8d48131650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.430+0000 7f8d4f684700 1 --2- 172.21.15.31:0/963926497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d48151c50 0x7f8d48131110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.430+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 >> 172.21.15.31:0/963926497 conn(0x7f8d480c3980 msgr2=0x7f8d480c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.430+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 shutdown_connections 2024-03-30T01:41:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.430+0000 7f8d4f684700 1 -- 172.21.15.31:0/963926497 wait complete. 2024-03-30T01:41:38.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.579+0000 7f004bb88700 1 Processor -- start 2024-03-30T01:41:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.579+0000 7f004bb88700 1 -- start start 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004bb88700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f00440ad010 0x7f00440c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004bb88700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f00440a74e0 0x7f00440c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004bb88700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0044153590 con 0x7f00440ac520 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004bb88700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00440ab680 con 0x7f00440a74e0 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004bb88700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00441500c0 con 0x7f00440ad010 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004a125700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f00440a74e0 0x7f00440c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004a125700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f00440a74e0 0x7f00440c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46420/0 (socket says 172.21.15.31:46420) 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f0049924700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f00440ac520 0x7f00440ac900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53036/0 (socket says 172.21.15.31:53036) 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004a125700 1 -- 172.21.15.31:0/3369068046 learned_addr learned my addr 172.21.15.31:0/3369068046 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f0049123700 1 --2- 172.21.15.31:0/3369068046 >> v2:172.21.15.59:3300/0 conn(0x7f00440ad010 0x7f00440c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004a125700 1 -- 172.21.15.31:0/3369068046 >> v2:172.21.15.59:3300/0 conn(0x7f00440ad010 msgr2=0x7f00440c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004a125700 1 --2- 172.21.15.31:0/3369068046 >> v2:172.21.15.59:3300/0 conn(0x7f00440ad010 0x7f00440c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004a125700 1 -- 172.21.15.31:0/3369068046 >> v1:172.21.15.31:6789/0 conn(0x7f00440ac520 legacy=0x7f00440ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f004a125700 1 -- 172.21.15.31:0/3369068046 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00440c7990 con 0x7f00440a74e0 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f0048922700 1 -- 172.21.15.31:0/3369068046 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065650128 0 0) 0x7f0044153590 con 0x7f00440ac520 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.580+0000 7f0049123700 1 --2- 172.21.15.31:0/3369068046 >> v2:172.21.15.59:3300/0 conn(0x7f00440ad010 0x7f00440c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f004a125700 1 --2- 172.21.15.31:0/3369068046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440a74e0 0x7f00440c6540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00440aa350 tx=0x7f003c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6348aaa6e29130e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f0048922700 1 -- 172.21.15.31:0/3369068046 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00e070 con 0x7f00440a74e0 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f0048922700 1 -- 172.21.15.31:0/3369068046 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c00b9c0 con 0x7f00440a74e0 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f0048922700 1 -- 172.21.15.31:0/3369068046 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00bd70 con 0x7f00440a74e0 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f004bb88700 1 -- 172.21.15.31:0/3369068046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440a74e0 msgr2=0x7f00440c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f004bb88700 1 --2- 172.21.15.31:0/3369068046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440a74e0 0x7f00440c6540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00440aa350 tx=0x7f003c00b330 comp rx=0 tx=0).stop 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f004bb88700 1 -- 172.21.15.31:0/3369068046 shutdown_connections 2024-03-30T01:41:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f004bb88700 1 --2- 172.21.15.31:0/3369068046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440a74e0 0x7f00440c6540 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f004bb88700 1 --2- 172.21.15.31:0/3369068046 >> v2:172.21.15.59:3300/0 conn(0x7f00440ad010 0x7f00440c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f004bb88700 1 -- 172.21.15.31:0/3369068046 >> 172.21.15.31:0/3369068046 conn(0x7f004406ad30 msgr2=0x7f00440a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f004bb88700 1 -- 172.21.15.31:0/3369068046 shutdown_connections 2024-03-30T01:41:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.581+0000 7f004bb88700 1 -- 172.21.15.31:0/3369068046 wait complete. 2024-03-30T01:41:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.583+0000 7f004bb88700 1 Processor -- start 2024-03-30T01:41:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.583+0000 7f004bb88700 1 -- start start 2024-03-30T01:41:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.583+0000 7f004bb88700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00440a74e0 0x7f00440a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.583+0000 7f004bb88700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00440ac520 0x7f004414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.583+0000 7f004bb88700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440ad010 0x7f004414e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.583+0000 7f004bb88700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0044153ae0 con 0x7f00440ad010 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.583+0000 7f004bb88700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004406d830 con 0x7f00440a74e0 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.583+0000 7f004bb88700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004414ec40 con 0x7f00440ac520 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004a125700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440ad010 0x7f004414e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004a125700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440ad010 0x7f004414e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46430/0 (socket says 172.21.15.31:46430) 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004a125700 1 -- 172.21.15.31:0/2422372424 learned_addr learned my addr 172.21.15.31:0/2422372424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f0049123700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00440ac520 0x7f004414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f0049924700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00440a74e0 0x7f00440a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004a125700 1 -- 172.21.15.31:0/2422372424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00440ac520 msgr2=0x7f004414e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004a125700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00440ac520 0x7f004414e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004a125700 1 -- 172.21.15.31:0/2422372424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00440a74e0 msgr2=0x7f00440a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004a125700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00440a74e0 0x7f00440a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004a125700 1 -- 172.21.15.31:0/2422372424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c009b30 con 0x7f00440ad010 2024-03-30T01:41:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004a125700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440ad010 0x7f004414e700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f00440c5690 tx=0x7f003c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:38.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f003a7fc700 1 -- 172.21.15.31:0/2422372424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c004920 con 0x7f00440ad010 2024-03-30T01:41:38.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f003a7fc700 1 -- 172.21.15.31:0/2422372424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c0092e0 con 0x7f00440ad010 2024-03-30T01:41:38.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00441edbe0 con 0x7f00440ad010 2024-03-30T01:41:38.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f003a7fc700 1 -- 172.21.15.31:0/2422372424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c01c660 con 0x7f00440ad010 2024-03-30T01:41:38.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.584+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004406d1e0 con 0x7f00440ad010 2024-03-30T01:41:38.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.585+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004404ea80 con 0x7f00440ad010 2024-03-30T01:41:38.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.586+0000 7f003a7fc700 1 -- 172.21.15.31:0/2422372424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f003c01c7c0 con 0x7f00440ad010 2024-03-30T01:41:38.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.586+0000 7f003a7fc700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f002806ac90 0x7f002806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.587+0000 7f003a7fc700 1 -- 172.21.15.31:0/2422372424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f003c08b210 con 0x7f00440ad010 2024-03-30T01:41:38.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.587+0000 7f0049924700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f002806ac90 0x7f002806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.587+0000 7f0049924700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f002806ac90 0x7f002806d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0040000f60 tx=0x7f0040007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:38.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.591+0000 7f003a7fc700 1 -- 172.21.15.31:0/2422372424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f003c056190 con 0x7f00440ad010 2024-03-30T01:41:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.806+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00440c29f0 con 0x7f00440ad010 2024-03-30T01:41:38.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.808+0000 7f003a7fc700 1 -- 172.21.15.31:0/2422372424 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f003c0597b0 con 0x7f00440ad010 2024-03-30T01:41:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:38.814 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f002806ac90 msgr2=0x7f002806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f002806ac90 0x7f002806d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0040000f60 tx=0x7f0040007c00 comp rx=0 tx=0).stop 2024-03-30T01:41:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440ad010 msgr2=0x7f004414e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440ad010 0x7f004414e700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f00440c5690 tx=0x7f003c004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 shutdown_connections 2024-03-30T01:41:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00440ad010 0x7f004414e700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f002806ac90 0x7f002806d150 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00440a74e0 0x7f00440a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 --2- 172.21.15.31:0/2422372424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00440ac520 0x7f004414e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 >> 172.21.15.31:0/2422372424 conn(0x7f004406ad30 msgr2=0x7f0044070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:38.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 shutdown_connections 2024-03-30T01:41:38.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.810+0000 7f004bb88700 1 -- 172.21.15.31:0/2422372424 wait complete. 2024-03-30T01:41:38.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:38.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.963+0000 7fd478498700 1 Processor -- start 2024-03-30T01:41:38.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.963+0000 7fd478498700 1 -- start start 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.964+0000 7fd478498700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd47006e630 0x7fd4700ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.964+0000 7fd478498700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4700c2e90 con 0x7fd4700ab860 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd478498700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd470152b40 con 0x7fd47006da30 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd478498700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47006ba60 con 0x7fd47006e630 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd476a35700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd4700ab860 0x7fd47014f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53046/0 (socket says 172.21.15.31:53046) 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd476a35700 1 -- 172.21.15.31:0/3262860725 learned_addr learned my addr 172.21.15.31:0/3262860725 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd475a33700 1 --2- 172.21.15.31:0/3262860725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd47006e630 0x7fd4700ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd475232700 1 -- 172.21.15.31:0/3262860725 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776725383 0 0) 0x7fd468001490 con 0x7fd4700ab860 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd475232700 1 -- 172.21.15.31:0/3262860725 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd45c0035b0 con 0x7fd4700ab860 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd475232700 1 -- 172.21.15.31:0/3262860725 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760259997 0 0) 0x7fd470152b40 con 0x7fd47006da30 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd475232700 1 -- 172.21.15.31:0/3262860725 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd468001490 con 0x7fd47006da30 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd475232700 1 -- 172.21.15.31:0/3262860725 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3502991575 0 0) 0x7fd45c0035b0 con 0x7fd4700ab860 2024-03-30T01:41:38.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd475a33700 1 -- 172.21.15.31:0/3262860725 >> v1:172.21.15.59:6789/0 conn(0x7fd47006da30 legacy=0x7fd470152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd475a33700 1 -- 172.21.15.31:0/3262860725 >> v1:172.21.15.31:6789/0 conn(0x7fd4700ab860 legacy=0x7fd47014f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.965+0000 7fd475a33700 1 -- 172.21.15.31:0/3262860725 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47006bd40 con 0x7fd47006e630 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd475a33700 1 --2- 172.21.15.31:0/3262860725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd47006e630 0x7fd4700ac2d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd460009870 tx=0x7fd460009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a98b94bd72ad90 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd475232700 1 -- 172.21.15.31:0/3262860725 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468001b80 con 0x7fd47006e630 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd475232700 1 -- 172.21.15.31:0/3262860725 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd460004030 con 0x7fd47006e630 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd475232700 1 -- 172.21.15.31:0/3262860725 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600043c0 con 0x7fd47006e630 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd478498700 1 -- 172.21.15.31:0/3262860725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd47006e630 msgr2=0x7fd4700ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd478498700 1 --2- 172.21.15.31:0/3262860725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd47006e630 0x7fd4700ac2d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd460009870 tx=0x7fd460009b80 comp rx=0 tx=0).stop 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd478498700 1 -- 172.21.15.31:0/3262860725 shutdown_connections 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd478498700 1 --2- 172.21.15.31:0/3262860725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd47006e630 0x7fd4700ac2d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd478498700 1 -- 172.21.15.31:0/3262860725 >> 172.21.15.31:0/3262860725 conn(0x7fd4700c65c0 msgr2=0x7fd4700c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd478498700 1 -- 172.21.15.31:0/3262860725 shutdown_connections 2024-03-30T01:41:38.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.966+0000 7fd478498700 1 -- 172.21.15.31:0/3262860725 wait complete. 2024-03-30T01:41:38.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.968+0000 7fd478498700 1 Processor -- start 2024-03-30T01:41:38.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.968+0000 7fd478498700 1 -- start start 2024-03-30T01:41:38.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.968+0000 7fd478498700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd47006da30 0x7fd470146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.968+0000 7fd478498700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd47006e630 0x7fd470150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.968+0000 7fd478498700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4700ab860 0x7fd470150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.968+0000 7fd478498700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4701414a0 con 0x7fd47006da30 2024-03-30T01:41:38.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.968+0000 7fd478498700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700c46a0 con 0x7fd47006e630 2024-03-30T01:41:38.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.968+0000 7fd478498700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4701510c0 con 0x7fd4700ab860 2024-03-30T01:41:38.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.968+0000 7fd476234700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd47006da30 0x7fd470146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476234700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd47006da30 0x7fd470146a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46436/0 (socket says 172.21.15.31:46436) 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476234700 1 -- 172.21.15.31:0/2318178885 learned_addr learned my addr 172.21.15.31:0/2318178885 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd475a33700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd47006e630 0x7fd470150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476a35700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4700ab860 0x7fd470150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476234700 1 -- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4700ab860 msgr2=0x7fd470150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476234700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4700ab860 0x7fd470150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476234700 1 -- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd47006e630 msgr2=0x7fd470150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476234700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd47006e630 0x7fd470150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476234700 1 -- 172.21.15.31:0/2318178885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4600094a0 con 0x7fd47006da30 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd475a33700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd47006e630 0x7fd470150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476a35700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4700ab860 0x7fd470150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd476234700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd47006da30 0x7fd470146a20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c500 tx=0x7fd46c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd466ffd700 1 -- 172.21.15.31:0/2318178885 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0052d0 con 0x7fd47006da30 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd466ffd700 1 -- 172.21.15.31:0/2318178885 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c005450 con 0x7fd47006da30 2024-03-30T01:41:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4700c3d70 con 0x7fd47006da30 2024-03-30T01:41:38.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd466ffd700 1 -- 172.21.15.31:0/2318178885 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0038c0 con 0x7fd47006da30 2024-03-30T01:41:38.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.969+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4700c4330 con 0x7fd47006da30 2024-03-30T01:41:38.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.970+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4700747d0 con 0x7fd47006da30 2024-03-30T01:41:38.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.971+0000 7fd466ffd700 1 -- 172.21.15.31:0/2318178885 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd46c003a20 con 0x7fd47006da30 2024-03-30T01:41:38.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.971+0000 7fd466ffd700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd45406ac90 0x7fd45406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:38.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.971+0000 7fd466ffd700 1 -- 172.21.15.31:0/2318178885 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd46c088a50 con 0x7fd47006da30 2024-03-30T01:41:38.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.971+0000 7fd475a33700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd45406ac90 0x7fd45406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:38.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.972+0000 7fd475a33700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd45406ac90 0x7fd45406d150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd4600097c0 tx=0x7fd460008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:38.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:38.976+0000 7fd466ffd700 1 -- 172.21.15.31:0/2318178885 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd46c0539d0 con 0x7fd47006da30 2024-03-30T01:41:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.188+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4700c44c0 con 0x7fd47006da30 2024-03-30T01:41:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.188+0000 7fd466ffd700 1 -- 172.21.15.31:0/2318178885 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd46c0170a0 con 0x7fd47006da30 2024-03-30T01:41:39.194 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd45406ac90 msgr2=0x7fd45406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd45406ac90 0x7fd45406d150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd4600097c0 tx=0x7fd460008030 comp rx=0 tx=0).stop 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd47006da30 msgr2=0x7fd470146a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd47006da30 0x7fd470146a20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c500 tx=0x7fd46c00c8c0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 shutdown_connections 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd47006da30 0x7fd470146a20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd45406ac90 0x7fd45406d150 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4700ab860 0x7fd470150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 --2- 172.21.15.31:0/2318178885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd47006e630 0x7fd470150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 >> 172.21.15.31:0/2318178885 conn(0x7fd4700c65c0 msgr2=0x7fd47014c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 shutdown_connections 2024-03-30T01:41:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.191+0000 7fd478498700 1 -- 172.21.15.31:0/2318178885 wait complete. 2024-03-30T01:41:39.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.343+0000 7f807778d700 1 Processor -- start 2024-03-30T01:41:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.343+0000 7f807778d700 1 -- start start 2024-03-30T01:41:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f807778d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f807006f310 0x7f807006f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f807778d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c2670 0x7f8070156620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f807778d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80700a8460 con 0x7f80700c18f0 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f807778d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8070156b60 con 0x7f807006f310 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f807778d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80701526a0 con 0x7f80700c2670 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f8075529700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f807006f310 0x7f807006f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f8075d2a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c2670 0x7f8070156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f8074d28700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f80700c18f0 0x7f8070152dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:47168/0 (socket says 172.21.15.31:47168) 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f8074d28700 1 -- 172.21.15.31:0/2991364090 learned_addr learned my addr 172.21.15.31:0/2991364090 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.344+0000 7f8067fff700 1 -- 172.21.15.31:0/2991364090 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147719606 0 0) 0x7f80700a8460 con 0x7f80700c18f0 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.345+0000 7f8075d2a700 1 -- 172.21.15.31:0/2991364090 >> v2:172.21.15.59:3300/0 conn(0x7f807006f310 msgr2=0x7f807006f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.345+0000 7f8075d2a700 1 --2- 172.21.15.31:0/2991364090 >> v2:172.21.15.59:3300/0 conn(0x7f807006f310 0x7f807006f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.345+0000 7f8075d2a700 1 -- 172.21.15.31:0/2991364090 >> v1:172.21.15.59:6789/0 conn(0x7f80700c18f0 legacy=0x7f8070152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.345+0000 7f8075d2a700 1 -- 172.21.15.31:0/2991364090 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700a86b0 con 0x7f80700c2670 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.345+0000 7f8075d2a700 1 --2- 172.21.15.31:0/2991364090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c2670 0x7f8070156620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8068009d90 tx=0x7f806800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72fe903b0165a041 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.345+0000 7f8067fff700 1 -- 172.21.15.31:0/2991364090 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806800e070 con 0x7f80700c2670 2024-03-30T01:41:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.345+0000 7f8067fff700 1 -- 172.21.15.31:0/2991364090 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806800bb20 con 0x7f80700c2670 2024-03-30T01:41:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.345+0000 7f8067fff700 1 -- 172.21.15.31:0/2991364090 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068012430 con 0x7f80700c2670 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.346+0000 7f807778d700 1 -- 172.21.15.31:0/2991364090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c2670 msgr2=0x7f8070156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.346+0000 7f807778d700 1 --2- 172.21.15.31:0/2991364090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c2670 0x7f8070156620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8068009d90 tx=0x7f806800b330 comp rx=0 tx=0).stop 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.346+0000 7f807778d700 1 -- 172.21.15.31:0/2991364090 shutdown_connections 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.346+0000 7f807778d700 1 --2- 172.21.15.31:0/2991364090 >> v2:172.21.15.59:3300/0 conn(0x7f807006f310 0x7f807006f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.346+0000 7f807778d700 1 --2- 172.21.15.31:0/2991364090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c2670 0x7f8070156620 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.346+0000 7f807778d700 1 -- 172.21.15.31:0/2991364090 >> 172.21.15.31:0/2991364090 conn(0x7f80700c3980 msgr2=0x7f80700c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.346+0000 7f807778d700 1 -- 172.21.15.31:0/2991364090 shutdown_connections 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.346+0000 7f807778d700 1 -- 172.21.15.31:0/2991364090 wait complete. 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.347+0000 7f807778d700 1 Processor -- start 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.347+0000 7f807778d700 1 -- start start 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f807778d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f807006f310 0x7f8070133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f807778d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c18f0 0x7f80701ea370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f807778d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80700c2670 0x7f80701ea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f807778d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8070139de0 con 0x7f80700c2670 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f807778d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807006f1b0 con 0x7f807006f310 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f807778d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8070139b00 con 0x7f80700c18f0 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075d2a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80700c2670 0x7f80701ea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075d2a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80700c2670 0x7f80701ea8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46440/0 (socket says 172.21.15.31:46440) 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075d2a700 1 -- 172.21.15.31:0/712446978 learned_addr learned my addr 172.21.15.31:0/712446978 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8074d28700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c18f0 0x7f80701ea370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075529700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f807006f310 0x7f8070133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075d2a700 1 -- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c18f0 msgr2=0x7f80701ea370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075d2a700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c18f0 0x7f80701ea370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075d2a700 1 -- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f807006f310 msgr2=0x7f8070133830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075d2a700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f807006f310 0x7f8070133830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075d2a700 1 -- 172.21.15.31:0/712446978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8068009a70 con 0x7f80700c2670 2024-03-30T01:41:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075529700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f807006f310 0x7f8070133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8074d28700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c18f0 0x7f80701ea370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.348+0000 7f8075d2a700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80700c2670 0x7f80701ea8b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8068013000 tx=0x7f8068004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.349+0000 7f8065ffb700 1 -- 172.21.15.31:0/712446978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680098f0 con 0x7f80700c2670 2024-03-30T01:41:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.349+0000 7f8065ffb700 1 -- 172.21.15.31:0/712446978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8068020070 con 0x7f80700c2670 2024-03-30T01:41:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.349+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807006eed0 con 0x7f80700c2670 2024-03-30T01:41:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.349+0000 7f8065ffb700 1 -- 172.21.15.31:0/712446978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680276c0 con 0x7f80700c2670 2024-03-30T01:41:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.349+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8070133d70 con 0x7f80700c2670 2024-03-30T01:41:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.350+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80701ee2f0 con 0x7f80700c2670 2024-03-30T01:41:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.350+0000 7f8065ffb700 1 -- 172.21.15.31:0/712446978 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8068012430 con 0x7f80700c2670 2024-03-30T01:41:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.351+0000 7f8065ffb700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f805406ac90 0x7f805406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.351+0000 7f8065ffb700 1 -- 172.21.15.31:0/712446978 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8068022070 con 0x7f80700c2670 2024-03-30T01:41:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.351+0000 7f8075529700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f805406ac90 0x7f805406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.352+0000 7f8075529700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f805406ac90 0x7f805406d150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f806c009c30 tx=0x7f806c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:39.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.356+0000 7f8065ffb700 1 -- 172.21.15.31:0/712446978 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f80701ee2f0 con 0x7f80700c2670 2024-03-30T01:41:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.568+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80701ee6d0 con 0x7f80700c2670 2024-03-30T01:41:39.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.568+0000 7f8065ffb700 1 -- 172.21.15.31:0/712446978 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80701ee6d0 con 0x7f80700c2670 2024-03-30T01:41:39.574 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f805406ac90 msgr2=0x7f805406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f805406ac90 0x7f805406d150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f806c009c30 tx=0x7f806c008040 comp rx=0 tx=0).stop 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80700c2670 msgr2=0x7f80701ea8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80700c2670 0x7f80701ea8b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8068013000 tx=0x7f8068004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 shutdown_connections 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80700c2670 0x7f80701ea8b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f805406ac90 0x7f805406d150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f807006f310 0x7f8070133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 --2- 172.21.15.31:0/712446978 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80700c18f0 0x7f80701ea370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 >> 172.21.15.31:0/712446978 conn(0x7f80700c3980 msgr2=0x7f80700c2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 shutdown_connections 2024-03-30T01:41:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.571+0000 7f807778d700 1 -- 172.21.15.31:0/712446978 wait complete. 2024-03-30T01:41:39.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:39.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.723+0000 7f622d1da700 1 Processor -- start 2024-03-30T01:41:39.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.723+0000 7f622d1da700 1 -- start start 2024-03-30T01:41:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622d1da700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62281549a0 0x7f6228072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622d1da700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f622806d040 0x7f6228154d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622d1da700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d900 0x7f622806dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622d1da700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622814ee60 con 0x7f622806d040 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622d1da700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62280c2290 con 0x7f62281549a0 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622d1da700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622806e2b0 con 0x7f622806d900 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f622806d040 0x7f6228154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f622806d040 0x7f6228154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46446/0 (socket says 172.21.15.31:46446) 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622659c700 1 -- 172.21.15.31:0/2821551303 learned_addr learned my addr 172.21.15.31:0/2821551303 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f6226d9d700 1 --2- 172.21.15.31:0/2821551303 >> v2:172.21.15.59:3300/0 conn(0x7f62281549a0 0x7f6228072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.724+0000 7f622759e700 1 --2- 172.21.15.31:0/2821551303 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d900 0x7f622806dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.725+0000 7f622659c700 1 -- 172.21.15.31:0/2821551303 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d900 msgr2=0x7f622806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.725+0000 7f622659c700 1 --2- 172.21.15.31:0/2821551303 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d900 0x7f622806dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.725+0000 7f622659c700 1 -- 172.21.15.31:0/2821551303 >> v2:172.21.15.59:3300/0 conn(0x7f62281549a0 msgr2=0x7f6228072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.725+0000 7f622659c700 1 --2- 172.21.15.31:0/2821551303 >> v2:172.21.15.59:3300/0 conn(0x7f62281549a0 0x7f6228072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.725+0000 7f622659c700 1 -- 172.21.15.31:0/2821551303 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228155720 con 0x7f622806d040 2024-03-30T01:41:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.725+0000 7f622659c700 1 --2- 172.21.15.31:0/2821551303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622806d040 0x7f6228154d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f62280733c0 tx=0x7f6218009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee89ade1680255e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.725+0000 7f6225d9b700 1 -- 172.21.15.31:0/2821551303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218011830 con 0x7f622806d040 2024-03-30T01:41:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.725+0000 7f6225d9b700 1 -- 172.21.15.31:0/2821551303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6218011990 con 0x7f622806d040 2024-03-30T01:41:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.725+0000 7f6225d9b700 1 -- 172.21.15.31:0/2821551303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218011d20 con 0x7f622806d040 2024-03-30T01:41:39.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.726+0000 7f622d1da700 1 -- 172.21.15.31:0/2821551303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622806d040 msgr2=0x7f6228154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.726+0000 7f622d1da700 1 --2- 172.21.15.31:0/2821551303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622806d040 0x7f6228154d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f62280733c0 tx=0x7f6218009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.726+0000 7f622d1da700 1 -- 172.21.15.31:0/2821551303 shutdown_connections 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.726+0000 7f622d1da700 1 --2- 172.21.15.31:0/2821551303 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d900 0x7f622806dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.726+0000 7f622d1da700 1 --2- 172.21.15.31:0/2821551303 >> v2:172.21.15.59:3300/0 conn(0x7f62281549a0 0x7f6228072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.726+0000 7f622d1da700 1 --2- 172.21.15.31:0/2821551303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f622806d040 0x7f6228154d80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.726+0000 7f622d1da700 1 -- 172.21.15.31:0/2821551303 >> 172.21.15.31:0/2821551303 conn(0x7f62280c3980 msgr2=0x7f62280c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.726+0000 7f622d1da700 1 -- 172.21.15.31:0/2821551303 shutdown_connections 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.726+0000 7f622d1da700 1 -- 172.21.15.31:0/2821551303 wait complete. 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.727+0000 7f622d1da700 1 Processor -- start 2024-03-30T01:41:39.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.727+0000 7f622d1da700 1 -- start start 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622d1da700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d040 0x7f62280c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622d1da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f622806d900 0x7f622813aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622d1da700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281549a0 0x7f622813b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622d1da700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62281e1740 con 0x7f62281549a0 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622d1da700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622813b550 con 0x7f622806d900 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622d1da700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622813b910 con 0x7f622806d040 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281549a0 0x7f622813b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281549a0 0x7f622813b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46452/0 (socket says 172.21.15.31:46452) 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622759e700 1 -- 172.21.15.31:0/126811044 learned_addr learned my addr 172.21.15.31:0/126811044 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:39.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f6226d9d700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d040 0x7f62280c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.728+0000 7f622659c700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f622806d900 0x7f622813aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f622759e700 1 -- 172.21.15.31:0/126811044 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d040 msgr2=0x7f62280c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f622759e700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d040 0x7f62280c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f622759e700 1 -- 172.21.15.31:0/126811044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f622806d900 msgr2=0x7f622813aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f622759e700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f622806d900 0x7f622813aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f622759e700 1 -- 172.21.15.31:0/126811044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6218009550 con 0x7f62281549a0 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f622759e700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281549a0 0x7f622813b010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f622000eed0 tx=0x7f622000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f620f7fe700 1 -- 172.21.15.31:0/126811044 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220010040 con 0x7f62281549a0 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f620f7fe700 1 -- 172.21.15.31:0/126811044 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622000bc80 con 0x7f62281549a0 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62281e1950 con 0x7f62281549a0 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f620f7fe700 1 -- 172.21.15.31:0/126811044 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220009e40 con 0x7f62281549a0 2024-03-30T01:41:39.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.729+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6228154600 con 0x7f62281549a0 2024-03-30T01:41:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.730+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62280c4870 con 0x7f62281549a0 2024-03-30T01:41:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.731+0000 7f620f7fe700 1 -- 172.21.15.31:0/126811044 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6220014400 con 0x7f62281549a0 2024-03-30T01:41:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.731+0000 7f620f7fe700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f621406ac90 0x7f621406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.731+0000 7f620f7fe700 1 -- 172.21.15.31:0/126811044 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6220089930 con 0x7f62281549a0 2024-03-30T01:41:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.731+0000 7f6226d9d700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f621406ac90 0x7f621406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.732+0000 7f6226d9d700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f621406ac90 0x7f621406d150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f621c009b40 tx=0x7f621c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:39.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.736+0000 7f620f7fe700 1 -- 172.21.15.31:0/126811044 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f62200548b0 con 0x7f62281549a0 2024-03-30T01:41:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.947+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62280747d0 con 0x7f62281549a0 2024-03-30T01:41:39.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.948+0000 7f620f7fe700 1 -- 172.21.15.31:0/126811044 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62200180a0 con 0x7f62281549a0 2024-03-30T01:41:39.953 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f621406ac90 msgr2=0x7f621406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f621406ac90 0x7f621406d150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f621c009b40 tx=0x7f621c009210 comp rx=0 tx=0).stop 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281549a0 msgr2=0x7f622813b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281549a0 0x7f622813b010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f622000eed0 tx=0x7f622000b390 comp rx=0 tx=0).stop 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 shutdown_connections 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62281549a0 0x7f622813b010 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f621406ac90 0x7f621406d150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f622806d040 0x7f62280c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 --2- 172.21.15.31:0/126811044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f622806d900 0x7f622813aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 >> 172.21.15.31:0/126811044 conn(0x7f62280c3980 msgr2=0x7f62280c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 shutdown_connections 2024-03-30T01:41:39.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:39.950+0000 7f622d1da700 1 -- 172.21.15.31:0/126811044 wait complete. 2024-03-30T01:41:39.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:40.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.103+0000 7f74db125700 1 Processor -- start 2024-03-30T01:41:40.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.103+0000 7f74db125700 1 -- start start 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74db125700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74d41549a0 0x7f74d4072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74db125700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f74d406d900 0x7f74d406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74db125700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d414ee60 con 0x7f74d406d040 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74db125700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d40c2290 con 0x7f74d41549a0 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74db125700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d4155490 con 0x7f74d406d900 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74d8ec1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74d41549a0 0x7f74d4072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74d8ec1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74d41549a0 0x7f74d4072410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46460/0 (socket says 172.21.15.31:46460) 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74d8ec1700 1 -- 172.21.15.31:0/3205699866 learned_addr learned my addr 172.21.15.31:0/3205699866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74d96c2700 1 --2- 172.21.15.31:0/3205699866 >> v2:172.21.15.59:3300/0 conn(0x7f74d406d900 0x7f74d406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74d8ec1700 1 -- 172.21.15.31:0/3205699866 >> v2:172.21.15.59:3300/0 conn(0x7f74d406d900 msgr2=0x7f74d406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74d8ec1700 1 --2- 172.21.15.31:0/3205699866 >> v2:172.21.15.59:3300/0 conn(0x7f74d406d900 0x7f74d406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74d8ec1700 1 -- 172.21.15.31:0/3205699866 >> v1:172.21.15.59:6789/0 conn(0x7f74d406d040 legacy=0x7f74d4154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.104+0000 7f74d8ec1700 1 -- 172.21.15.31:0/3205699866 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d406e270 con 0x7f74d41549a0 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74d8ec1700 1 --2- 172.21.15.31:0/3205699866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d41549a0 0x7f74d4072410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f74d40c6e30 tx=0x7f74c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910548049f637f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74d37fe700 1 -- 172.21.15.31:0/3205699866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c800f8a0 con 0x7f74d41549a0 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74d37fe700 1 -- 172.21.15.31:0/3205699866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c800fa00 con 0x7f74d41549a0 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74d37fe700 1 -- 172.21.15.31:0/3205699866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c800fd90 con 0x7f74d41549a0 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74db125700 1 -- 172.21.15.31:0/3205699866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d41549a0 msgr2=0x7f74d4072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74db125700 1 --2- 172.21.15.31:0/3205699866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d41549a0 0x7f74d4072410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f74d40c6e30 tx=0x7f74c8009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74db125700 1 -- 172.21.15.31:0/3205699866 shutdown_connections 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74db125700 1 --2- 172.21.15.31:0/3205699866 >> v2:172.21.15.59:3300/0 conn(0x7f74d406d900 0x7f74d406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74db125700 1 --2- 172.21.15.31:0/3205699866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d41549a0 0x7f74d4072410 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74db125700 1 -- 172.21.15.31:0/3205699866 >> 172.21.15.31:0/3205699866 conn(0x7f74d40c3980 msgr2=0x7f74d40c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74db125700 1 -- 172.21.15.31:0/3205699866 shutdown_connections 2024-03-30T01:41:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.105+0000 7f74db125700 1 -- 172.21.15.31:0/3205699866 wait complete. 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.107+0000 7f74db125700 1 Processor -- start 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.107+0000 7f74db125700 1 -- start start 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.107+0000 7f74db125700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d406d040 0x7f74d4146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.107+0000 7f74db125700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74d406d900 0x7f74d4146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.107+0000 7f74db125700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74d41549a0 0x7f74d40c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.107+0000 7f74db125700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4146590 con 0x7f74d406d040 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.107+0000 7f74db125700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4145db0 con 0x7f74d406d900 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.107+0000 7f74db125700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d41ed490 con 0x7f74d41549a0 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d8ec1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d406d040 0x7f74d4146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d8ec1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d406d040 0x7f74d4146050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46464/0 (socket says 172.21.15.31:46464) 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d8ec1700 1 -- 172.21.15.31:0/2692318910 learned_addr learned my addr 172.21.15.31:0/2692318910 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d96c2700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74d41549a0 0x7f74d40c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d3fff700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74d406d900 0x7f74d4146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d8ec1700 1 -- 172.21.15.31:0/2692318910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74d41549a0 msgr2=0x7f74d40c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d8ec1700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74d41549a0 0x7f74d40c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d8ec1700 1 -- 172.21.15.31:0/2692318910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74d406d900 msgr2=0x7f74d4146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d8ec1700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74d406d900 0x7f74d4146e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d8ec1700 1 -- 172.21.15.31:0/2692318910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c8009550 con 0x7f74d406d040 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d8ec1700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d406d040 0x7f74d4146050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009c10 tx=0x7f74c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d17fa700 1 -- 172.21.15.31:0/2692318910 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c800fd70 con 0x7f74d406d040 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d17fa700 1 -- 172.21.15.31:0/2692318910 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c8004030 con 0x7f74d406d040 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d41ed730 con 0x7f74d406d040 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74d17fa700 1 -- 172.21.15.31:0/2692318910 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c80043a0 con 0x7f74d406d040 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.108+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d4147390 con 0x7f74d406d040 2024-03-30T01:41:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.109+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d40c4870 con 0x7f74d406d040 2024-03-30T01:41:40.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.110+0000 7f74d17fa700 1 -- 172.21.15.31:0/2692318910 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f74c80036a0 con 0x7f74d406d040 2024-03-30T01:41:40.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.110+0000 7f74d17fa700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f74b806ac90 0x7f74b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.110+0000 7f74d17fa700 1 -- 172.21.15.31:0/2692318910 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f74c808b0a0 con 0x7f74d406d040 2024-03-30T01:41:40.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.110+0000 7f74d3fff700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f74b806ac90 0x7f74b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.111+0000 7f74d3fff700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f74b806ac90 0x7f74b806d150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f74d406f890 tx=0x7f74c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:40.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.115+0000 7f74d17fa700 1 -- 172.21.15.31:0/2692318910 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f74c8055fa0 con 0x7f74d406d040 2024-03-30T01:41:40.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.327+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f74d40747d0 con 0x7f74d406d040 2024-03-30T01:41:40.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.328+0000 7f74d17fa700 1 -- 172.21.15.31:0/2692318910 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f74c80595c0 con 0x7f74d406d040 2024-03-30T01:41:40.333 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f74b806ac90 msgr2=0x7f74b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f74b806ac90 0x7f74b806d150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f74d406f890 tx=0x7f74c0009040 comp rx=0 tx=0).stop 2024-03-30T01:41:40.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d406d040 msgr2=0x7f74d4146050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d406d040 0x7f74d4146050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009c10 tx=0x7f74c8004750 comp rx=0 tx=0).stop 2024-03-30T01:41:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 shutdown_connections 2024-03-30T01:41:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74d406d040 0x7f74d4146050 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f74b806ac90 0x7f74b806d150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74d41549a0 0x7f74d40c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 --2- 172.21.15.31:0/2692318910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74d406d900 0x7f74d4146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 >> 172.21.15.31:0/2692318910 conn(0x7f74d40c3980 msgr2=0x7f74d414c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 shutdown_connections 2024-03-30T01:41:40.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.330+0000 7f74db125700 1 -- 172.21.15.31:0/2692318910 wait complete. 2024-03-30T01:41:40.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:40.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.479+0000 7f3e601d6700 1 Processor -- start 2024-03-30T01:41:40.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.480+0000 7f3e601d6700 1 -- start start 2024-03-30T01:41:40.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.480+0000 7f3e601d6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e5814fd30 0x7f3e58156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.480+0000 7f3e601d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e580c47c0 0x7f3e580ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.480+0000 7f3e601d6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e580c4d60 con 0x7f3e58150110 2024-03-30T01:41:40.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.480+0000 7f3e601d6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e580c5040 con 0x7f3e5814fd30 2024-03-30T01:41:40.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.480+0000 7f3e601d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580c5320 con 0x7f3e580c47c0 2024-03-30T01:41:40.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5df72700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e5814fd30 0x7f3e58156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5e773700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e580c47c0 0x7f3e580ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5d771700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3e58150110 0x7f3e580c8380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:47188/0 (socket says 172.21.15.31:47188) 2024-03-30T01:41:40.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5d771700 1 -- 172.21.15.31:0/2453492767 learned_addr learned my addr 172.21.15.31:0/2453492767 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:40.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5cf70700 1 -- 172.21.15.31:0/2453492767 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672787491 0 0) 0x7f3e580c4d60 con 0x7f3e58150110 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5cf70700 1 -- 172.21.15.31:0/2453492767 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e440035b0 con 0x7f3e58150110 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5df72700 1 -- 172.21.15.31:0/2453492767 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e580c47c0 msgr2=0x7f3e580ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5df72700 1 --2- 172.21.15.31:0/2453492767 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e580c47c0 0x7f3e580ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5df72700 1 -- 172.21.15.31:0/2453492767 >> v1:172.21.15.59:6789/0 conn(0x7f3e58150110 legacy=0x7f3e580c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5df72700 1 -- 172.21.15.31:0/2453492767 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e580c5600 con 0x7f3e5814fd30 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.481+0000 7f3e5e773700 1 --2- 172.21.15.31:0/2453492767 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e580c47c0 0x7f3e580ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e5df72700 1 --2- 172.21.15.31:0/2453492767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5814fd30 0x7f3e58156800 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009840 tx=0x7f3e54009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d4a86b01a78a6d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e5cf70700 1 -- 172.21.15.31:0/2453492767 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54011830 con 0x7f3e5814fd30 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e5cf70700 1 -- 172.21.15.31:0/2453492767 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e54011990 con 0x7f3e5814fd30 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e5cf70700 1 -- 172.21.15.31:0/2453492767 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54011d20 con 0x7f3e5814fd30 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e601d6700 1 -- 172.21.15.31:0/2453492767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5814fd30 msgr2=0x7f3e58156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e601d6700 1 --2- 172.21.15.31:0/2453492767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5814fd30 0x7f3e58156800 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009840 tx=0x7f3e54009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e601d6700 1 -- 172.21.15.31:0/2453492767 shutdown_connections 2024-03-30T01:41:40.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e601d6700 1 --2- 172.21.15.31:0/2453492767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e5814fd30 0x7f3e58156800 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e601d6700 1 --2- 172.21.15.31:0/2453492767 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e580c47c0 0x7f3e580ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e601d6700 1 -- 172.21.15.31:0/2453492767 >> 172.21.15.31:0/2453492767 conn(0x7f3e5806ab20 msgr2=0x7f3e5806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e601d6700 1 -- 172.21.15.31:0/2453492767 shutdown_connections 2024-03-30T01:41:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.482+0000 7f3e601d6700 1 -- 172.21.15.31:0/2453492767 wait complete. 2024-03-30T01:41:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.484+0000 7f3e601d6700 1 Processor -- start 2024-03-30T01:41:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.484+0000 7f3e601d6700 1 -- start start 2024-03-30T01:41:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.484+0000 7f3e601d6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e580c47c0 0x7f3e581ed500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.484+0000 7f3e601d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e5814fd30 0x7f3e581e7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.484+0000 7f3e601d6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e58150110 0x7f3e581e8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.484+0000 7f3e601d6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e581dbc90 con 0x7f3e58150110 2024-03-30T01:41:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.484+0000 7f3e601d6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e581edf80 con 0x7f3e580c47c0 2024-03-30T01:41:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.484+0000 7f3e601d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e58154660 con 0x7f3e5814fd30 2024-03-30T01:41:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.484+0000 7f3e5e773700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e58150110 0x7f3e581e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5e773700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e58150110 0x7f3e581e8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46466/0 (socket says 172.21.15.31:46466) 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5d771700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e5814fd30 0x7f3e581e7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5e773700 1 -- 172.21.15.31:0/383511923 learned_addr learned my addr 172.21.15.31:0/383511923 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5df72700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e580c47c0 0x7f3e581ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5e773700 1 -- 172.21.15.31:0/383511923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e5814fd30 msgr2=0x7f3e581e7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5e773700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e5814fd30 0x7f3e581e7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5e773700 1 -- 172.21.15.31:0/383511923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e580c47c0 msgr2=0x7f3e581ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5e773700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e580c47c0 0x7f3e581ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5e773700 1 -- 172.21.15.31:0/383511923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50009a70 con 0x7f3e58150110 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e5e773700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e58150110 0x7f3e581e8240 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000efc0 tx=0x7f3e5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e4e7fc700 1 -- 172.21.15.31:0/383511923 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000ca90 con 0x7f3e58150110 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e4e7fc700 1 -- 172.21.15.31:0/383511923 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5000f040 con 0x7f3e58150110 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e540094a0 con 0x7f3e58150110 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e4e7fc700 1 -- 172.21.15.31:0/383511923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000ce10 con 0x7f3e58150110 2024-03-30T01:41:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.485+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e58154c20 con 0x7f3e58150110 2024-03-30T01:41:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.486+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e58073760 con 0x7f3e58150110 2024-03-30T01:41:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.487+0000 7f3e4e7fc700 1 -- 172.21.15.31:0/383511923 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3e50003a20 con 0x7f3e58150110 2024-03-30T01:41:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.488+0000 7f3e4e7fc700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e3c06ac90 0x7f3e3c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.488+0000 7f3e4e7fc700 1 -- 172.21.15.31:0/383511923 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3e5008a710 con 0x7f3e58150110 2024-03-30T01:41:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.488+0000 7f3e5df72700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e3c06ac90 0x7f3e3c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.488+0000 7f3e5df72700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e3c06ac90 0x7f3e3c06d150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e54000c00 tx=0x7f3e54008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.492+0000 7f3e4e7fc700 1 -- 172.21.15.31:0/383511923 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3e50055670 con 0x7f3e58150110 2024-03-30T01:41:40.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.712+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e58154e40 con 0x7f3e58150110 2024-03-30T01:41:40.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.713+0000 7f3e4e7fc700 1 -- 172.21.15.31:0/383511923 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f3e50009d70 con 0x7f3e58150110 2024-03-30T01:41:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:40.719 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:03.229400+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:40.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e3c06ac90 msgr2=0x7f3e3c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e3c06ac90 0x7f3e3c06d150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e54000c00 tx=0x7f3e54008030 comp rx=0 tx=0).stop 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e58150110 msgr2=0x7f3e581e8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e58150110 0x7f3e581e8240 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000efc0 tx=0x7f3e5000c3a0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 shutdown_connections 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e58150110 0x7f3e581e8240 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3e3c06ac90 0x7f3e3c06d150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e580c47c0 0x7f3e581ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 --2- 172.21.15.31:0/383511923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e5814fd30 0x7f3e581e7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 >> 172.21.15.31:0/383511923 conn(0x7f3e5806ab20 msgr2=0x7f3e580c8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 shutdown_connections 2024-03-30T01:41:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.716+0000 7f3e601d6700 1 -- 172.21.15.31:0/383511923 wait complete. 2024-03-30T01:41:40.735 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:40.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.869+0000 7fc753d78700 1 Processor -- start 2024-03-30T01:41:40.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.870+0000 7fc753d78700 1 -- start start 2024-03-30T01:41:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.870+0000 7fc753d78700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc74c0c46c0 0x7fc74c06ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc753d78700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74c070610 con 0x7fc74c0c5d10 2024-03-30T01:41:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc753d78700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74c0708f0 con 0x7fc74c0c4aa0 2024-03-30T01:41:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc753d78700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c0c3210 con 0x7fc74c0c46c0 2024-03-30T01:41:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc751313700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc74c0c5d10 0x7fc74c0c26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53048/0 (socket says 172.21.15.31:53048) 2024-03-30T01:41:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc751313700 1 -- 172.21.15.31:0/478959949 learned_addr learned my addr 172.21.15.31:0/478959949 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc751b14700 1 --2- 172.21.15.31:0/478959949 >> v2:172.21.15.59:3300/0 conn(0x7fc74c0c46c0 0x7fc74c06ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc750b12700 1 -- 172.21.15.31:0/478959949 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775046679 0 0) 0x7fc74c070610 con 0x7fc74c0c5d10 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc750b12700 1 -- 172.21.15.31:0/478959949 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7380035b0 con 0x7fc74c0c5d10 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc750b12700 1 -- 172.21.15.31:0/478959949 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913395961 0 0) 0x7fc74c0708f0 con 0x7fc74c0c4aa0 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc750b12700 1 -- 172.21.15.31:0/478959949 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc74c070610 con 0x7fc74c0c4aa0 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc751b14700 1 -- 172.21.15.31:0/478959949 >> v1:172.21.15.59:6789/0 conn(0x7fc74c0c4aa0 legacy=0x7fc74c15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc751b14700 1 -- 172.21.15.31:0/478959949 >> v1:172.21.15.31:6789/0 conn(0x7fc74c0c5d10 legacy=0x7fc74c0c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.871+0000 7fc751b14700 1 -- 172.21.15.31:0/478959949 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c0c34f0 con 0x7fc74c0c46c0 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc751b14700 1 --2- 172.21.15.31:0/478959949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c46c0 0x7fc74c06ce80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc748009840 tx=0x7fc748009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c570db59d2f2bde server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc750b12700 1 -- 172.21.15.31:0/478959949 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748011830 con 0x7fc74c0c46c0 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc750b12700 1 -- 172.21.15.31:0/478959949 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748011990 con 0x7fc74c0c46c0 2024-03-30T01:41:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc750b12700 1 -- 172.21.15.31:0/478959949 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748011d20 con 0x7fc74c0c46c0 2024-03-30T01:41:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc753d78700 1 -- 172.21.15.31:0/478959949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c46c0 msgr2=0x7fc74c06ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc753d78700 1 --2- 172.21.15.31:0/478959949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c46c0 0x7fc74c06ce80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc748009840 tx=0x7fc748009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc753d78700 1 -- 172.21.15.31:0/478959949 shutdown_connections 2024-03-30T01:41:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc753d78700 1 --2- 172.21.15.31:0/478959949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c46c0 0x7fc74c06ce80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc753d78700 1 -- 172.21.15.31:0/478959949 >> 172.21.15.31:0/478959949 conn(0x7fc74c0c7cf0 msgr2=0x7fc74c0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:40.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc753d78700 1 -- 172.21.15.31:0/478959949 shutdown_connections 2024-03-30T01:41:40.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.872+0000 7fc753d78700 1 -- 172.21.15.31:0/478959949 wait complete. 2024-03-30T01:41:40.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.874+0000 7fc753d78700 1 Processor -- start 2024-03-30T01:41:40.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.874+0000 7fc753d78700 1 -- start start 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.874+0000 7fc753d78700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc74c0c46c0 0x7fc74c1ea980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.874+0000 7fc753d78700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c4aa0 0x7fc74c1d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.874+0000 7fc753d78700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc74c0c5d10 0x7fc74c1d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.874+0000 7fc753d78700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0a6a80 con 0x7fc74c0c5d10 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.874+0000 7fc753d78700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c1eaec0 con 0x7fc74c0c4aa0 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.874+0000 7fc753d78700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0a61e0 con 0x7fc74c0c46c0 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc752315700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc74c0c5d10 0x7fc74c1d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc752315700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc74c0c5d10 0x7fc74c1d6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46480/0 (socket says 172.21.15.31:46480) 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc752315700 1 -- 172.21.15.31:0/57141243 learned_addr learned my addr 172.21.15.31:0/57141243 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc751313700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c4aa0 0x7fc74c1d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc751b14700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc74c0c46c0 0x7fc74c1ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc752315700 1 -- 172.21.15.31:0/57141243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc74c0c46c0 msgr2=0x7fc74c1ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc752315700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc74c0c46c0 0x7fc74c1ea980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc752315700 1 -- 172.21.15.31:0/57141243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c4aa0 msgr2=0x7fc74c1d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc752315700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c4aa0 0x7fc74c1d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc752315700 1 -- 172.21.15.31:0/57141243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480094a0 con 0x7fc74c0c5d10 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc751313700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c4aa0 0x7fc74c1d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc752315700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc74c0c5d10 0x7fc74c1d6c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc74400cc20 tx=0x7fc74400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc7427fc700 1 -- 172.21.15.31:0/57141243 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400ac20 con 0x7fc74c0c5d10 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc7427fc700 1 -- 172.21.15.31:0/57141243 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74400ad80 con 0x7fc74c0c5d10 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74c0a64c0 con 0x7fc74c0c5d10 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc7427fc700 1 -- 172.21.15.31:0/57141243 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744003c20 con 0x7fc74c0c5d10 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.875+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c0a8c00 con 0x7fc74c0c5d10 2024-03-30T01:41:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.876+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c0a8a60 con 0x7fc74c0c5d10 2024-03-30T01:41:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.877+0000 7fc7427fc700 1 -- 172.21.15.31:0/57141243 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc7440050d0 con 0x7fc74c0c5d10 2024-03-30T01:41:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.877+0000 7fc7427fc700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc73006ac90 0x7fc73006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.877+0000 7fc7427fc700 1 -- 172.21.15.31:0/57141243 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc744089710 con 0x7fc74c0c5d10 2024-03-30T01:41:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.878+0000 7fc751b14700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc73006ac90 0x7fc73006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:40.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.878+0000 7fc751b14700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc73006ac90 0x7fc73006d150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc748009fd0 tx=0x7fc74801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:40.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:40.882+0000 7fc7427fc700 1 -- 172.21.15.31:0/57141243 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc74c0a8a60 con 0x7fc74c0c5d10 2024-03-30T01:41:41.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.095+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc74c0a8a60 con 0x7fc74c0c5d10 2024-03-30T01:41:41.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.096+0000 7fc7427fc700 1 -- 172.21.15.31:0/57141243 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc74c0a8a60 con 0x7fc74c0c5d10 2024-03-30T01:41:41.101 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc73006ac90 msgr2=0x7fc73006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc73006ac90 0x7fc73006d150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc748009fd0 tx=0x7fc74801c040 comp rx=0 tx=0).stop 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc74c0c5d10 msgr2=0x7fc74c1d6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc74c0c5d10 0x7fc74c1d6c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc74400cc20 tx=0x7fc74400a060 comp rx=0 tx=0).stop 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 shutdown_connections 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc74c0c5d10 0x7fc74c1d6c50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc73006ac90 0x7fc73006d150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc74c0c46c0 0x7fc74c1ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 --2- 172.21.15.31:0/57141243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc74c0c4aa0 0x7fc74c1d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 >> 172.21.15.31:0/57141243 conn(0x7fc74c0c7cf0 msgr2=0x7fc74c0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 shutdown_connections 2024-03-30T01:41:41.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.098+0000 7fc753d78700 1 -- 172.21.15.31:0/57141243 wait complete. 2024-03-30T01:41:41.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.251+0000 7f4d4517e700 1 Processor -- start 2024-03-30T01:41:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.251+0000 7f4d4517e700 1 -- start start 2024-03-30T01:41:41.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d4517e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4d401549a0 0x7f4d40072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d4517e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d4006d040 0x7f4d40154d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d4517e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4014ee60 con 0x7f4d4006d900 2024-03-30T01:41:41.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d4517e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d400c2290 con 0x7f4d401549a0 2024-03-30T01:41:41.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d4517e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40155720 con 0x7f4d4006d040 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3f59e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4d4006d900 0x7f4d401513b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53064/0 (socket says 172.21.15.31:53064) 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3f59e700 1 -- 172.21.15.31:0/3910316430 learned_addr learned my addr 172.21.15.31:0/3910316430 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612735002 0 0) 0x7f4d4014ee60 con 0x7f4d4006d900 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3ed9d700 1 --2- 172.21.15.31:0/3910316430 >> v2:172.21.15.59:3300/0 conn(0x7f4d401549a0 0x7f4d40072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d280035b0 con 0x7f4d4006d900 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3e59c700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d4006d040 0x7f4d40154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 599438056 0 0) 0x7f4d280035b0 con 0x7f4d4006d900 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d4006d040 msgr2=0x7f4d40154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3dd9b700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d4006d040 0x7f4d40154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 >> v2:172.21.15.59:3300/0 conn(0x7f4d401549a0 msgr2=0x7f4d40072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:41.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3dd9b700 1 --2- 172.21.15.31:0/3910316430 >> v2:172.21.15.59:3300/0 conn(0x7f4d401549a0 0x7f4d40072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d400c24e0 con 0x7f4d4006d900 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.252+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f4d380019f0 con 0x7f4d4006d900 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 >> v1:172.21.15.31:6789/0 conn(0x7f4d4006d900 legacy=0x7f4d401513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d4006d040 0x7f4d28003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28004060 0x7f4d28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28004d90 0x7f4d28005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400c2290 con 0x7f4d28004d90 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40155720 con 0x7f4d28004060 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2800bb40 con 0x7f4d4006d040 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3f59e700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28004d90 0x7f4d28005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3ed9d700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d4006d040 0x7f4d28003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3e59c700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28004060 0x7f4d28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3f59e700 1 -- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d4006d040 msgr2=0x7f4d28003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:41.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3f59e700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d4006d040 0x7f4d28003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3f59e700 1 -- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28004060 msgr2=0x7f4d28004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3f59e700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28004060 0x7f4d28004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3f59e700 1 -- 172.21.15.31:0/3910316430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280035b0 con 0x7f4d28004d90 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3f59e700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28004d90 0x7f4d28005210 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800a8d0 tx=0x7f4d28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfcf6183d11f63 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800a340 con 0x7f4d28004d90 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d38012600 con 0x7f4d28004d90 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.253+0000 7f4d3dd9b700 1 -- 172.21.15.31:0/3910316430 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380129b0 con 0x7f4d28004d90 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.254+0000 7f4d4517e700 1 -- 172.21.15.31:0/3910316430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28004d90 msgr2=0x7f4d28005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.254+0000 7f4d4517e700 1 --2- 172.21.15.31:0/3910316430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d28004d90 0x7f4d28005210 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800a8d0 tx=0x7f4d28005880 comp rx=0 tx=0).stop 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.254+0000 7f4d3f59e700 1 -- 172.21.15.31:0/3910316430 reap_dead start 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.254+0000 7f4d4517e700 1 -- 172.21.15.31:0/3910316430 shutdown_connections 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.254+0000 7f4d4517e700 1 -- 172.21.15.31:0/3910316430 >> 172.21.15.31:0/3910316430 conn(0x7f4d400c3980 msgr2=0x7f4d400c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.254+0000 7f4d4517e700 1 -- 172.21.15.31:0/3910316430 shutdown_connections 2024-03-30T01:41:41.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.254+0000 7f4d4517e700 1 -- 172.21.15.31:0/3910316430 wait complete. 2024-03-30T01:41:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.256+0000 7f4d4517e700 1 Processor -- start 2024-03-30T01:41:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.256+0000 7f4d4517e700 1 -- start start 2024-03-30T01:41:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.256+0000 7f4d4517e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4014b320 0x7f4d400a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.256+0000 7f4d4517e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d400ab990 0x7f4d400a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.256+0000 7f4d4517e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d40143bb0 0x7f4d40144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.256+0000 7f4d4517e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d401487d0 con 0x7f4d400ab990 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.256+0000 7f4d4517e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40144d30 con 0x7f4d4014b320 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.256+0000 7f4d4517e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40145c20 con 0x7f4d40143bb0 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.256+0000 7f4d3e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d400ab990 0x7f4d400a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d400ab990 0x7f4d400a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46506/0 (socket says 172.21.15.31:46506) 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3e59c700 1 -- 172.21.15.31:0/3667614584 learned_addr learned my addr 172.21.15.31:0/3667614584 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3f59e700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d40143bb0 0x7f4d40144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3ed9d700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4014b320 0x7f4d400a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3e59c700 1 -- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d40143bb0 msgr2=0x7f4d40144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3e59c700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d40143bb0 0x7f4d40144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3e59c700 1 -- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4014b320 msgr2=0x7f4d400a92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3e59c700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4014b320 0x7f4d400a92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3e59c700 1 -- 172.21.15.31:0/3667614584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40145f00 con 0x7f4d400ab990 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3f59e700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d40143bb0 0x7f4d40144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3ed9d700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4014b320 0x7f4d400a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d3e59c700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d400ab990 0x7f4d400a9800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d30011940 tx=0x7f4d30011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d277fe700 1 -- 172.21.15.31:0/3667614584 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30004240 con 0x7f4d400ab990 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d277fe700 1 -- 172.21.15.31:0/3667614584 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d300043a0 con 0x7f4d400ab990 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d401461e0 con 0x7f4d400ab990 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d277fe700 1 -- 172.21.15.31:0/3667614584 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000f950 con 0x7f4d400ab990 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.257+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d40144570 con 0x7f4d400ab990 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.258+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d400747d0 con 0x7f4d400ab990 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.259+0000 7f4d277fe700 1 -- 172.21.15.31:0/3667614584 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4d3000fab0 con 0x7f4d400ab990 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.259+0000 7f4d277fe700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4d2c06ac90 0x7f4d2c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.259+0000 7f4d277fe700 1 -- 172.21.15.31:0/3667614584 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4d30089c40 con 0x7f4d400ab990 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.259+0000 7f4d3ed9d700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4d2c06ac90 0x7f4d2c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.260+0000 7f4d3ed9d700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4d2c06ac90 0x7f4d2c06d150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009bd0 tx=0x7f4d34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.264+0000 7f4d277fe700 1 -- 172.21.15.31:0/3667614584 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4d30054a90 con 0x7f4d400ab990 2024-03-30T01:41:41.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.477+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d40154780 con 0x7f4d400ab990 2024-03-30T01:41:41.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.478+0000 7f4d277fe700 1 -- 172.21.15.31:0/3667614584 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d30023020 con 0x7f4d400ab990 2024-03-30T01:41:41.483 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4d2c06ac90 msgr2=0x7f4d2c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4d2c06ac90 0x7f4d2c06d150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009bd0 tx=0x7f4d34008040 comp rx=0 tx=0).stop 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d400ab990 msgr2=0x7f4d400a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d400ab990 0x7f4d400a9800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d30011940 tx=0x7f4d30011d00 comp rx=0 tx=0).stop 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 shutdown_connections 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4d2c06ac90 0x7f4d2c06d150 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d40143bb0 0x7f4d40144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d4014b320 0x7f4d400a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 --2- 172.21.15.31:0/3667614584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d400ab990 0x7f4d400a9800 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 >> 172.21.15.31:0/3667614584 conn(0x7f4d400c3980 msgr2=0x7f4d400c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:41.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 shutdown_connections 2024-03-30T01:41:41.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.480+0000 7f4d4517e700 1 -- 172.21.15.31:0/3667614584 wait complete. 2024-03-30T01:41:41.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.633+0000 7f5ac1d30700 1 Processor -- start 2024-03-30T01:41:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.633+0000 7f5ac1d30700 1 -- start start 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5ac1d30700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5abc151c90 0x7f5abc06a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5ac1d30700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5abc06d340 0x7f5abc154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5ac1d30700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc152070 con 0x7f5abc0c1250 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5ac1d30700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0acaa0 con 0x7f5abc06d340 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5ac1d30700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0c3790 con 0x7f5abc151c90 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5abbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5abc06d340 0x7f5abc154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5abbfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5abc06d340 0x7f5abc154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46522/0 (socket says 172.21.15.31:46522) 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5abbfff700 1 -- 172.21.15.31:0/3373313448 learned_addr learned my addr 172.21.15.31:0/3373313448 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5abb7fe700 1 --2- 172.21.15.31:0/3373313448 >> v2:172.21.15.59:3300/0 conn(0x7f5abc151c90 0x7f5abc06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5abbfff700 1 -- 172.21.15.31:0/3373313448 >> v2:172.21.15.59:3300/0 conn(0x7f5abc151c90 msgr2=0x7f5abc06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5abbfff700 1 --2- 172.21.15.31:0/3373313448 >> v2:172.21.15.59:3300/0 conn(0x7f5abc151c90 0x7f5abc06a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5abbfff700 1 -- 172.21.15.31:0/3373313448 >> v1:172.21.15.59:6789/0 conn(0x7f5abc0c1250 legacy=0x7f5abc154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5abbfff700 1 -- 172.21.15.31:0/3373313448 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0acef0 con 0x7f5abc06d340 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5abbfff700 1 --2- 172.21.15.31:0/3373313448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 0x7f5abc154d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0c7eb0 tx=0x7f5ab400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ae5c6cb3b3dbf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.634+0000 7f5aba7fc700 1 -- 172.21.15.31:0/3373313448 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400e070 con 0x7f5abc06d340 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5aba7fc700 1 -- 172.21.15.31:0/3373313448 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab400b9c0 con 0x7f5abc06d340 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5aba7fc700 1 -- 172.21.15.31:0/3373313448 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400bd70 con 0x7f5abc06d340 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3373313448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 msgr2=0x7f5abc154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5ac1d30700 1 --2- 172.21.15.31:0/3373313448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 0x7f5abc154d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0c7eb0 tx=0x7f5ab400b330 comp rx=0 tx=0).stop 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3373313448 shutdown_connections 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5ac1d30700 1 --2- 172.21.15.31:0/3373313448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 0x7f5abc154d90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5ac1d30700 1 --2- 172.21.15.31:0/3373313448 >> v2:172.21.15.59:3300/0 conn(0x7f5abc151c90 0x7f5abc06a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3373313448 >> 172.21.15.31:0/3373313448 conn(0x7f5abc0c3910 msgr2=0x7f5abc0c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3373313448 shutdown_connections 2024-03-30T01:41:41.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.635+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3373313448 wait complete. 2024-03-30T01:41:41.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.636+0000 7f5ac1d30700 1 Processor -- start 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.637+0000 7f5ac1d30700 1 -- start start 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.637+0000 7f5ac1d30700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 0x7f5abc1ea400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.637+0000 7f5ac1d30700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5abc0c1250 0x7f5abc0c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.637+0000 7f5ac1d30700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5abc151c90 0x7f5abc14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.637+0000 7f5ac1d30700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc157ce0 con 0x7f5abc06d340 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.637+0000 7f5ac1d30700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc1ea940 con 0x7f5abc151c90 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.637+0000 7f5ac1d30700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc158460 con 0x7f5abc0c1250 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 0x7f5abc1ea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 0x7f5abc1ea400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46530/0 (socket says 172.21.15.31:46530) 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abb7fe700 1 -- 172.21.15.31:0/3690358685 learned_addr learned my addr 172.21.15.31:0/3690358685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abbfff700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5abc151c90 0x7f5abc14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abaffd700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5abc0c1250 0x7f5abc0c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abb7fe700 1 -- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5abc0c1250 msgr2=0x7f5abc0c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abb7fe700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5abc0c1250 0x7f5abc0c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abb7fe700 1 -- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5abc151c90 msgr2=0x7f5abc14b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abb7fe700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5abc151c90 0x7f5abc14b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abb7fe700 1 -- 172.21.15.31:0/3690358685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab4009b30 con 0x7f5abc06d340 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abbfff700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5abc151c90 0x7f5abc14b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abaffd700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5abc0c1250 0x7f5abc0c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5abb7fe700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 0x7f5abc1ea400 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400e810 tx=0x7f5aa400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5ab3fff700 1 -- 172.21.15.31:0/3690358685 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4013610 con 0x7f5abc06d340 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5ab3fff700 1 -- 172.21.15.31:0/3690358685 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa4013770 con 0x7f5abc06d340 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc158720 con 0x7f5abc06d340 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5ab3fff700 1 -- 172.21.15.31:0/3690358685 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4013b20 con 0x7f5abc06d340 2024-03-30T01:41:41.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.638+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc157690 con 0x7f5abc06d340 2024-03-30T01:41:41.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.639+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc155a50 con 0x7f5abc06d340 2024-03-30T01:41:41.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.640+0000 7f5ab3fff700 1 -- 172.21.15.31:0/3690358685 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5aa4013c80 con 0x7f5abc06d340 2024-03-30T01:41:41.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.640+0000 7f5ab3fff700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a9c06ac90 0x7f5a9c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:41.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.640+0000 7f5ab3fff700 1 -- 172.21.15.31:0/3690358685 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5aa4089330 con 0x7f5abc06d340 2024-03-30T01:41:41.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.640+0000 7f5abaffd700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a9c06ac90 0x7f5a9c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:41.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.641+0000 7f5abaffd700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a9c06ac90 0x7f5a9c06d150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0c00a0 tx=0x7f5aac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.645+0000 7f5ab3fff700 1 -- 172.21.15.31:0/3690358685 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5aa4054230 con 0x7f5abc06d340 2024-03-30T01:41:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.861+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5abc066f90 con 0x7f5abc06d340 2024-03-30T01:41:41.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.862+0000 7f5ab3fff700 1 -- 172.21.15.31:0/3690358685 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5aa4057850 con 0x7f5abc06d340 2024-03-30T01:41:41.867 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a9c06ac90 msgr2=0x7f5a9c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a9c06ac90 0x7f5a9c06d150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0c00a0 tx=0x7f5aac009040 comp rx=0 tx=0).stop 2024-03-30T01:41:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 msgr2=0x7f5abc1ea400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 0x7f5abc1ea400 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400e810 tx=0x7f5aa400ebd0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 shutdown_connections 2024-03-30T01:41:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5abc06d340 0x7f5abc1ea400 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5a9c06ac90 0x7f5a9c06d150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5abc151c90 0x7f5abc14b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 --2- 172.21.15.31:0/3690358685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5abc0c1250 0x7f5abc0c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 >> 172.21.15.31:0/3690358685 conn(0x7f5abc0c3910 msgr2=0x7f5abc150f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 shutdown_connections 2024-03-30T01:41:41.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:41.864+0000 7f5ac1d30700 1 -- 172.21.15.31:0/3690358685 wait complete. 2024-03-30T01:41:41.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.023+0000 7f9794bae700 1 Processor -- start 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.023+0000 7f9794bae700 1 -- start start 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.024+0000 7f9794bae700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97900721f0 0x7f979006a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.024+0000 7f9794bae700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9790155490 0x7f979006ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.024+0000 7f9794bae700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9790155000 con 0x7f9790151ea0 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.024+0000 7f9794bae700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979006e930 con 0x7f9790155490 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.024+0000 7f9794bae700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979006df40 con 0x7f97900721f0 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.024+0000 7f978e59c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9790151ea0 0x7f97900c02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53080/0 (socket says 172.21.15.31:53080) 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978e59c700 1 -- 172.21.15.31:0/1111735803 learned_addr learned my addr 172.21.15.31:0/1111735803 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978ed9d700 1 --2- 172.21.15.31:0/1111735803 >> v2:172.21.15.59:3300/0 conn(0x7f9790155490 0x7f979006ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978dd9b700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97900721f0 0x7f979006a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426653204 0 0) 0x7f9790155000 con 0x7f9790151ea0 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97780035b0 con 0x7f9790151ea0 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2730768144 0 0) 0x7f97780035b0 con 0x7f9790151ea0 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97900721f0 msgr2=0x7f979006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97900721f0 0x7f979006a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 >> v2:172.21.15.59:3300/0 conn(0x7f9790155490 msgr2=0x7f979006ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 --2- 172.21.15.31:0/1111735803 >> v2:172.21.15.59:3300/0 conn(0x7f9790155490 0x7f979006ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979006e650 con 0x7f9790151ea0 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f9780001670 con 0x7f9790151ea0 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 >> v1:172.21.15.31:6789/0 conn(0x7f9790151ea0 legacy=0x7f97900c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9778003ae0 0x7f9778003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97900721f0 0x7f9778004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9778004d90 0x7f9778005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979006e930 con 0x7f9778004d90 2024-03-30T01:41:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977800ba60 con 0x7f9778003ae0 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977800bc20 con 0x7f97900721f0 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978ed9d700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9778004d90 0x7f9778005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978e59c700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9778003ae0 0x7f9778003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.025+0000 7f978dd9b700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97900721f0 0x7f9778004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.026+0000 7f978ed9d700 1 -- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97900721f0 msgr2=0x7f9778004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.026+0000 7f978ed9d700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97900721f0 0x7f9778004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.026+0000 7f978ed9d700 1 -- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9778003ae0 msgr2=0x7f9778003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.026+0000 7f978ed9d700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9778003ae0 0x7f9778003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.026+0000 7f978ed9d700 1 -- 172.21.15.31:0/1111735803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780035b0 con 0x7f9778004d90 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.026+0000 7f978ed9d700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9778004d90 0x7f9778005210 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9788009fc0 tx=0x7f9778005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208bebb0820e1734 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.026+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978800ec30 con 0x7f9778004d90 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.026+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978800f040 con 0x7f9778004d90 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.026+0000 7f978d59a700 1 -- 172.21.15.31:0/1111735803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788014480 con 0x7f9778004d90 2024-03-30T01:41:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.027+0000 7f9794bae700 1 -- 172.21.15.31:0/1111735803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9778004d90 msgr2=0x7f9778005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.027+0000 7f9794bae700 1 --2- 172.21.15.31:0/1111735803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9778004d90 0x7f9778005210 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9788009fc0 tx=0x7f9778005880 comp rx=0 tx=0).stop 2024-03-30T01:41:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.027+0000 7f978ed9d700 1 -- 172.21.15.31:0/1111735803 reap_dead start 2024-03-30T01:41:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.027+0000 7f9794bae700 1 -- 172.21.15.31:0/1111735803 shutdown_connections 2024-03-30T01:41:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.027+0000 7f9794bae700 1 -- 172.21.15.31:0/1111735803 >> 172.21.15.31:0/1111735803 conn(0x7f97900c3980 msgr2=0x7f97900c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.027+0000 7f9794bae700 1 -- 172.21.15.31:0/1111735803 shutdown_connections 2024-03-30T01:41:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.027+0000 7f9794bae700 1 -- 172.21.15.31:0/1111735803 wait complete. 2024-03-30T01:41:42.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.028+0000 7f9794bae700 1 Processor -- start 2024-03-30T01:41:42.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.028+0000 7f9794bae700 1 -- start start 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f9794bae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9778004d90 0x7f9790152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f9794bae700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97901537d0 0x7f9790144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f9794bae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790148f00 0x7f9790149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f9794bae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9790158400 con 0x7f9790148f00 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f9794bae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97901563d0 con 0x7f9778004d90 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f9794bae700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97901435f0 con 0x7f97901537d0 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790148f00 0x7f9790149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790148f00 0x7f9790149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46552/0 (socket says 172.21.15.31:46552) 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978ed9d700 1 -- 172.21.15.31:0/236992738 learned_addr learned my addr 172.21.15.31:0/236992738 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978e59c700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9778004d90 0x7f9790152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978dd9b700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97901537d0 0x7f9790144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978ed9d700 1 -- 172.21.15.31:0/236992738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97901537d0 msgr2=0x7f9790144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978ed9d700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97901537d0 0x7f9790144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978ed9d700 1 -- 172.21.15.31:0/236992738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9778004d90 msgr2=0x7f9790152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978ed9d700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9778004d90 0x7f9790152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.029+0000 7f978ed9d700 1 -- 172.21.15.31:0/236992738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97901560a0 con 0x7f9790148f00 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.030+0000 7f978ed9d700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790148f00 0x7f9790149380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9788016000 tx=0x7f9788009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.030+0000 7f977effd700 1 -- 172.21.15.31:0/236992738 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788007000 con 0x7f9790148f00 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.030+0000 7f977effd700 1 -- 172.21.15.31:0/236992738 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9788014d20 con 0x7f9790148f00 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.030+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9790152e40 con 0x7f9790148f00 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.030+0000 7f977effd700 1 -- 172.21.15.31:0/236992738 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788027680 con 0x7f9790148f00 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.030+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9790153400 con 0x7f9790148f00 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.031+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9790156510 con 0x7f9790148f00 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.031+0000 7f977effd700 1 -- 172.21.15.31:0/236992738 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f978801d420 con 0x7f9790148f00 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.032+0000 7f977effd700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f977006ac90 0x7f977006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.032+0000 7f977effd700 1 -- 172.21.15.31:0/236992738 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f978808a7a0 con 0x7f9790148f00 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.032+0000 7f978e59c700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f977006ac90 0x7f977006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.032+0000 7f978e59c700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f977006ac90 0x7f977006d150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9780000c00 tx=0x7f9780007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:42.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.036+0000 7f977effd700 1 -- 172.21.15.31:0/236992738 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9790156510 con 0x7f9790148f00 2024-03-30T01:41:42.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.251+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9790156510 con 0x7f9790148f00 2024-03-30T01:41:42.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.251+0000 7f977effd700 1 -- 172.21.15.31:0/236992738 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9790156510 con 0x7f9790148f00 2024-03-30T01:41:42.257 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:42.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f977006ac90 msgr2=0x7f977006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f977006ac90 0x7f977006d150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9780000c00 tx=0x7f9780007980 comp rx=0 tx=0).stop 2024-03-30T01:41:42.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790148f00 msgr2=0x7f9790149380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790148f00 0x7f9790149380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9788016000 tx=0x7f9788009750 comp rx=0 tx=0).stop 2024-03-30T01:41:42.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 shutdown_connections 2024-03-30T01:41:42.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790148f00 0x7f9790149380 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f977006ac90 0x7f977006d150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9778004d90 0x7f9790152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 --2- 172.21.15.31:0/236992738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f97901537d0 0x7f9790144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.253+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 >> 172.21.15.31:0/236992738 conn(0x7f97900c3980 msgr2=0x7f97900c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:42.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.254+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 shutdown_connections 2024-03-30T01:41:42.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.254+0000 7f9794bae700 1 -- 172.21.15.31:0/236992738 wait complete. 2024-03-30T01:41:42.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:42.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.403+0000 7fa68f92f700 1 Processor -- start 2024-03-30T01:41:42.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.403+0000 7fa68f92f700 1 -- start start 2024-03-30T01:41:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68f92f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa68806e6a0 0x7fa688156270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68f92f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa688153120 con 0x7fa68806b930 2024-03-30T01:41:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68f92f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa688152570 con 0x7fa68814c4a0 2024-03-30T01:41:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68f92f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6881567b0 con 0x7fa68806e6a0 2024-03-30T01:41:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68decc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa68806e6a0 0x7fa688156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68ceca700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa68806b930 0x7fa688155b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53086/0 (socket says 172.21.15.31:53086) 2024-03-30T01:41:42.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68ceca700 1 -- 172.21.15.31:0/1511198507 learned_addr learned my addr 172.21.15.31:0/1511198507 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68decc700 1 -- 172.21.15.31:0/1511198507 >> v1:172.21.15.59:6789/0 conn(0x7fa68814c4a0 legacy=0x7fa68806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68decc700 1 -- 172.21.15.31:0/1511198507 >> v1:172.21.15.31:6789/0 conn(0x7fa68806b930 legacy=0x7fa688155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa67ffff700 1 -- 172.21.15.31:0/1511198507 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811913869 0 0) 0x7fa688153120 con 0x7fa68806b930 2024-03-30T01:41:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68decc700 1 -- 172.21.15.31:0/1511198507 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6881527f0 con 0x7fa68806e6a0 2024-03-30T01:41:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.404+0000 7fa68decc700 1 --2- 172.21.15.31:0/1511198507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806e6a0 0x7fa688156270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa680009d90 tx=0x7fa68000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbaa8dae05d0d294 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa67ffff700 1 -- 172.21.15.31:0/1511198507 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68000e070 con 0x7fa68806e6a0 2024-03-30T01:41:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa67ffff700 1 -- 172.21.15.31:0/1511198507 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68000bb90 con 0x7fa68806e6a0 2024-03-30T01:41:42.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa67ffff700 1 -- 172.21.15.31:0/1511198507 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680003a20 con 0x7fa68806e6a0 2024-03-30T01:41:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa68f92f700 1 -- 172.21.15.31:0/1511198507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806e6a0 msgr2=0x7fa688156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa68f92f700 1 --2- 172.21.15.31:0/1511198507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806e6a0 0x7fa688156270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa680009d90 tx=0x7fa68000b330 comp rx=0 tx=0).stop 2024-03-30T01:41:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa68f92f700 1 -- 172.21.15.31:0/1511198507 shutdown_connections 2024-03-30T01:41:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa68f92f700 1 --2- 172.21.15.31:0/1511198507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806e6a0 0x7fa688156270 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa68f92f700 1 -- 172.21.15.31:0/1511198507 >> 172.21.15.31:0/1511198507 conn(0x7fa6880a82d0 msgr2=0x7fa6880a86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa68f92f700 1 -- 172.21.15.31:0/1511198507 shutdown_connections 2024-03-30T01:41:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.405+0000 7fa68f92f700 1 -- 172.21.15.31:0/1511198507 wait complete. 2024-03-30T01:41:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.407+0000 7fa68f92f700 1 Processor -- start 2024-03-30T01:41:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.407+0000 7fa68f92f700 1 -- start start 2024-03-30T01:41:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.407+0000 7fa68f92f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806b930 0x7fa6880a7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.407+0000 7fa68f92f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa68806e6a0 0x7fa6881e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.407+0000 7fa68f92f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68814c4a0 0x7fa6881ea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.407+0000 7fa68f92f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6880c5b00 con 0x7fa68806b930 2024-03-30T01:41:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.407+0000 7fa68f92f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881ea9d0 con 0x7fa68814c4a0 2024-03-30T01:41:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.407+0000 7fa68f92f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881eacb0 con 0x7fa68806e6a0 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.407+0000 7fa68d6cb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806b930 0x7fa6880a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68d6cb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806b930 0x7fa6880a7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46574/0 (socket says 172.21.15.31:46574) 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68d6cb700 1 -- 172.21.15.31:0/582349317 learned_addr learned my addr 172.21.15.31:0/582349317 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68decc700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68814c4a0 0x7fa6881ea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68ceca700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa68806e6a0 0x7fa6881e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68d6cb700 1 -- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa68806e6a0 msgr2=0x7fa6881e9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68d6cb700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa68806e6a0 0x7fa6881e9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68d6cb700 1 -- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68814c4a0 msgr2=0x7fa6881ea3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68d6cb700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68814c4a0 0x7fa6881ea3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68d6cb700 1 -- 172.21.15.31:0/582349317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa680009a70 con 0x7fa68806b930 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68ceca700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa68806e6a0 0x7fa6881e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68decc700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68814c4a0 0x7fa6881ea3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68d6cb700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806b930 0x7fa6880a7810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c4d0 tx=0x7fa68400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa67dffb700 1 -- 172.21.15.31:0/582349317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684011790 con 0x7fa68806b930 2024-03-30T01:41:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa67dffb700 1 -- 172.21.15.31:0/582349317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6840118f0 con 0x7fa68806b930 2024-03-30T01:41:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6881eaf90 con 0x7fa68806b930 2024-03-30T01:41:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.408+0000 7fa67dffb700 1 -- 172.21.15.31:0/582349317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684011ca0 con 0x7fa68806b930 2024-03-30T01:41:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.409+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6881eb550 con 0x7fa68806b930 2024-03-30T01:41:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.410+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68804ea80 con 0x7fa68806b930 2024-03-30T01:41:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.410+0000 7fa67dffb700 1 -- 172.21.15.31:0/582349317 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa68401c460 con 0x7fa68806b930 2024-03-30T01:41:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.410+0000 7fa67dffb700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa66c06ac90 0x7fa66c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.410+0000 7fa67dffb700 1 -- 172.21.15.31:0/582349317 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa684089530 con 0x7fa68806b930 2024-03-30T01:41:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.410+0000 7fa68ceca700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa66c06ac90 0x7fa66c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.411+0000 7fa68ceca700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa66c06ac90 0x7fa66c06d150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa678007fd0 tx=0x7fa678009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:42.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.415+0000 7fa67dffb700 1 -- 172.21.15.31:0/582349317 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa6840543e0 con 0x7fa68806b930 2024-03-30T01:41:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.631+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa68806b710 con 0x7fa68806b930 2024-03-30T01:41:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.632+0000 7fa67dffb700 1 -- 172.21.15.31:0/582349317 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18219 (secure 0 0 0) 0x7fa684057a00 con 0x7fa68806b930 2024-03-30T01:41:42.640 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:42.640 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:41.918800+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa66c06ac90 msgr2=0x7fa66c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa66c06ac90 0x7fa66c06d150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa678007fd0 tx=0x7fa678009040 comp rx=0 tx=0).stop 2024-03-30T01:41:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806b930 msgr2=0x7fa6880a7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806b930 0x7fa6880a7810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa68400c4d0 tx=0x7fa68400c890 comp rx=0 tx=0).stop 2024-03-30T01:41:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 shutdown_connections 2024-03-30T01:41:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa68806b930 0x7fa6880a7810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa66c06ac90 0x7fa66c06d150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa68814c4a0 0x7fa6881ea3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 --2- 172.21.15.31:0/582349317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa68806e6a0 0x7fa6881e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.634+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 >> 172.21.15.31:0/582349317 conn(0x7fa6880a82d0 msgr2=0x7fa688150370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.635+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 shutdown_connections 2024-03-30T01:41:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.635+0000 7fa68f92f700 1 -- 172.21.15.31:0/582349317 wait complete. 2024-03-30T01:41:42.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:42.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.790+0000 7f836a40e700 1 Processor -- start 2024-03-30T01:41:42.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.790+0000 7f836a40e700 1 -- start start 2024-03-30T01:41:42.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f836a40e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83641549a0 0x7f8364072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f836a40e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f836406d040 0x7f8364154d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f836a40e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d900 0x7f836406dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f836a40e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836414ee60 con 0x7f836406d040 2024-03-30T01:41:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f836a40e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83640c2290 con 0x7f83641549a0 2024-03-30T01:41:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f836a40e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836406e2b0 con 0x7f836406d900 2024-03-30T01:41:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f83637fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f836406d040 0x7f8364154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f83637fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f836406d040 0x7f8364154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46586/0 (socket says 172.21.15.31:46586) 2024-03-30T01:41:42.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f83637fe700 1 -- 172.21.15.31:0/506065382 learned_addr learned my addr 172.21.15.31:0/506065382 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:42.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f83689ab700 1 --2- 172.21.15.31:0/506065382 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d900 0x7f836406dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f8363fff700 1 --2- 172.21.15.31:0/506065382 >> v2:172.21.15.59:3300/0 conn(0x7f83641549a0 0x7f8364072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f83637fe700 1 -- 172.21.15.31:0/506065382 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d900 msgr2=0x7f836406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:42.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f83637fe700 1 --2- 172.21.15.31:0/506065382 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d900 0x7f836406dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f83637fe700 1 -- 172.21.15.31:0/506065382 >> v2:172.21.15.59:3300/0 conn(0x7f83641549a0 msgr2=0x7f8364072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f83637fe700 1 --2- 172.21.15.31:0/506065382 >> v2:172.21.15.59:3300/0 conn(0x7f83641549a0 0x7f8364072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.791+0000 7f83637fe700 1 -- 172.21.15.31:0/506065382 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364155720 con 0x7f836406d040 2024-03-30T01:41:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.792+0000 7f83637fe700 1 --2- 172.21.15.31:0/506065382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f836406d040 0x7f8364154d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f83640733c0 tx=0x7f8358009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa0099b6930bee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:42.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.792+0000 7f8362ffd700 1 -- 172.21.15.31:0/506065382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358011830 con 0x7f836406d040 2024-03-30T01:41:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.792+0000 7f8362ffd700 1 -- 172.21.15.31:0/506065382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8358011990 con 0x7f836406d040 2024-03-30T01:41:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.792+0000 7f8362ffd700 1 -- 172.21.15.31:0/506065382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358011d20 con 0x7f836406d040 2024-03-30T01:41:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.792+0000 7f836a40e700 1 -- 172.21.15.31:0/506065382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f836406d040 msgr2=0x7f8364154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.792+0000 7f836a40e700 1 --2- 172.21.15.31:0/506065382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f836406d040 0x7f8364154d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f83640733c0 tx=0x7f8358009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.793+0000 7f836a40e700 1 -- 172.21.15.31:0/506065382 shutdown_connections 2024-03-30T01:41:42.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.793+0000 7f836a40e700 1 --2- 172.21.15.31:0/506065382 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d900 0x7f836406dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.793+0000 7f836a40e700 1 --2- 172.21.15.31:0/506065382 >> v2:172.21.15.59:3300/0 conn(0x7f83641549a0 0x7f8364072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.793+0000 7f836a40e700 1 --2- 172.21.15.31:0/506065382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f836406d040 0x7f8364154d80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.793+0000 7f836a40e700 1 -- 172.21.15.31:0/506065382 >> 172.21.15.31:0/506065382 conn(0x7f83640c3980 msgr2=0x7f83640c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.793+0000 7f836a40e700 1 -- 172.21.15.31:0/506065382 shutdown_connections 2024-03-30T01:41:42.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.793+0000 7f836a40e700 1 -- 172.21.15.31:0/506065382 wait complete. 2024-03-30T01:41:42.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.794+0000 7f836a40e700 1 Processor -- start 2024-03-30T01:41:42.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.794+0000 7f836a40e700 1 -- start start 2024-03-30T01:41:42.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f836a40e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d040 0x7f83641df090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f836a40e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f836406d900 0x7f83641415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f836a40e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83641549a0 0x7f8364141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f836a40e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640aa890 con 0x7f83641549a0 2024-03-30T01:41:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f836a40e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8364142050 con 0x7f836406d900 2024-03-30T01:41:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f836a40e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83641edb00 con 0x7f836406d040 2024-03-30T01:41:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f8363fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d040 0x7f83641df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f83637fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f836406d900 0x7f83641415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f83689ab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83641549a0 0x7f8364141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f8363fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d040 0x7f83641df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:51150/0 (socket says 172.21.15.31:51150) 2024-03-30T01:41:42.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f83637fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f836406d900 0x7f83641415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:56618/0 (socket says 172.21.15.31:56618) 2024-03-30T01:41:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.795+0000 7f8363fff700 1 -- 172.21.15.31:0/991728023 learned_addr learned my addr 172.21.15.31:0/991728023 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f83689ab700 1 -- 172.21.15.31:0/991728023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d040 msgr2=0x7f83641df090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f83689ab700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d040 0x7f83641df090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f83689ab700 1 -- 172.21.15.31:0/991728023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f836406d900 msgr2=0x7f83641415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f83689ab700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f836406d900 0x7f83641415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:42.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f83689ab700 1 -- 172.21.15.31:0/991728023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8358009550 con 0x7f83641549a0 2024-03-30T01:41:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f83689ab700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83641549a0 0x7f8364141b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f835c00eed0 tx=0x7f835c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f8360ff9700 1 -- 172.21.15.31:0/991728023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c010040 con 0x7f83641549a0 2024-03-30T01:41:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83641edde0 con 0x7f83641549a0 2024-03-30T01:41:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83641ee3a0 con 0x7f83641549a0 2024-03-30T01:41:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f8360ff9700 1 -- 172.21.15.31:0/991728023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c00ba80 con 0x7f83641549a0 2024-03-30T01:41:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.796+0000 7f8360ff9700 1 -- 172.21.15.31:0/991728023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c00be30 con 0x7f83641549a0 2024-03-30T01:41:42.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.798+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83440052f0 con 0x7f83641549a0 2024-03-30T01:41:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.798+0000 7f8360ff9700 1 -- 172.21.15.31:0/991728023 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f835c01d400 con 0x7f83641549a0 2024-03-30T01:41:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.799+0000 7f8360ff9700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f835006ac90 0x7f835006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.799+0000 7f8360ff9700 1 -- 172.21.15.31:0/991728023 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f835c089e00 con 0x7f83641549a0 2024-03-30T01:41:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.799+0000 7f8363fff700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f835006ac90 0x7f835006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:42.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.799+0000 7f8363fff700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f835006ac90 0x7f835006d150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8354009c00 tx=0x7f8354009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:42.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:42.803+0000 7f8360ff9700 1 -- 172.21.15.31:0/991728023 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f835c054ba0 con 0x7f83641549a0 2024-03-30T01:41:43.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.025+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8344005b20 con 0x7f83641549a0 2024-03-30T01:41:43.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.025+0000 7f8360ff9700 1 -- 172.21.15.31:0/991728023 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f835c0581c0 con 0x7f83641549a0 2024-03-30T01:41:43.031 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:43.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.027+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f835006ac90 msgr2=0x7f835006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.027+0000 7f836a40e700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f835006ac90 0x7f835006d150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8354009c00 tx=0x7f8354009210 comp rx=0 tx=0).stop 2024-03-30T01:41:43.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.027+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83641549a0 msgr2=0x7f8364141b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.027+0000 7f836a40e700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83641549a0 0x7f8364141b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f835c00eed0 tx=0x7f835c00b390 comp rx=0 tx=0).stop 2024-03-30T01:41:43.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.027+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 shutdown_connections 2024-03-30T01:41:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.027+0000 7f836a40e700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83641549a0 0x7f8364141b10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.028+0000 7f836a40e700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f835006ac90 0x7f835006d150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.028+0000 7f836a40e700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f836406d040 0x7f83641df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.028+0000 7f836a40e700 1 --2- 172.21.15.31:0/991728023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f836406d900 0x7f83641415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.028+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 >> 172.21.15.31:0/991728023 conn(0x7f83640c3980 msgr2=0x7f83640c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.028+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 shutdown_connections 2024-03-30T01:41:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.028+0000 7f836a40e700 1 -- 172.21.15.31:0/991728023 wait complete. 2024-03-30T01:41:43.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.182+0000 7f5067075700 1 Processor -- start 2024-03-30T01:41:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.182+0000 7f5067075700 1 -- start start 2024-03-30T01:41:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f5067075700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f50601549a0 0x7f5060072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f5067075700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f506006d040 0x7f5060154d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f5067075700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506014ee60 con 0x7f506006d900 2024-03-30T01:41:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f5067075700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50600c2290 con 0x7f506006d040 2024-03-30T01:41:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f5067075700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5060155720 con 0x7f50601549a0 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f505ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f506006d040 0x7f5060154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f5065612700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f506006d900 0x7f50601513b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53102/0 (socket says 172.21.15.31:53102) 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f505ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f506006d040 0x7f5060154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46608/0 (socket says 172.21.15.31:46608) 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f5065612700 1 -- 172.21.15.31:0/3920114803 learned_addr learned my addr 172.21.15.31:0/3920114803 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f5064e11700 1 --2- 172.21.15.31:0/3920114803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f50601549a0 0x7f5060072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f505ffff700 1 -- 172.21.15.31:0/3920114803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f50601549a0 msgr2=0x7f5060072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f505ffff700 1 --2- 172.21.15.31:0/3920114803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f50601549a0 0x7f5060072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f505ffff700 1 -- 172.21.15.31:0/3920114803 >> v1:172.21.15.31:6789/0 conn(0x7f506006d900 legacy=0x7f50601513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f505f7fe700 1 -- 172.21.15.31:0/3920114803 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397944722 0 0) 0x7f506014ee60 con 0x7f506006d900 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.183+0000 7f505ffff700 1 -- 172.21.15.31:0/3920114803 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600c24e0 con 0x7f506006d040 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f505ffff700 1 --2- 172.21.15.31:0/3920114803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 0x7f5060154d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f504c009840 tx=0x7f504c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200fa77047355a8f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f505f7fe700 1 -- 172.21.15.31:0/3920114803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011830 con 0x7f506006d040 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f505f7fe700 1 -- 172.21.15.31:0/3920114803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c011990 con 0x7f506006d040 2024-03-30T01:41:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f505f7fe700 1 -- 172.21.15.31:0/3920114803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011d20 con 0x7f506006d040 2024-03-30T01:41:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f5067075700 1 -- 172.21.15.31:0/3920114803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 msgr2=0x7f5060154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f5067075700 1 --2- 172.21.15.31:0/3920114803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 0x7f5060154d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f504c009840 tx=0x7f504c009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f5067075700 1 -- 172.21.15.31:0/3920114803 shutdown_connections 2024-03-30T01:41:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f5067075700 1 --2- 172.21.15.31:0/3920114803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f50601549a0 0x7f5060072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f5067075700 1 --2- 172.21.15.31:0/3920114803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 0x7f5060154d80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f5067075700 1 -- 172.21.15.31:0/3920114803 >> 172.21.15.31:0/3920114803 conn(0x7f50600c3980 msgr2=0x7f50600c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f5067075700 1 -- 172.21.15.31:0/3920114803 shutdown_connections 2024-03-30T01:41:43.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.184+0000 7f5067075700 1 -- 172.21.15.31:0/3920114803 wait complete. 2024-03-30T01:41:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.186+0000 7f5067075700 1 Processor -- start 2024-03-30T01:41:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.186+0000 7f5067075700 1 -- start start 2024-03-30T01:41:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.186+0000 7f5067075700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 0x7f50600ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.186+0000 7f5067075700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506006d900 0x7f50600abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.186+0000 7f5067075700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50601549a0 0x7f5060149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.186+0000 7f5067075700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50600ac310 con 0x7f506006d040 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.186+0000 7f5067075700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50600ab190 con 0x7f50601549a0 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.186+0000 7f5067075700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50600a9cd0 con 0x7f506006d900 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5064e11700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 0x7f50600ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5064e11700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 0x7f50600ab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46622/0 (socket says 172.21.15.31:46622) 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5064e11700 1 -- 172.21.15.31:0/3591986502 learned_addr learned my addr 172.21.15.31:0/3591986502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5065612700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50601549a0 0x7f5060149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f505ffff700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506006d900 0x7f50600abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5064e11700 1 -- 172.21.15.31:0/3591986502 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506006d900 msgr2=0x7f50600abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5064e11700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506006d900 0x7f50600abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5064e11700 1 -- 172.21.15.31:0/3591986502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50601549a0 msgr2=0x7f5060149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5064e11700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50601549a0 0x7f5060149830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5064e11700 1 -- 172.21.15.31:0/3591986502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c0094a0 con 0x7f506006d040 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5065612700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50601549a0 0x7f5060149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5064e11700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 0x7f50600ab890 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f505400d800 tx=0x7f505400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f505d7fa700 1 -- 172.21.15.31:0/3591986502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540125f0 con 0x7f506006d040 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f505d7fa700 1 -- 172.21.15.31:0/3591986502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5054012750 con 0x7f506006d040 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f505d7fa700 1 -- 172.21.15.31:0/3591986502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054012b00 con 0x7f506006d040 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.187+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50601493a0 con 0x7f506006d040 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.188+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506014b0b0 con 0x7f506006d040 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.188+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5060149680 con 0x7f506006d040 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.189+0000 7f505d7fa700 1 -- 172.21.15.31:0/3591986502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5054012c60 con 0x7f506006d040 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.189+0000 7f505d7fa700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f504406ac90 0x7f504406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.190+0000 7f505d7fa700 1 -- 172.21.15.31:0/3591986502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f50540899c0 con 0x7f506006d040 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.190+0000 7f505ffff700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f504406ac90 0x7f504406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.190+0000 7f505ffff700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f504406ac90 0x7f504406d150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f506006d460 tx=0x7f504c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:43.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.194+0000 7f505d7fa700 1 -- 172.21.15.31:0/3591986502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5054054870 con 0x7f506006d040 2024-03-30T01:41:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.406+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50600c4870 con 0x7f506006d040 2024-03-30T01:41:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.407+0000 7f505d7fa700 1 -- 172.21.15.31:0/3591986502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5054057e90 con 0x7f506006d040 2024-03-30T01:41:43.412 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f504406ac90 msgr2=0x7f504406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f504406ac90 0x7f504406d150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f506006d460 tx=0x7f504c008030 comp rx=0 tx=0).stop 2024-03-30T01:41:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 msgr2=0x7f50600ab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 0x7f50600ab890 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f505400d800 tx=0x7f505400db10 comp rx=0 tx=0).stop 2024-03-30T01:41:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 shutdown_connections 2024-03-30T01:41:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506006d040 0x7f50600ab890 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f504406ac90 0x7f504406d150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50601549a0 0x7f5060149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 --2- 172.21.15.31:0/3591986502 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506006d900 0x7f50600abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.409+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 >> 172.21.15.31:0/3591986502 conn(0x7f50600c3980 msgr2=0x7f50600c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.410+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 shutdown_connections 2024-03-30T01:41:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.410+0000 7f5067075700 1 -- 172.21.15.31:0/3591986502 wait complete. 2024-03-30T01:41:43.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:43.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.562+0000 7f56009eb700 1 Processor -- start 2024-03-30T01:41:43.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.562+0000 7f56009eb700 1 -- start start 2024-03-30T01:41:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f56009eb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 0x7f55fc0c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f56009eb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55fc06cfc0 0x7f55fc0c1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f56009eb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55fc0c1660 con 0x7f55fc06c700 2024-03-30T01:41:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f56009eb700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc06bc60 con 0x7f55fc06cfc0 2024-03-30T01:41:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f56009eb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0a77e0 con 0x7f55fc0c64a0 2024-03-30T01:41:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f55f9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55fc06cfc0 0x7f55fc0c1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f55f9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f55fc06cfc0 0x7f55fc0c1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46626/0 (socket says 172.21.15.31:46626) 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f55f9d9b700 1 -- 172.21.15.31:0/3553321258 learned_addr learned my addr 172.21.15.31:0/3553321258 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f55fa59c700 1 --2- 172.21.15.31:0/3553321258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 0x7f55fc0c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f55f9d9b700 1 -- 172.21.15.31:0/3553321258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 msgr2=0x7f55fc0c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.563+0000 7f55f9d9b700 1 --2- 172.21.15.31:0/3553321258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 0x7f55fc0c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.564+0000 7f55f9d9b700 1 -- 172.21.15.31:0/3553321258 >> v1:172.21.15.59:6789/0 conn(0x7f55fc06c700 legacy=0x7f55fc0705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.564+0000 7f55f9d9b700 1 -- 172.21.15.31:0/3553321258 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc0c5d90 con 0x7f55fc06cfc0 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.564+0000 7f55fa59c700 1 --2- 172.21.15.31:0/3553321258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 0x7f55fc0c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.564+0000 7f55f9d9b700 1 --2- 172.21.15.31:0/3553321258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 0x7f55fc0c1120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55e40098f0 tx=0x7f55e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92a062e2752b221 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.564+0000 7f55f959a700 1 -- 172.21.15.31:0/3553321258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4004230 con 0x7f55fc06cfc0 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.564+0000 7f55f959a700 1 -- 172.21.15.31:0/3553321258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e4004390 con 0x7f55fc06cfc0 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.564+0000 7f55f959a700 1 -- 172.21.15.31:0/3553321258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e40036a0 con 0x7f55fc06cfc0 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.564+0000 7f56009eb700 1 -- 172.21.15.31:0/3553321258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 msgr2=0x7f55fc0c1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.564+0000 7f56009eb700 1 --2- 172.21.15.31:0/3553321258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 0x7f55fc0c1120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55e40098f0 tx=0x7f55e4009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.565+0000 7f56009eb700 1 -- 172.21.15.31:0/3553321258 shutdown_connections 2024-03-30T01:41:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.565+0000 7f56009eb700 1 --2- 172.21.15.31:0/3553321258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 0x7f55fc0c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.565+0000 7f56009eb700 1 --2- 172.21.15.31:0/3553321258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 0x7f55fc0c1120 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.565+0000 7f56009eb700 1 -- 172.21.15.31:0/3553321258 >> 172.21.15.31:0/3553321258 conn(0x7f55fc0c7cf0 msgr2=0x7f55fc0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.565+0000 7f56009eb700 1 -- 172.21.15.31:0/3553321258 shutdown_connections 2024-03-30T01:41:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.565+0000 7f56009eb700 1 -- 172.21.15.31:0/3553321258 wait complete. 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.566+0000 7f56009eb700 1 Processor -- start 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.566+0000 7f56009eb700 1 -- start start 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f56009eb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55fc06c700 0x7f55fc0a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f56009eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 0x7f55fc1eab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f56009eb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 0x7f55fc0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f56009eb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc1dfa70 con 0x7f55fc06cfc0 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f56009eb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc1eb070 con 0x7f55fc06c700 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f56009eb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0a9fb0 con 0x7f55fc0c64a0 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55f9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 0x7f55fc1eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55f9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 0x7f55fc1eab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46632/0 (socket says 172.21.15.31:46632) 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55f9d9b700 1 -- 172.21.15.31:0/1983986353 learned_addr learned my addr 172.21.15.31:0/1983986353 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55fa59c700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55fc06c700 0x7f55fc0a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55fad9d700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 0x7f55fc0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55f9d9b700 1 -- 172.21.15.31:0/1983986353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 msgr2=0x7f55fc0a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55f9d9b700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 0x7f55fc0a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55f9d9b700 1 -- 172.21.15.31:0/1983986353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55fc06c700 msgr2=0x7f55fc0a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55f9d9b700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55fc06c700 0x7f55fc0a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.567+0000 7f55f9d9b700 1 -- 172.21.15.31:0/1983986353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e4009550 con 0x7f55fc06cfc0 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.568+0000 7f55f9d9b700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 0x7f55fc1eab30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009130 tx=0x7f55e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.568+0000 7f55f2ffd700 1 -- 172.21.15.31:0/1983986353 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4003cd0 con 0x7f55fc06cfc0 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.568+0000 7f55f2ffd700 1 -- 172.21.15.31:0/1983986353 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e401f070 con 0x7f55fc06cfc0 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.568+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc0a7ac0 con 0x7f55fc06cfc0 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.568+0000 7f55f2ffd700 1 -- 172.21.15.31:0/1983986353 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e400b8a0 con 0x7f55fc06cfc0 2024-03-30T01:41:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.568+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc0a8080 con 0x7f55fc06cfc0 2024-03-30T01:41:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.569+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc0a7920 con 0x7f55fc06cfc0 2024-03-30T01:41:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.570+0000 7f55f2ffd700 1 -- 172.21.15.31:0/1983986353 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f55e4021030 con 0x7f55fc06cfc0 2024-03-30T01:41:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.570+0000 7f55f2ffd700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f55dc06ac90 0x7f55dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.570+0000 7f55f2ffd700 1 -- 172.21.15.31:0/1983986353 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f55e408ac60 con 0x7f55fc06cfc0 2024-03-30T01:41:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.570+0000 7f55fa59c700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f55dc06ac90 0x7f55dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.570+0000 7f55fa59c700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f55dc06ac90 0x7f55dc06d150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55ec006fd0 tx=0x7f55ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:43.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.575+0000 7f55f2ffd700 1 -- 172.21.15.31:0/1983986353 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f55fc0a7920 con 0x7f55fc06cfc0 2024-03-30T01:41:43.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.786+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55fc0a7920 con 0x7f55fc06cfc0 2024-03-30T01:41:43.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.786+0000 7f55f2ffd700 1 -- 172.21.15.31:0/1983986353 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55fc0a7920 con 0x7f55fc06cfc0 2024-03-30T01:41:43.792 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f55dc06ac90 msgr2=0x7f55dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f55dc06ac90 0x7f55dc06d150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55ec006fd0 tx=0x7f55ec008040 comp rx=0 tx=0).stop 2024-03-30T01:41:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 msgr2=0x7f55fc1eab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 0x7f55fc1eab30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009130 tx=0x7f55e4004750 comp rx=0 tx=0).stop 2024-03-30T01:41:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 shutdown_connections 2024-03-30T01:41:43.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f55dc06ac90 0x7f55dc06d150 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f55fc0c64a0 0x7f55fc0a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55fc06c700 0x7f55fc0a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 --2- 172.21.15.31:0/1983986353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55fc06cfc0 0x7f55fc1eab30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 >> 172.21.15.31:0/1983986353 conn(0x7f55fc0c7cf0 msgr2=0x7f55fc0c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:43.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 shutdown_connections 2024-03-30T01:41:43.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.789+0000 7f56009eb700 1 -- 172.21.15.31:0/1983986353 wait complete. 2024-03-30T01:41:43.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:43.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.946+0000 7f5e85941700 1 Processor -- start 2024-03-30T01:41:43.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.946+0000 7f5e85941700 1 -- start start 2024-03-30T01:41:43.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.947+0000 7f5e85941700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e80151cf0 0x7f5e800a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.947+0000 7f5e85941700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 0x7f5e80157c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.947+0000 7f5e85941700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e80071110 con 0x7f5e800c6e70 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.947+0000 7f5e85941700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e80155540 con 0x7f5e80151cf0 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.947+0000 7f5e85941700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80071830 con 0x7f5e800bffa0 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.947+0000 7f5e7effd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e80151cf0 0x7f5e800a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.947+0000 7f5e7e7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 0x7f5e80157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.947+0000 7f5e7f7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5e800c6e70 0x7f5e8006a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:47218/0 (socket says 172.21.15.31:47218) 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.947+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/330650917 learned_addr learned my addr 172.21.15.31:0/330650917 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.948+0000 7f5e7effd700 1 -- 172.21.15.31:0/330650917 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 msgr2=0x7f5e80157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.948+0000 7f5e7effd700 1 --2- 172.21.15.31:0/330650917 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 0x7f5e80157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.948+0000 7f5e7effd700 1 -- 172.21.15.31:0/330650917 >> v1:172.21.15.59:6789/0 conn(0x7f5e800c6e70 legacy=0x7f5e8006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.948+0000 7f5e7effd700 1 -- 172.21.15.31:0/330650917 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e801520d0 con 0x7f5e80151cf0 2024-03-30T01:41:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.948+0000 7f5e7e7fc700 1 --2- 172.21.15.31:0/330650917 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 0x7f5e80157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.948+0000 7f5e7effd700 1 --2- 172.21.15.31:0/330650917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e80151cf0 0x7f5e800a61a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009840 tx=0x7f5e70009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee58dfba606d671 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.948+0000 7f5e7dffb700 1 -- 172.21.15.31:0/330650917 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70011830 con 0x7f5e80151cf0 2024-03-30T01:41:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.948+0000 7f5e7dffb700 1 -- 172.21.15.31:0/330650917 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e70011990 con 0x7f5e80151cf0 2024-03-30T01:41:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.948+0000 7f5e7dffb700 1 -- 172.21.15.31:0/330650917 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70011d20 con 0x7f5e80151cf0 2024-03-30T01:41:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.949+0000 7f5e85941700 1 -- 172.21.15.31:0/330650917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e80151cf0 msgr2=0x7f5e800a61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.949+0000 7f5e85941700 1 --2- 172.21.15.31:0/330650917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e80151cf0 0x7f5e800a61a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009840 tx=0x7f5e70009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.949+0000 7f5e85941700 1 -- 172.21.15.31:0/330650917 shutdown_connections 2024-03-30T01:41:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.949+0000 7f5e85941700 1 --2- 172.21.15.31:0/330650917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e80151cf0 0x7f5e800a61a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.949+0000 7f5e85941700 1 --2- 172.21.15.31:0/330650917 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 0x7f5e80157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.949+0000 7f5e85941700 1 -- 172.21.15.31:0/330650917 >> 172.21.15.31:0/330650917 conn(0x7f5e8006b9f0 msgr2=0x7f5e8006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.949+0000 7f5e85941700 1 -- 172.21.15.31:0/330650917 shutdown_connections 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.949+0000 7f5e85941700 1 -- 172.21.15.31:0/330650917 wait complete. 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.950+0000 7f5e85941700 1 Processor -- start 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.950+0000 7f5e85941700 1 -- start start 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e85941700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 0x7f5e8014b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e85941700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e800c6e70 0x7f5e800abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e85941700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e80151cf0 0x7f5e800ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e85941700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8014b8f0 con 0x7f5e80151cf0 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e85941700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80154270 con 0x7f5e800c6e70 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e85941700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e800ac620 con 0x7f5e800bffa0 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e80151cf0 0x7f5e800ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e80151cf0 0x7f5e800ac0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46648/0 (socket says 172.21.15.31:46648) 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3974990092 learned_addr learned my addr 172.21.15.31:0/3974990092 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7effd700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 0x7f5e8014b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7e7fc700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e800c6e70 0x7f5e800abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3974990092 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 msgr2=0x7f5e8014b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7f7fe700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 0x7f5e8014b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3974990092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e800c6e70 msgr2=0x7f5e800abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7f7fe700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e800c6e70 0x7f5e800abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.951+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3974990092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740094a0 con 0x7f5e80151cf0 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.952+0000 7f5e7f7fe700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e80151cf0 0x7f5e800ac0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800cbf0 tx=0x7f5e780021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.952+0000 7f5e67fff700 1 -- 172.21.15.31:0/3974990092 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800ab40 con 0x7f5e80151cf0 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.952+0000 7f5e67fff700 1 -- 172.21.15.31:0/3974990092 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7800aca0 con 0x7f5e80151cf0 2024-03-30T01:41:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.952+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e700094a0 con 0x7f5e80151cf0 2024-03-30T01:41:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.952+0000 7f5e67fff700 1 -- 172.21.15.31:0/3974990092 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78012600 con 0x7f5e80151cf0 2024-03-30T01:41:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.952+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8014d800 con 0x7f5e80151cf0 2024-03-30T01:41:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.953+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e800747d0 con 0x7f5e80151cf0 2024-03-30T01:41:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.953+0000 7f5e67fff700 1 -- 172.21.15.31:0/3974990092 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5e7801b3b0 con 0x7f5e80151cf0 2024-03-30T01:41:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.954+0000 7f5e67fff700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e6c06ac90 0x7f5e6c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.954+0000 7f5e67fff700 1 -- 172.21.15.31:0/3974990092 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5e78089330 con 0x7f5e80151cf0 2024-03-30T01:41:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.954+0000 7f5e7effd700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e6c06ac90 0x7f5e6c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.954+0000 7f5e7effd700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e6c06ac90 0x7f5e6c06d150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e70000c00 tx=0x7f5e70008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:43.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:43.958+0000 7f5e67fff700 1 -- 172.21.15.31:0/3974990092 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5e78054200 con 0x7f5e80151cf0 2024-03-30T01:41:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.170+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5e800c44a0 con 0x7f5e80151cf0 2024-03-30T01:41:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.170+0000 7f5e67fff700 1 -- 172.21.15.31:0/3974990092 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5e78057820 con 0x7f5e80151cf0 2024-03-30T01:41:44.176 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e6c06ac90 msgr2=0x7f5e6c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e6c06ac90 0x7f5e6c06d150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e70000c00 tx=0x7f5e70008030 comp rx=0 tx=0).stop 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e80151cf0 msgr2=0x7f5e800ac0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e80151cf0 0x7f5e800ac0e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800cbf0 tx=0x7f5e780021c0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 shutdown_connections 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e80151cf0 0x7f5e800ac0e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e6c06ac90 0x7f5e6c06d150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e800bffa0 0x7f5e8014b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 --2- 172.21.15.31:0/3974990092 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e800c6e70 0x7f5e800abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 >> 172.21.15.31:0/3974990092 conn(0x7f5e8006b9f0 msgr2=0x7f5e80150770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:44.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 shutdown_connections 2024-03-30T01:41:44.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.173+0000 7f5e85941700 1 -- 172.21.15.31:0/3974990092 wait complete. 2024-03-30T01:41:44.245 INFO:tasks.cephfs_test_runner:Starting test: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) 2024-03-30T01:41:44.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.379+0000 7f1973790700 1 Processor -- start 2024-03-30T01:41:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.379+0000 7f1973790700 1 -- start start 2024-03-30T01:41:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1973790700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f196c154a10 0x7f196c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1973790700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0c22c0 con 0x7f196c147f10 2024-03-30T01:41:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1973790700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0c25a0 con 0x7f196c0c3450 2024-03-30T01:41:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1973790700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c0c2f40 con 0x7f196c154a10 2024-03-30T01:41:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f197152c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f196c147f10 0x7f196c1529f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53112/0 (socket says 172.21.15.31:53112) 2024-03-30T01:41:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f197152c700 1 -- 172.21.15.31:0/3178063380 learned_addr learned my addr 172.21.15.31:0/3178063380 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1971d2d700 1 --2- 172.21.15.31:0/3178063380 >> v2:172.21.15.59:3300/0 conn(0x7f196c154a10 0x7f196c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1963fff700 1 -- 172.21.15.31:0/3178063380 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541912727 0 0) 0x7f196c0c22c0 con 0x7f196c147f10 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1963fff700 1 -- 172.21.15.31:0/3178063380 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19580035b0 con 0x7f196c147f10 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1963fff700 1 -- 172.21.15.31:0/3178063380 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2230735561 0 0) 0x7f19580035b0 con 0x7f196c147f10 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1971d2d700 1 -- 172.21.15.31:0/3178063380 >> v1:172.21.15.59:6789/0 conn(0x7f196c0c3450 legacy=0x7f196c153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1971d2d700 1 -- 172.21.15.31:0/3178063380 >> v1:172.21.15.31:6789/0 conn(0x7f196c147f10 legacy=0x7f196c1529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.380+0000 7f1971d2d700 1 -- 172.21.15.31:0/3178063380 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0c2880 con 0x7f196c154a10 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1971d2d700 1 --2- 172.21.15.31:0/3178063380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c154a10 0x7f196c154e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1964009d90 tx=0x7f196400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b50014ea5cbcd223 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1963fff700 1 -- 172.21.15.31:0/3178063380 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400e070 con 0x7f196c154a10 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1963fff700 1 -- 172.21.15.31:0/3178063380 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1964003a20 con 0x7f196c154a10 2024-03-30T01:41:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1963fff700 1 -- 172.21.15.31:0/3178063380 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968001670 con 0x7f196c154a10 2024-03-30T01:41:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1973790700 1 -- 172.21.15.31:0/3178063380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c154a10 msgr2=0x7f196c154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1973790700 1 --2- 172.21.15.31:0/3178063380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c154a10 0x7f196c154e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1964009d90 tx=0x7f196400b330 comp rx=0 tx=0).stop 2024-03-30T01:41:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1973790700 1 -- 172.21.15.31:0/3178063380 shutdown_connections 2024-03-30T01:41:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1973790700 1 --2- 172.21.15.31:0/3178063380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c154a10 0x7f196c154e40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1973790700 1 -- 172.21.15.31:0/3178063380 >> 172.21.15.31:0/3178063380 conn(0x7f196c0c3910 msgr2=0x7f196c0c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.381+0000 7f1973790700 1 -- 172.21.15.31:0/3178063380 shutdown_connections 2024-03-30T01:41:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.382+0000 7f1973790700 1 -- 172.21.15.31:0/3178063380 wait complete. 2024-03-30T01:41:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.383+0000 7f1973790700 1 Processor -- start 2024-03-30T01:41:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.383+0000 7f1973790700 1 -- start start 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.383+0000 7f1973790700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f196c0c3450 0x7f196c1e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.383+0000 7f1973790700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f196c147f10 0x7f196c1e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.383+0000 7f1973790700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c154a10 0x7f196c140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.383+0000 7f1973790700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0a9eb0 con 0x7f196c147f10 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.383+0000 7f1973790700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1e9250 con 0x7f196c154a10 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1973790700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c14e580 con 0x7f196c0c3450 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1970d2b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f196c147f10 0x7f196c1e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1970d2b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f196c147f10 0x7f196c1e8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46664/0 (socket says 172.21.15.31:46664) 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1970d2b700 1 -- 172.21.15.31:0/4009473279 learned_addr learned my addr 172.21.15.31:0/4009473279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1971d2d700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c154a10 0x7f196c140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f197152c700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f196c0c3450 0x7f196c1e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1970d2b700 1 -- 172.21.15.31:0/4009473279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f196c0c3450 msgr2=0x7f196c1e87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1970d2b700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f196c0c3450 0x7f196c1e87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1970d2b700 1 -- 172.21.15.31:0/4009473279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c154a10 msgr2=0x7f196c140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1970d2b700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c154a10 0x7f196c140570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1970d2b700 1 -- 172.21.15.31:0/4009473279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1964009a70 con 0x7f196c147f10 2024-03-30T01:41:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f197152c700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f196c0c3450 0x7f196c1e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1970d2b700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f196c147f10 0x7f196c1e8d10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c780 tx=0x7f195c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1961ffb700 1 -- 172.21.15.31:0/4009473279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c005480 con 0x7f196c147f10 2024-03-30T01:41:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1961ffb700 1 -- 172.21.15.31:0/4009473279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c003680 con 0x7f196c147f10 2024-03-30T01:41:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c14e860 con 0x7f196c147f10 2024-03-30T01:41:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.384+0000 7f1961ffb700 1 -- 172.21.15.31:0/4009473279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c003a30 con 0x7f196c147f10 2024-03-30T01:41:44.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.385+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c0aa870 con 0x7f196c147f10 2024-03-30T01:41:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.386+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c04ea80 con 0x7f196c147f10 2024-03-30T01:41:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.386+0000 7f1961ffb700 1 -- 172.21.15.31:0/4009473279 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f195c003b90 con 0x7f196c147f10 2024-03-30T01:41:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.386+0000 7f1961ffb700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f195006acc0 0x7f195006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.386+0000 7f1961ffb700 1 -- 172.21.15.31:0/4009473279 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f195c089270 con 0x7f196c147f10 2024-03-30T01:41:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.386+0000 7f197152c700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f195006acc0 0x7f195006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.387+0000 7f197152c700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f195006acc0 0x7f195006d180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19680011e0 tx=0x7f1968009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:44.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.391+0000 7f1961ffb700 1 -- 172.21.15.31:0/4009473279 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f195c054090 con 0x7f196c147f10 2024-03-30T01:41:44.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.608+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f196c0c31f0 con 0x7f196c147f10 2024-03-30T01:41:44.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.609+0000 7f1961ffb700 1 -- 172.21.15.31:0/4009473279 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18219 (secure 0 0 0) 0x7f195c0576b0 con 0x7f196c147f10 2024-03-30T01:41:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:44.615 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:42.922828+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:44.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f195006acc0 msgr2=0x7f195006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f195006acc0 0x7f195006d180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19680011e0 tx=0x7f1968009040 comp rx=0 tx=0).stop 2024-03-30T01:41:44.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f196c147f10 msgr2=0x7f196c1e8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f196c147f10 0x7f196c1e8d10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c780 tx=0x7f195c00cb40 comp rx=0 tx=0).stop 2024-03-30T01:41:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 shutdown_connections 2024-03-30T01:41:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f195006acc0 0x7f195006d180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f196c154a10 0x7f196c140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f196c0c3450 0x7f196c1e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 --2- 172.21.15.31:0/4009473279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f196c147f10 0x7f196c1e8d10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 >> 172.21.15.31:0/4009473279 conn(0x7f196c0c3910 msgr2=0x7f196c141a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 shutdown_connections 2024-03-30T01:41:44.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.612+0000 7f1973790700 1 -- 172.21.15.31:0/4009473279 wait complete. 2024-03-30T01:41:44.631 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:44.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:44.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.766+0000 7f0d699e2700 1 Processor -- start 2024-03-30T01:41:44.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.767+0000 7f0d699e2700 1 -- start start 2024-03-30T01:41:44.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.767+0000 7f0d699e2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d64151d80 0x7f0d640ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.767+0000 7f0d699e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640ca890 0x7f0d640c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.767+0000 7f0d699e2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d64155240 con 0x7f0d640c7f10 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.767+0000 7f0d699e2700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6406cb20 con 0x7f0d64151d80 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.767+0000 7f0d699e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d641556d0 con 0x7f0d640ca890 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.767+0000 7f0d62ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d64151d80 0x7f0d640ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.767+0000 7f0d62ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d64151d80 0x7f0d640ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46668/0 (socket says 172.21.15.31:46668) 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.767+0000 7f0d62ffd700 1 -- 172.21.15.31:0/2329339909 learned_addr learned my addr 172.21.15.31:0/2329339909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.768+0000 7f0d637fe700 1 --2- 172.21.15.31:0/2329339909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640ca890 0x7f0d640c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.768+0000 7f0d62ffd700 1 -- 172.21.15.31:0/2329339909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640ca890 msgr2=0x7f0d640c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.768+0000 7f0d62ffd700 1 --2- 172.21.15.31:0/2329339909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640ca890 0x7f0d640c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.768+0000 7f0d62ffd700 1 -- 172.21.15.31:0/2329339909 >> v1:172.21.15.59:6789/0 conn(0x7f0d640c7f10 legacy=0x7f0d640c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.768+0000 7f0d62ffd700 1 -- 172.21.15.31:0/2329339909 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d64154910 con 0x7f0d64151d80 2024-03-30T01:41:44.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.768+0000 7f0d62ffd700 1 --2- 172.21.15.31:0/2329339909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 0x7f0d640ca350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d540098f0 tx=0x7f0d54009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf859a013a701a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:44.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.768+0000 7f0d61ffb700 1 -- 172.21.15.31:0/2329339909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400b8a0 con 0x7f0d64151d80 2024-03-30T01:41:44.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.768+0000 7f0d61ffb700 1 -- 172.21.15.31:0/2329339909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5400ba00 con 0x7f0d64151d80 2024-03-30T01:41:44.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.768+0000 7f0d61ffb700 1 -- 172.21.15.31:0/2329339909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400bd90 con 0x7f0d64151d80 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.769+0000 7f0d699e2700 1 -- 172.21.15.31:0/2329339909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 msgr2=0x7f0d640ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.769+0000 7f0d699e2700 1 --2- 172.21.15.31:0/2329339909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 0x7f0d640ca350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d540098f0 tx=0x7f0d54009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.769+0000 7f0d699e2700 1 -- 172.21.15.31:0/2329339909 shutdown_connections 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.769+0000 7f0d699e2700 1 --2- 172.21.15.31:0/2329339909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 0x7f0d640ca350 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.769+0000 7f0d699e2700 1 --2- 172.21.15.31:0/2329339909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640ca890 0x7f0d640c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.769+0000 7f0d699e2700 1 -- 172.21.15.31:0/2329339909 >> 172.21.15.31:0/2329339909 conn(0x7f0d64072010 msgr2=0x7f0d64072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.769+0000 7f0d699e2700 1 -- 172.21.15.31:0/2329339909 shutdown_connections 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.769+0000 7f0d699e2700 1 -- 172.21.15.31:0/2329339909 wait complete. 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.770+0000 7f0d699e2700 1 Processor -- start 2024-03-30T01:41:44.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.770+0000 7f0d699e2700 1 -- start start 2024-03-30T01:41:44.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d699e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640c7f10 0x7f0d640ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d699e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d640ca890 0x7f0d6414a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d699e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 0x7f0d641f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d699e2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d64156dc0 con 0x7f0d64151d80 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d699e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d640c1c40 con 0x7f0d640ca890 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d699e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d64157060 con 0x7f0d640c7f10 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 0x7f0d641f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 0x7f0d641f17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46684/0 (socket says 172.21.15.31:46684) 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d637fe700 1 -- 172.21.15.31:0/279249835 learned_addr learned my addr 172.21.15.31:0/279249835 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d62ffd700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640c7f10 0x7f0d640ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d627fc700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d640ca890 0x7f0d6414a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d637fe700 1 -- 172.21.15.31:0/279249835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640c7f10 msgr2=0x7f0d640ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d637fe700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640c7f10 0x7f0d640ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d637fe700 1 -- 172.21.15.31:0/279249835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d640ca890 msgr2=0x7f0d6414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d637fe700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d640ca890 0x7f0d6414a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.771+0000 7f0d637fe700 1 -- 172.21.15.31:0/279249835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54009550 con 0x7f0d64151d80 2024-03-30T01:41:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.772+0000 7f0d627fc700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d640ca890 0x7f0d6414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:44.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.772+0000 7f0d637fe700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 0x7f0d641f17b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c007e30 tx=0x7f0d5c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:44.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.772+0000 7f0d4bfff700 1 -- 172.21.15.31:0/279249835 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c00ede0 con 0x7f0d64151d80 2024-03-30T01:41:44.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.772+0000 7f0d4bfff700 1 -- 172.21.15.31:0/279249835 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c014420 con 0x7f0d64151d80 2024-03-30T01:41:44.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.772+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d64157340 con 0x7f0d64151d80 2024-03-30T01:41:44.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.772+0000 7f0d4bfff700 1 -- 172.21.15.31:0/279249835 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c0147d0 con 0x7f0d64151d80 2024-03-30T01:41:44.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.772+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d640abc80 con 0x7f0d64151d80 2024-03-30T01:41:44.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.773+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d640747d0 con 0x7f0d64151d80 2024-03-30T01:41:44.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.773+0000 7f0d4bfff700 1 -- 172.21.15.31:0/279249835 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0d5c014930 con 0x7f0d64151d80 2024-03-30T01:41:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.774+0000 7f0d4bfff700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d5006ac90 0x7f0d5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.774+0000 7f0d4bfff700 1 -- 172.21.15.31:0/279249835 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0d5c089d80 con 0x7f0d64151d80 2024-03-30T01:41:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.774+0000 7f0d62ffd700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d5006ac90 0x7f0d5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.774+0000 7f0d62ffd700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d5006ac90 0x7f0d5006d150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0d54014040 tx=0x7f0d54008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:44.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.779+0000 7f0d4bfff700 1 -- 172.21.15.31:0/279249835 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0d5c054c30 con 0x7f0d64151d80 2024-03-30T01:41:44.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.992+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d6414f460 con 0x7f0d64151d80 2024-03-30T01:41:44.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.992+0000 7f0d4bfff700 1 -- 172.21.15.31:0/279249835 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d5c058250 con 0x7f0d64151d80 2024-03-30T01:41:44.998 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d5006ac90 msgr2=0x7f0d5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d5006ac90 0x7f0d5006d150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0d54014040 tx=0x7f0d54008030 comp rx=0 tx=0).stop 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 msgr2=0x7f0d641f17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 0x7f0d641f17b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c007e30 tx=0x7f0d5c00e3c0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 shutdown_connections 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d64151d80 0x7f0d641f17b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d5006ac90 0x7f0d5006d150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d640c7f10 0x7f0d640ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 --2- 172.21.15.31:0/279249835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d640ca890 0x7f0d6414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 >> 172.21.15.31:0/279249835 conn(0x7f0d64072010 msgr2=0x7f0d640c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 shutdown_connections 2024-03-30T01:41:45.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:44.995+0000 7f0d699e2700 1 -- 172.21.15.31:0/279249835 wait complete. 2024-03-30T01:41:45.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:45.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.148+0000 7f11fdabf700 1 Processor -- start 2024-03-30T01:41:45.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.148+0000 7f11fdabf700 1 -- start start 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.149+0000 7f11fdabf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f806f310 0x7f11f806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.149+0000 7f11fdabf700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80a8460 con 0x7f11f80c18f0 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.149+0000 7f11fdabf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80a86b0 con 0x7f11f80c2670 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.149+0000 7f11fdabf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81526a0 con 0x7f11f806f310 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.149+0000 7f11f6ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f11f80c18f0 0x7f11f8152dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53118/0 (socket says 172.21.15.31:53118) 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.149+0000 7f11f6ffd700 1 -- 172.21.15.31:0/762627942 learned_addr learned my addr 172.21.15.31:0/762627942 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.149+0000 7f11f77fe700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f806f310 0x7f11f806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.149+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079471244 0 0) 0x7f11f80a8460 con 0x7f11f80c18f0 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.149+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e00035b0 con 0x7f11f80c18f0 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4184784611 0 0) 0x7f11e00035b0 con 0x7f11f80c18f0 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f806f310 msgr2=0x7f11f806f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f806f310 0x7f11f806f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 >> v1:172.21.15.59:6789/0 conn(0x7f11f80c2670 legacy=0x7f11f815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80a8990 con 0x7f11f80c18f0 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f11ec001670 con 0x7f11f80c18f0 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 >> v1:172.21.15.31:6789/0 conn(0x7f11f80c18f0 legacy=0x7f11f8152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f806f310 0x7f11e0003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e0004d90 0x7f11e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81526a0 con 0x7f11e0004050 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e000bb50 con 0x7f11e0004d90 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e000bd10 con 0x7f11f806f310 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f6ffd700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f77fe700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f806f310 0x7f11e0003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f7fff700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e0004d90 0x7f11e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f6ffd700 1 -- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f806f310 msgr2=0x7f11e0003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f6ffd700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f806f310 0x7f11e0003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f6ffd700 1 -- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e0004d90 msgr2=0x7f11e0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f6ffd700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11e0004d90 0x7f11e0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f6ffd700 1 -- 172.21.15.31:0/762627942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e00035b0 con 0x7f11e0004050 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.150+0000 7f11f6ffd700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11e0004850 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11f80c1d30 tx=0x7f11e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfcc18aa21d97c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00cd40 con 0x7f11e0004050 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec005960 con 0x7f11e0004050 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11f67fc700 1 -- 172.21.15.31:0/762627942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec005d10 con 0x7f11e0004050 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11fdabf700 1 -- 172.21.15.31:0/762627942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 msgr2=0x7f11e0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11fdabf700 1 --2- 172.21.15.31:0/762627942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11e0004850 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11f80c1d30 tx=0x7f11e0005880 comp rx=0 tx=0).stop 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11f7fff700 1 -- 172.21.15.31:0/762627942 reap_dead start 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11fdabf700 1 -- 172.21.15.31:0/762627942 shutdown_connections 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11fdabf700 1 -- 172.21.15.31:0/762627942 >> 172.21.15.31:0/762627942 conn(0x7f11f80c3980 msgr2=0x7f11f80c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11fdabf700 1 -- 172.21.15.31:0/762627942 shutdown_connections 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.151+0000 7f11fdabf700 1 -- 172.21.15.31:0/762627942 wait complete. 2024-03-30T01:41:45.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.152+0000 7f11fdabf700 1 Processor -- start 2024-03-30T01:41:45.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.153+0000 7f11fdabf700 1 -- start start 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.153+0000 7f11fdabf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11f80c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.153+0000 7f11fdabf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11f806d550 0x7f11f80ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.153+0000 7f11fdabf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f814f520 0x7f11f814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.153+0000 7f11fdabf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80c6a40 con 0x7f11e0004050 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.153+0000 7f11fdabf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81ee2d0 con 0x7f11f806d550 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.153+0000 7f11fdabf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81ed6c0 con 0x7f11f814f520 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11f80c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11f80c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46704/0 (socket says 172.21.15.31:46704) 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f77fe700 1 -- 172.21.15.31:0/362563610 learned_addr learned my addr 172.21.15.31:0/362563610 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f6ffd700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11f806d550 0x7f11f80ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f7fff700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f814f520 0x7f11f814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f77fe700 1 -- 172.21.15.31:0/362563610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f814f520 msgr2=0x7f11f814f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f77fe700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f814f520 0x7f11f814f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f77fe700 1 -- 172.21.15.31:0/362563610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11f806d550 msgr2=0x7f11f80ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f77fe700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11f806d550 0x7f11f80ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f77fe700 1 -- 172.21.15.31:0/362563610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f81ed9a0 con 0x7f11e0004050 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f6ffd700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11f806d550 0x7f11f80ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11f77fe700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11f80c6500 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f11e8011ad0 tx=0x7f11e8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11dffff700 1 -- 172.21.15.31:0/362563610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800f9a0 con 0x7f11e0004050 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11dffff700 1 -- 172.21.15.31:0/362563610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800fb00 con 0x7f11e0004050 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f81edc80 con 0x7f11e0004050 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11dffff700 1 -- 172.21.15.31:0/362563610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800feb0 con 0x7f11e0004050 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.154+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f80abcb0 con 0x7f11e0004050 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.155+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f80747d0 con 0x7f11e0004050 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.156+0000 7f11dffff700 1 -- 172.21.15.31:0/362563610 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f11e8014440 con 0x7f11e0004050 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.156+0000 7f11dffff700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f11e406ac90 0x7f11e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.156+0000 7f11dffff700 1 -- 172.21.15.31:0/362563610 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f11e8089e80 con 0x7f11e0004050 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.156+0000 7f11f6ffd700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f11e406ac90 0x7f11e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.157+0000 7f11f6ffd700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f11e406ac90 0x7f11e406d150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f11f81dbcd0 tx=0x7f11ec007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.161+0000 7f11dffff700 1 -- 172.21.15.31:0/362563610 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f11e8054d50 con 0x7f11e0004050 2024-03-30T01:41:45.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.373+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11f80c2220 con 0x7f11e0004050 2024-03-30T01:41:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.374+0000 7f11dffff700 1 -- 172.21.15.31:0/362563610 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11e8058370 con 0x7f11e0004050 2024-03-30T01:41:45.379 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f11e406ac90 msgr2=0x7f11e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f11e406ac90 0x7f11e406d150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f11f81dbcd0 tx=0x7f11ec007000 comp rx=0 tx=0).stop 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 msgr2=0x7f11f80c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11f80c6500 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f11e8011ad0 tx=0x7f11e8011e90 comp rx=0 tx=0).stop 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 shutdown_connections 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11e0004050 0x7f11f80c6500 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f11e406ac90 0x7f11e406d150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11f814f520 0x7f11f814f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 --2- 172.21.15.31:0/362563610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11f806d550 0x7f11f80ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 >> 172.21.15.31:0/362563610 conn(0x7f11f80c3980 msgr2=0x7f11f806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 shutdown_connections 2024-03-30T01:41:45.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.376+0000 7f11fdabf700 1 -- 172.21.15.31:0/362563610 wait complete. 2024-03-30T01:41:45.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.528+0000 7f2f3f377700 1 Processor -- start 2024-03-30T01:41:45.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.528+0000 7f2f3f377700 1 -- start start 2024-03-30T01:41:45.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.529+0000 7f2f3f377700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2f3814fd40 0x7f2f38152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.529+0000 7f2f3f377700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f380704d0 con 0x7f2f380aada0 2024-03-30T01:41:45.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.529+0000 7f2f3f377700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f38070720 con 0x7f2f380c4ed0 2024-03-30T01:41:45.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.529+0000 7f2f3f377700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f380c6d30 con 0x7f2f3814fd40 2024-03-30T01:41:45.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.529+0000 7f2f3d113700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2f3814fd40 0x7f2f38152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.529+0000 7f2f3c912700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2f380aada0 0x7f2f380ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46264/0 (socket says 172.21.15.31:46264) 2024-03-30T01:41:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.529+0000 7f2f3c912700 1 -- 172.21.15.31:0/1086608479 learned_addr learned my addr 172.21.15.31:0/1086608479 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.530+0000 7f2f3d113700 1 -- 172.21.15.31:0/1086608479 >> v1:172.21.15.59:6789/0 conn(0x7f2f380c4ed0 legacy=0x7f2f380a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.530+0000 7f2f3d113700 1 -- 172.21.15.31:0/1086608479 >> v1:172.21.15.31:6789/0 conn(0x7f2f380aada0 legacy=0x7f2f380ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.530+0000 7f2f3d113700 1 -- 172.21.15.31:0/1086608479 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38070a00 con 0x7f2f3814fd40 2024-03-30T01:41:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.530+0000 7f2f3d113700 1 --2- 172.21.15.31:0/1086608479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814fd40 0x7f2f38152ed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009840 tx=0x7f2f34009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9574d8fc0029f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.530+0000 7f2f2ffff700 1 -- 172.21.15.31:0/1086608479 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34011830 con 0x7f2f3814fd40 2024-03-30T01:41:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.530+0000 7f2f2ffff700 1 -- 172.21.15.31:0/1086608479 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f34011990 con 0x7f2f3814fd40 2024-03-30T01:41:45.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.530+0000 7f2f2ffff700 1 -- 172.21.15.31:0/1086608479 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34011d20 con 0x7f2f3814fd40 2024-03-30T01:41:45.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.531+0000 7f2f3f377700 1 -- 172.21.15.31:0/1086608479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814fd40 msgr2=0x7f2f38152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.531+0000 7f2f3f377700 1 --2- 172.21.15.31:0/1086608479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814fd40 0x7f2f38152ed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009840 tx=0x7f2f34009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:45.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.531+0000 7f2f3f377700 1 -- 172.21.15.31:0/1086608479 shutdown_connections 2024-03-30T01:41:45.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.531+0000 7f2f3f377700 1 --2- 172.21.15.31:0/1086608479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814fd40 0x7f2f38152ed0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.531+0000 7f2f3f377700 1 -- 172.21.15.31:0/1086608479 >> 172.21.15.31:0/1086608479 conn(0x7f2f380c7cf0 msgr2=0x7f2f380c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:45.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.531+0000 7f2f3f377700 1 -- 172.21.15.31:0/1086608479 shutdown_connections 2024-03-30T01:41:45.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.531+0000 7f2f3f377700 1 -- 172.21.15.31:0/1086608479 wait complete. 2024-03-30T01:41:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.532+0000 7f2f3f377700 1 Processor -- start 2024-03-30T01:41:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.532+0000 7f2f3f377700 1 -- start start 2024-03-30T01:41:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3f377700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f380c4ed0 0x7f2f381dbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3f377700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f3814fd40 0x7f2f38156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3f377700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814c1f0 0x7f2f3814c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3f377700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f38155130 con 0x7f2f3814c1f0 2024-03-30T01:41:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3f377700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f381548e0 con 0x7f2f380c4ed0 2024-03-30T01:41:45.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3f377700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f38154b70 con 0x7f2f3814fd40 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d914700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814c1f0 0x7f2f3814c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d914700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814c1f0 0x7f2f3814c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55396/0 (socket says 172.21.15.31:55396) 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d914700 1 -- 172.21.15.31:0/3813656680 learned_addr learned my addr 172.21.15.31:0/3813656680 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d113700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f380c4ed0 0x7f2f381dbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3c912700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f3814fd40 0x7f2f38156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d914700 1 -- 172.21.15.31:0/3813656680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f3814fd40 msgr2=0x7f2f38156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d914700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f3814fd40 0x7f2f38156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d914700 1 -- 172.21.15.31:0/3813656680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f380c4ed0 msgr2=0x7f2f381dbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d914700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f380c4ed0 0x7f2f381dbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d914700 1 -- 172.21.15.31:0/3813656680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f340094a0 con 0x7f2f3814c1f0 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.533+0000 7f2f3d914700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814c1f0 0x7f2f3814c6a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000cc50 tx=0x7f2f3000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.534+0000 7f2f2dffb700 1 -- 172.21.15.31:0/3813656680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30003c00 con 0x7f2f3814c1f0 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.534+0000 7f2f2dffb700 1 -- 172.21.15.31:0/3813656680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30003d60 con 0x7f2f3814c1f0 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.534+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f38154e50 con 0x7f2f3814c1f0 2024-03-30T01:41:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.534+0000 7f2f2dffb700 1 -- 172.21.15.31:0/3813656680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3000aab0 con 0x7f2f3814c1f0 2024-03-30T01:41:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.534+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f381dc4d0 con 0x7f2f3814c1f0 2024-03-30T01:41:45.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.535+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f380a8da0 con 0x7f2f3814c1f0 2024-03-30T01:41:45.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.535+0000 7f2f2dffb700 1 -- 172.21.15.31:0/3813656680 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2f3000ac10 con 0x7f2f3814c1f0 2024-03-30T01:41:45.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.536+0000 7f2f2dffb700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f1c06acc0 0x7f2f1c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.536+0000 7f2f2dffb700 1 -- 172.21.15.31:0/3813656680 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2f300896c0 con 0x7f2f3814c1f0 2024-03-30T01:41:45.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.536+0000 7f2f3d113700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f1c06acc0 0x7f2f1c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.536+0000 7f2f3d113700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f1c06acc0 0x7f2f1c06d180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2f340090b0 tx=0x7f2f34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:45.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.540+0000 7f2f2dffb700 1 -- 172.21.15.31:0/3813656680 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2f380a8da0 con 0x7f2f3814c1f0 2024-03-30T01:41:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.753+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f380a9180 con 0x7f2f3814c1f0 2024-03-30T01:41:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.754+0000 7f2f2dffb700 1 -- 172.21.15.31:0/3813656680 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f380a9180 con 0x7f2f3814c1f0 2024-03-30T01:41:45.759 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f1c06acc0 msgr2=0x7f2f1c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f1c06acc0 0x7f2f1c06d180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2f340090b0 tx=0x7f2f34008030 comp rx=0 tx=0).stop 2024-03-30T01:41:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814c1f0 msgr2=0x7f2f3814c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814c1f0 0x7f2f3814c6a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000cc50 tx=0x7f2f3000cf60 comp rx=0 tx=0).stop 2024-03-30T01:41:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 shutdown_connections 2024-03-30T01:41:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3814c1f0 0x7f2f3814c6a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2f1c06acc0 0x7f2f1c06d180 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f380c4ed0 0x7f2f381dbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 --2- 172.21.15.31:0/3813656680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f3814fd40 0x7f2f38156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 >> 172.21.15.31:0/3813656680 conn(0x7f2f380c7cf0 msgr2=0x7f2f380c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:45.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 shutdown_connections 2024-03-30T01:41:45.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.756+0000 7f2f3f377700 1 -- 172.21.15.31:0/3813656680 wait complete. 2024-03-30T01:41:45.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:45.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.910+0000 7f5d9cb17700 1 Processor -- start 2024-03-30T01:41:45.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.910+0000 7f5d9cb17700 1 -- start start 2024-03-30T01:41:45.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9cb17700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d9814e9a0 0x7f5d9806e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9cb17700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d9806d340 0x7f5d9806e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9cb17700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5d98154930 0x7f5d98154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9cb17700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d980c2220 con 0x7f5d9814e9a0 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9cb17700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d980c31f0 con 0x7f5d98154930 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9cb17700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9814ed80 con 0x7f5d9806d340 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d9814e9a0 0x7f5d9806e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d9814e9a0 0x7f5d9806e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55406/0 (socket says 172.21.15.31:55406) 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9659c700 1 -- 172.21.15.31:0/3282263352 learned_addr learned my addr 172.21.15.31:0/3282263352 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d96d9d700 1 --2- 172.21.15.31:0/3282263352 >> v2:172.21.15.59:3300/0 conn(0x7f5d98154930 0x7f5d98154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d95d9b700 1 --2- 172.21.15.31:0/3282263352 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d9806d340 0x7f5d9806e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9659c700 1 -- 172.21.15.31:0/3282263352 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d9806d340 msgr2=0x7f5d9806e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9659c700 1 --2- 172.21.15.31:0/3282263352 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d9806d340 0x7f5d9806e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9659c700 1 -- 172.21.15.31:0/3282263352 >> v2:172.21.15.59:3300/0 conn(0x7f5d98154930 msgr2=0x7f5d98154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9659c700 1 --2- 172.21.15.31:0/3282263352 >> v2:172.21.15.59:3300/0 conn(0x7f5d98154930 0x7f5d98154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d9659c700 1 -- 172.21.15.31:0/3282263352 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d980a8790 con 0x7f5d9814e9a0 2024-03-30T01:41:45.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.911+0000 7f5d96d9d700 1 --2- 172.21.15.31:0/3282263352 >> v2:172.21.15.59:3300/0 conn(0x7f5d98154930 0x7f5d98154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:45.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9659c700 1 --2- 172.21.15.31:0/3282263352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9814e9a0 0x7f5d9806e050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d98072920 tx=0x7f5d80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9565c01e40efd97 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:45.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9559a700 1 -- 172.21.15.31:0/3282263352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000b870 con 0x7f5d9814e9a0 2024-03-30T01:41:45.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9559a700 1 -- 172.21.15.31:0/3282263352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8000b9d0 con 0x7f5d9814e9a0 2024-03-30T01:41:45.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9559a700 1 -- 172.21.15.31:0/3282263352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000bd60 con 0x7f5d9814e9a0 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3282263352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9814e9a0 msgr2=0x7f5d9806e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3282263352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9814e9a0 0x7f5d9806e050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d98072920 tx=0x7f5d80009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3282263352 shutdown_connections 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3282263352 >> v2:172.21.15.59:3300/0 conn(0x7f5d98154930 0x7f5d98154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3282263352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9814e9a0 0x7f5d9806e050 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3282263352 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d9806d340 0x7f5d9806e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.912+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3282263352 >> 172.21.15.31:0/3282263352 conn(0x7f5d980c3910 msgr2=0x7f5d980c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.913+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3282263352 shutdown_connections 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.913+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3282263352 wait complete. 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.914+0000 7f5d9cb17700 1 Processor -- start 2024-03-30T01:41:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.914+0000 7f5d9cb17700 1 -- start start 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9cb17700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9806d340 0x7f5d980c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9cb17700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9814e9a0 0x7f5d98148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9cb17700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d98154930 0x7f5d98148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9cb17700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981509c0 con 0x7f5d9806d340 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9cb17700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d980c5c10 con 0x7f5d9814e9a0 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9cb17700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d98151e60 con 0x7f5d98154930 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9806d340 0x7f5d980c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9806d340 0x7f5d980c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55416/0 (socket says 172.21.15.31:55416) 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9659c700 1 -- 172.21.15.31:0/3368497621 learned_addr learned my addr 172.21.15.31:0/3368497621 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d96d9d700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d98154930 0x7f5d98148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d95d9b700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9814e9a0 0x7f5d98148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9659c700 1 -- 172.21.15.31:0/3368497621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d98154930 msgr2=0x7f5d98148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9659c700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d98154930 0x7f5d98148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9659c700 1 -- 172.21.15.31:0/3368497621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9814e9a0 msgr2=0x7f5d98148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:45.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9659c700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9814e9a0 0x7f5d98148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:45.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9659c700 1 -- 172.21.15.31:0/3368497621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d80009550 con 0x7f5d9806d340 2024-03-30T01:41:45.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d9659c700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9806d340 0x7f5d980c6510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5d80000c00 tx=0x7f5d80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:45.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.915+0000 7f5d8effd700 1 -- 172.21.15.31:0/3368497621 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80004540 con 0x7f5d9806d340 2024-03-30T01:41:45.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.916+0000 7f5d8effd700 1 -- 172.21.15.31:0/3368497621 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d800036a0 con 0x7f5d9806d340 2024-03-30T01:41:45.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.916+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d98150c40 con 0x7f5d9806d340 2024-03-30T01:41:45.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.916+0000 7f5d8effd700 1 -- 172.21.15.31:0/3368497621 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80003a50 con 0x7f5d9806d340 2024-03-30T01:41:45.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.916+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d98151200 con 0x7f5d9806d340 2024-03-30T01:41:45.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.917+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d98066f90 con 0x7f5d9806d340 2024-03-30T01:41:45.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.917+0000 7f5d8effd700 1 -- 172.21.15.31:0/3368497621 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5d80021070 con 0x7f5d9806d340 2024-03-30T01:41:45.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.917+0000 7f5d8effd700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5d7806ac90 0x7f5d7806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:45.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.918+0000 7f5d8effd700 1 -- 172.21.15.31:0/3368497621 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5d8008ae90 con 0x7f5d9806d340 2024-03-30T01:41:45.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.918+0000 7f5d95d9b700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5d7806ac90 0x7f5d7806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:45.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.918+0000 7f5d95d9b700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5d7806ac90 0x7f5d7806d150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800c900 tx=0x7f5d88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:45.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:45.922+0000 7f5d8effd700 1 -- 172.21.15.31:0/3368497621 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5d80055e60 con 0x7f5d9806d340 2024-03-30T01:41:46.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.137+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5d980747d0 con 0x7f5d9806d340 2024-03-30T01:41:46.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.138+0000 7f5d8effd700 1 -- 172.21.15.31:0/3368497621 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5d80015020 con 0x7f5d9806d340 2024-03-30T01:41:46.146 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5d7806ac90 msgr2=0x7f5d7806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5d7806ac90 0x7f5d7806d150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800c900 tx=0x7f5d88008040 comp rx=0 tx=0).stop 2024-03-30T01:41:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9806d340 msgr2=0x7f5d980c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9806d340 0x7f5d980c6510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5d80000c00 tx=0x7f5d80004750 comp rx=0 tx=0).stop 2024-03-30T01:41:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 shutdown_connections 2024-03-30T01:41:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9806d340 0x7f5d980c6510 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5d7806ac90 0x7f5d7806d150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d98154930 0x7f5d98148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 --2- 172.21.15.31:0/3368497621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9814e9a0 0x7f5d98148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 >> 172.21.15.31:0/3368497621 conn(0x7f5d980c3910 msgr2=0x7f5d98155160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 shutdown_connections 2024-03-30T01:41:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.140+0000 7f5d9cb17700 1 -- 172.21.15.31:0/3368497621 wait complete. 2024-03-30T01:41:46.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent' 2024-03-30T01:41:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.293+0000 7f1cb0745700 1 Processor -- start 2024-03-30T01:41:46.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.293+0000 7f1cb0745700 1 -- start start 2024-03-30T01:41:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cb0745700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1ca80c3170 0x7f1ca8072460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cb0745700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1ca80c4430 0x7f1ca813d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cb0745700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca8158aa0 con 0x7f1ca813d940 2024-03-30T01:41:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cb0745700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca806f3a0 con 0x7f1ca80c3170 2024-03-30T01:41:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cb0745700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca8157e90 con 0x7f1ca80c4430 2024-03-30T01:41:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cae4e1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1ca80c3170 0x7f1ca8072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1caece2700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1ca813d940 0x7f1ca8154800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46278/0 (socket says 172.21.15.31:46278) 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1caece2700 1 -- 172.21.15.31:0/1820311293 learned_addr learned my addr 172.21.15.31:0/1820311293 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cae4e1700 1 --2- 172.21.15.31:0/1820311293 >> v2:172.21.15.31:3300/0 conn(0x7f1ca80c3170 0x7f1ca8072460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55420/0 (socket says 172.21.15.31:55420) 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cadce0700 1 --2- 172.21.15.31:0/1820311293 >> v2:172.21.15.59:3300/0 conn(0x7f1ca80c4430 0x7f1ca813d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cae4e1700 1 -- 172.21.15.31:0/1820311293 >> v2:172.21.15.59:3300/0 conn(0x7f1ca80c4430 msgr2=0x7f1ca813d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cae4e1700 1 --2- 172.21.15.31:0/1820311293 >> v2:172.21.15.59:3300/0 conn(0x7f1ca80c4430 0x7f1ca813d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.294+0000 7f1cae4e1700 1 -- 172.21.15.31:0/1820311293 >> v1:172.21.15.31:6789/0 conn(0x7f1ca813d940 legacy=0x7f1ca8154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.295+0000 7f1cae4e1700 1 -- 172.21.15.31:0/1820311293 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8158170 con 0x7f1ca80c3170 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.295+0000 7f1cad4df700 1 -- 172.21.15.31:0/1820311293 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725544952 0 0) 0x7f1ca8158aa0 con 0x7f1ca813d940 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.295+0000 7f1cadce0700 1 --2- 172.21.15.31:0/1820311293 >> v2:172.21.15.59:3300/0 conn(0x7f1ca80c4430 0x7f1ca813d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.295+0000 7f1cae4e1700 1 --2- 172.21.15.31:0/1820311293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca80c3170 0x7f1ca8072460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80c1b30 tx=0x7f1ca4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd72595986cad90 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.295+0000 7f1cad4df700 1 -- 172.21.15.31:0/1820311293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4011830 con 0x7f1ca80c3170 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.295+0000 7f1cad4df700 1 -- 172.21.15.31:0/1820311293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca4011990 con 0x7f1ca80c3170 2024-03-30T01:41:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.295+0000 7f1cad4df700 1 -- 172.21.15.31:0/1820311293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4011d20 con 0x7f1ca80c3170 2024-03-30T01:41:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.295+0000 7f1cb0745700 1 -- 172.21.15.31:0/1820311293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca80c3170 msgr2=0x7f1ca8072460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.295+0000 7f1cb0745700 1 --2- 172.21.15.31:0/1820311293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca80c3170 0x7f1ca8072460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80c1b30 tx=0x7f1ca4009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.296+0000 7f1cb0745700 1 -- 172.21.15.31:0/1820311293 shutdown_connections 2024-03-30T01:41:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.296+0000 7f1cb0745700 1 --2- 172.21.15.31:0/1820311293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca80c3170 0x7f1ca8072460 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.296+0000 7f1cb0745700 1 --2- 172.21.15.31:0/1820311293 >> v2:172.21.15.59:3300/0 conn(0x7f1ca80c4430 0x7f1ca813d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.296+0000 7f1cb0745700 1 -- 172.21.15.31:0/1820311293 >> 172.21.15.31:0/1820311293 conn(0x7f1ca80c3910 msgr2=0x7f1ca80c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.296+0000 7f1cb0745700 1 -- 172.21.15.31:0/1820311293 shutdown_connections 2024-03-30T01:41:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.296+0000 7f1cb0745700 1 -- 172.21.15.31:0/1820311293 wait complete. 2024-03-30T01:41:46.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.297+0000 7f1cb0745700 1 Processor -- start 2024-03-30T01:41:46.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.297+0000 7f1cb0745700 1 -- start start 2024-03-30T01:41:46.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1cb0745700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca80c3170 0x7f1ca80c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:46.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1cb0745700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca80c4430 0x7f1ca8138c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:46.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1cb0745700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca813d940 0x7f1ca8133510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:46.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1cb0745700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca80c5d30 con 0x7f1ca813d940 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1cb0745700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca80c5f80 con 0x7f1ca80c4430 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1cb0745700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca80c6260 con 0x7f1ca80c3170 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1caece2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca813d940 0x7f1ca8133510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1caece2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca813d940 0x7f1ca8133510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55434/0 (socket says 172.21.15.31:55434) 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1caece2700 1 -- 172.21.15.31:0/1521369121 learned_addr learned my addr 172.21.15.31:0/1521369121 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1cadce0700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca80c4430 0x7f1ca8138c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1cae4e1700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca80c3170 0x7f1ca80c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1caece2700 1 -- 172.21.15.31:0/1521369121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca80c3170 msgr2=0x7f1ca80c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1caece2700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca80c3170 0x7f1ca80c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1caece2700 1 -- 172.21.15.31:0/1521369121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca80c4430 msgr2=0x7f1ca8138c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1caece2700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca80c4430 0x7f1ca8138c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1caece2700 1 -- 172.21.15.31:0/1521369121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40094a0 con 0x7f1ca813d940 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1cae4e1700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca80c3170 0x7f1ca80c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:46.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.298+0000 7f1caece2700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca813d940 0x7f1ca8133510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000caf0 tx=0x7f1ca000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:46.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.299+0000 7f1c9affd700 1 -- 172.21.15.31:0/1521369121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000aaa0 con 0x7f1ca813d940 2024-03-30T01:41:46.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.299+0000 7f1c9affd700 1 -- 172.21.15.31:0/1521369121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca000ac20 con 0x7f1ca813d940 2024-03-30T01:41:46.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.299+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca8134630 con 0x7f1ca813d940 2024-03-30T01:41:46.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.299+0000 7f1c9affd700 1 -- 172.21.15.31:0/1521369121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00070d0 con 0x7f1ca813d940 2024-03-30T01:41:46.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.299+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca8139180 con 0x7f1ca813d940 2024-03-30T01:41:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.300+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca8134490 con 0x7f1ca813d940 2024-03-30T01:41:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.300+0000 7f1c9affd700 1 -- 172.21.15.31:0/1521369121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1ca0007230 con 0x7f1ca813d940 2024-03-30T01:41:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.301+0000 7f1c9affd700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1c8c06ac90 0x7f1c8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.301+0000 7f1c9affd700 1 -- 172.21.15.31:0/1521369121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1ca0089370 con 0x7f1ca813d940 2024-03-30T01:41:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.301+0000 7f1cae4e1700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1c8c06ac90 0x7f1c8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:46.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.301+0000 7f1cae4e1700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1c8c06ac90 0x7f1c8c06d150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40090b0 tx=0x7f1ca4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:46.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.305+0000 7f1c9affd700 1 -- 172.21.15.31:0/1521369121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1ca8134490 con 0x7f1ca813d940 2024-03-30T01:41:46.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.611+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]} v 0) v1 -- 0x7f1ca8134490 con 0x7f1ca813d940 2024-03-30T01:41:46.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.945+0000 7f1c9affd700 1 -- 172.21.15.31:0/1521369121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]}]=0 v65) v1 ==== 162+0+0 (secure 0 0 0) 0x7f1ca8134490 con 0x7f1ca813d940 2024-03-30T01:41:46.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1c8c06ac90 msgr2=0x7f1c8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:46.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1c8c06ac90 0x7f1c8c06d150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40090b0 tx=0x7f1ca4008030 comp rx=0 tx=0).stop 2024-03-30T01:41:46.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca813d940 msgr2=0x7f1ca8133510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:46.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca813d940 0x7f1ca8133510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000caf0 tx=0x7f1ca000ce00 comp rx=0 tx=0).stop 2024-03-30T01:41:46.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 shutdown_connections 2024-03-30T01:41:46.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca813d940 0x7f1ca8133510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1c8c06ac90 0x7f1c8c06d150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca80c3170 0x7f1ca80c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 --2- 172.21.15.31:0/1521369121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca80c4430 0x7f1ca8138c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:46.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 >> 172.21.15.31:0/1521369121 conn(0x7f1ca80c3910 msgr2=0x7f1ca81503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:46.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 shutdown_connections 2024-03-30T01:41:46.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:46.948+0000 7f1cb0745700 1 -- 172.21.15.31:0/1521369121 wait complete. 2024-03-30T01:41:46.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T01:41:47.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.107+0000 7f2b440c3700 1 Processor -- start 2024-03-30T01:41:47.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.107+0000 7f2b440c3700 1 -- start start 2024-03-30T01:41:47.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.108+0000 7f2b440c3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2b3c072c00 0x7f2b3c0c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.108+0000 7f2b440c3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c0c2be0 0x7f2b3c0c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.108+0000 7f2b440c3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3c06ded0 con 0x7f2b3c072820 2024-03-30T01:41:47.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.108+0000 7f2b440c3700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3c06e1a0 con 0x7f2b3c072c00 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.108+0000 7f2b440c3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c158e40 con 0x7f2b3c0c2be0 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.108+0000 7f2b4165e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2b3c072c00 0x7f2b3c0c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.108+0000 7f2b4165e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2b3c072c00 0x7f2b3c0c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55446/0 (socket says 172.21.15.31:55446) 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.108+0000 7f2b4165e700 1 -- 172.21.15.31:0/2387522465 learned_addr learned my addr 172.21.15.31:0/2387522465 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.108+0000 7f2b42660700 1 --2- 172.21.15.31:0/2387522465 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c0c2be0 0x7f2b3c0c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.109+0000 7f2b4165e700 1 -- 172.21.15.31:0/2387522465 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c0c2be0 msgr2=0x7f2b3c0c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.109+0000 7f2b4165e700 1 --2- 172.21.15.31:0/2387522465 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c0c2be0 0x7f2b3c0c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.109+0000 7f2b4165e700 1 -- 172.21.15.31:0/2387522465 >> v1:172.21.15.59:6789/0 conn(0x7f2b3c072820 legacy=0x7f2b3c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.109+0000 7f2b4165e700 1 -- 172.21.15.31:0/2387522465 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c159120 con 0x7f2b3c072c00 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.109+0000 7f2b4165e700 1 --2- 172.21.15.31:0/2387522465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072c00 0x7f2b3c0c26a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b380098f0 tx=0x7f2b38009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe188eb18421ea95 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.109+0000 7f2b40e5d700 1 -- 172.21.15.31:0/2387522465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38011830 con 0x7f2b3c072c00 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.109+0000 7f2b40e5d700 1 -- 172.21.15.31:0/2387522465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b38011990 con 0x7f2b3c072c00 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.109+0000 7f2b40e5d700 1 -- 172.21.15.31:0/2387522465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38011d20 con 0x7f2b3c072c00 2024-03-30T01:41:47.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.110+0000 7f2b440c3700 1 -- 172.21.15.31:0/2387522465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072c00 msgr2=0x7f2b3c0c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.110+0000 7f2b440c3700 1 --2- 172.21.15.31:0/2387522465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072c00 0x7f2b3c0c26a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b380098f0 tx=0x7f2b38009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.110+0000 7f2b440c3700 1 -- 172.21.15.31:0/2387522465 shutdown_connections 2024-03-30T01:41:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.110+0000 7f2b440c3700 1 --2- 172.21.15.31:0/2387522465 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c0c2be0 0x7f2b3c0c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.110+0000 7f2b440c3700 1 --2- 172.21.15.31:0/2387522465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072c00 0x7f2b3c0c26a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.110+0000 7f2b440c3700 1 -- 172.21.15.31:0/2387522465 >> 172.21.15.31:0/2387522465 conn(0x7f2b3c0c65c0 msgr2=0x7f2b3c0c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.110+0000 7f2b440c3700 1 -- 172.21.15.31:0/2387522465 shutdown_connections 2024-03-30T01:41:47.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.110+0000 7f2b440c3700 1 -- 172.21.15.31:0/2387522465 wait complete. 2024-03-30T01:41:47.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.111+0000 7f2b440c3700 1 Processor -- start 2024-03-30T01:41:47.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.112+0000 7f2b440c3700 1 -- start start 2024-03-30T01:41:47.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.112+0000 7f2b440c3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072820 0x7f2b3c0bb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.112+0000 7f2b440c3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c072c00 0x7f2b3c1eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.112+0000 7f2b440c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b3c0c2be0 0x7f2b3c15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.112+0000 7f2b440c3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c1ef080 con 0x7f2b3c072820 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.112+0000 7f2b440c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0ab6f0 con 0x7f2b3c0c2be0 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.112+0000 7f2b440c3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c0ab940 con 0x7f2b3c072c00 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b41e5f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072820 0x7f2b3c0bb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b41e5f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072820 0x7f2b3c0bb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55452/0 (socket says 172.21.15.31:55452) 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b41e5f700 1 -- 172.21.15.31:0/2875997660 learned_addr learned my addr 172.21.15.31:0/2875997660 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b4165e700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c072c00 0x7f2b3c1eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b42660700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b3c0c2be0 0x7f2b3c15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b41e5f700 1 -- 172.21.15.31:0/2875997660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c072c00 msgr2=0x7f2b3c1eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b41e5f700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c072c00 0x7f2b3c1eeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b41e5f700 1 -- 172.21.15.31:0/2875997660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b3c0c2be0 msgr2=0x7f2b3c15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b41e5f700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b3c0c2be0 0x7f2b3c15c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b41e5f700 1 -- 172.21.15.31:0/2875997660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b38009550 con 0x7f2b3c072820 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b41e5f700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072820 0x7f2b3c0bb860 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c4e0 tx=0x7f2b3000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:47.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b2e7fc700 1 -- 172.21.15.31:0/2875997660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300052d0 con 0x7f2b3c072820 2024-03-30T01:41:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b2e7fc700 1 -- 172.21.15.31:0/2875997660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b30005430 con 0x7f2b3c072820 2024-03-30T01:41:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b2e7fc700 1 -- 172.21.15.31:0/2875997660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300038c0 con 0x7f2b3c072820 2024-03-30T01:41:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3c0abc20 con 0x7f2b3c072820 2024-03-30T01:41:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.113+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3c1f16b0 con 0x7f2b3c072820 2024-03-30T01:41:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.114+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3c066d90 con 0x7f2b3c072820 2024-03-30T01:41:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.115+0000 7f2b2e7fc700 1 -- 172.21.15.31:0/2875997660 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2b30003a20 con 0x7f2b3c072820 2024-03-30T01:41:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.115+0000 7f2b2e7fc700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2b2006ac90 0x7f2b2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.115+0000 7f2b2e7fc700 1 -- 172.21.15.31:0/2875997660 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2b30088bb0 con 0x7f2b3c072820 2024-03-30T01:41:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.116+0000 7f2b4165e700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2b2006ac90 0x7f2b2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.116+0000 7f2b4165e700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2b2006ac90 0x7f2b2006d150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c156900 tx=0x7f2b38008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:47.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.120+0000 7f2b2e7fc700 1 -- 172.21.15.31:0/2875997660 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2b300539d0 con 0x7f2b3c072820 2024-03-30T01:41:47.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.333+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2b3c0747d0 con 0x7f2b3c072820 2024-03-30T01:41:47.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.337+0000 7f2b2e7fc700 1 -- 172.21.15.31:0/2875997660 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b300170c0 con 0x7f2b3c072820 2024-03-30T01:41:47.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.339+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2b2006ac90 msgr2=0x7f2b2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.339+0000 7f2b440c3700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2b2006ac90 0x7f2b2006d150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c156900 tx=0x7f2b38008030 comp rx=0 tx=0).stop 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.339+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072820 msgr2=0x7f2b3c0bb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.339+0000 7f2b440c3700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072820 0x7f2b3c0bb860 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c4e0 tx=0x7f2b3000c8a0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.340+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 shutdown_connections 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.340+0000 7f2b440c3700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c072820 0x7f2b3c0bb860 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.340+0000 7f2b440c3700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2b2006ac90 0x7f2b2006d150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.340+0000 7f2b440c3700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b3c0c2be0 0x7f2b3c15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.340+0000 7f2b440c3700 1 --2- 172.21.15.31:0/2875997660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b3c072c00 0x7f2b3c1eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.340+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 >> 172.21.15.31:0/2875997660 conn(0x7f2b3c0c65c0 msgr2=0x7f2b3c1582d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.340+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 shutdown_connections 2024-03-30T01:41:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.340+0000 7f2b440c3700 1 -- 172.21.15.31:0/2875997660 wait complete. 2024-03-30T01:41:47.359 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T01:41:47.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T01:41:47.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T01:41:47.452 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T01:41:47.453 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:41:47.453 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:47.474 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:41:47.474 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:47.536 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:41:47.536 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:47.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:47.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.777+0000 7f305268c700 1 Processor -- start 2024-03-30T01:41:47.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.777+0000 7f305268c700 1 -- start start 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f305268c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c13da70 0x7f304c0c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f305268c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f304c0722b0 0x7f304c0c3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f305268c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c154b30 con 0x7f304c06b110 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f305268c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c1554d0 con 0x7f304c0722b0 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f305268c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c154e10 con 0x7f304c13da70 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f3050c29700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f304c0722b0 0x7f304c0c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f304b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f304c06b110 0x7f304c0c0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46280/0 (socket says 172.21.15.31:46280) 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f304b7fe700 1 -- 172.21.15.31:0/166386923 learned_addr learned my addr 172.21.15.31:0/166386923 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f304bfff700 1 --2- 172.21.15.31:0/166386923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c13da70 0x7f304c0c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f3050c29700 1 -- 172.21.15.31:0/166386923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c13da70 msgr2=0x7f304c0c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f3050c29700 1 --2- 172.21.15.31:0/166386923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c13da70 0x7f304c0c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f3050c29700 1 -- 172.21.15.31:0/166386923 >> v1:172.21.15.31:6789/0 conn(0x7f304c06b110 legacy=0x7f304c0c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f3050c29700 1 -- 172.21.15.31:0/166386923 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c1550f0 con 0x7f304c0722b0 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f304affd700 1 -- 172.21.15.31:0/166386923 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449001360 0 0) 0x7f304c154b30 con 0x7f304c06b110 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.778+0000 7f3050c29700 1 --2- 172.21.15.31:0/166386923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c0722b0 0x7f304c0c3310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3044009d90 tx=0x7f304400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7147797651d27c32 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f304affd700 1 -- 172.21.15.31:0/166386923 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304400e070 con 0x7f304c0722b0 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f304affd700 1 -- 172.21.15.31:0/166386923 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304400bbd0 con 0x7f304c0722b0 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f304affd700 1 -- 172.21.15.31:0/166386923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044012430 con 0x7f304c0722b0 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f305268c700 1 -- 172.21.15.31:0/166386923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c0722b0 msgr2=0x7f304c0c3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f305268c700 1 --2- 172.21.15.31:0/166386923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c0722b0 0x7f304c0c3310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3044009d90 tx=0x7f304400b330 comp rx=0 tx=0).stop 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f305268c700 1 -- 172.21.15.31:0/166386923 shutdown_connections 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f305268c700 1 --2- 172.21.15.31:0/166386923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c0722b0 0x7f304c0c3310 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f305268c700 1 --2- 172.21.15.31:0/166386923 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c13da70 0x7f304c0c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f305268c700 1 -- 172.21.15.31:0/166386923 >> 172.21.15.31:0/166386923 conn(0x7f304c0c3a60 msgr2=0x7f304c0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f305268c700 1 -- 172.21.15.31:0/166386923 shutdown_connections 2024-03-30T01:41:47.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.779+0000 7f305268c700 1 -- 172.21.15.31:0/166386923 wait complete. 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.780+0000 7f305268c700 1 Processor -- start 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.781+0000 7f305268c700 1 -- start start 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.781+0000 7f305268c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304c06b110 0x7f304c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.781+0000 7f305268c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c0722b0 0x7f304c0abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.781+0000 7f305268c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13da70 0x7f304c0ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.781+0000 7f305268c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1eadf0 con 0x7f304c13da70 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.781+0000 7f305268c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0c5690 con 0x7f304c06b110 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.781+0000 7f305268c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0a8790 con 0x7f304c0722b0 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3050c29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13da70 0x7f304c0ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3050c29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13da70 0x7f304c0ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55468/0 (socket says 172.21.15.31:55468) 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3050c29700 1 -- 172.21.15.31:0/2782364039 learned_addr learned my addr 172.21.15.31:0/2782364039 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f304b7fe700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c0722b0 0x7f304c0abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f304bfff700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304c06b110 0x7f304c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3050c29700 1 -- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c0722b0 msgr2=0x7f304c0abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3050c29700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c0722b0 0x7f304c0abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3050c29700 1 -- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304c06b110 msgr2=0x7f304c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3050c29700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304c06b110 0x7f304c155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3050c29700 1 -- 172.21.15.31:0/2782364039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044009a70 con 0x7f304c13da70 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f304bfff700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304c06b110 0x7f304c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f304b7fe700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c0722b0 0x7f304c0abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3050c29700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13da70 0x7f304c0ac1c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f304400b730 tx=0x7f3044004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3048ff9700 1 -- 172.21.15.31:0/2782364039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044012430 con 0x7f304c13da70 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3048ff9700 1 -- 172.21.15.31:0/2782364039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3044020070 con 0x7f304c13da70 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c0c4cf0 con 0x7f304c13da70 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f3048ff9700 1 -- 172.21.15.31:0/2782364039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044004280 con 0x7f304c13da70 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.782+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c0c52b0 con 0x7f304c13da70 2024-03-30T01:41:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.783+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c0c57d0 con 0x7f304c13da70 2024-03-30T01:41:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.784+0000 7f3048ff9700 1 -- 172.21.15.31:0/2782364039 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f30440043e0 con 0x7f304c13da70 2024-03-30T01:41:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.784+0000 7f3048ff9700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f303806ac90 0x7f303806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.784+0000 7f3048ff9700 1 -- 172.21.15.31:0/2782364039 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3044022070 con 0x7f304c13da70 2024-03-30T01:41:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.784+0000 7f304bfff700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f303806ac90 0x7f303806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:47.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.785+0000 7f304bfff700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f303806ac90 0x7f303806d150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f303c009820 tx=0x7f303c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:47.789+0000 7f3048ff9700 1 -- 172.21.15.31:0/2782364039 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f304c0c57d0 con 0x7f304c13da70 2024-03-30T01:41:48.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.004+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f304c0c57d0 con 0x7f304c13da70 2024-03-30T01:41:48.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.006+0000 7f3048ff9700 1 -- 172.21.15.31:0/2782364039 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18219 (secure 0 0 0) 0x7f304c0c57d0 con 0x7f304c13da70 2024-03-30T01:41:48.011 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:48.011 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:42.922828+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f303806ac90 msgr2=0x7f303806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f303806ac90 0x7f303806d150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f303c009820 tx=0x7f303c008040 comp rx=0 tx=0).stop 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13da70 msgr2=0x7f304c0ac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13da70 0x7f304c0ac1c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f304400b730 tx=0x7f3044004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 shutdown_connections 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f304c13da70 0x7f304c0ac1c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f303806ac90 0x7f303806d150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f304c06b110 0x7f304c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 --2- 172.21.15.31:0/2782364039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f304c0722b0 0x7f304c0abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 >> 172.21.15.31:0/2782364039 conn(0x7f304c0c3a60 msgr2=0x7f304c13a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 shutdown_connections 2024-03-30T01:41:48.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.008+0000 7f305268c700 1 -- 172.21.15.31:0/2782364039 wait complete. 2024-03-30T01:41:48.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:48.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:48.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.161+0000 7fad5fd93700 1 Processor -- start 2024-03-30T01:41:48.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.161+0000 7fad5fd93700 1 -- start start 2024-03-30T01:41:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.162+0000 7fad5fd93700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fad5806a700 0x7fad5806aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.162+0000 7fad5fd93700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad5806b330 0x7fad581594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.162+0000 7fad5fd93700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5806fb90 con 0x7fad580c1da0 2024-03-30T01:41:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.162+0000 7fad5fd93700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5806e9a0 con 0x7fad5806b330 2024-03-30T01:41:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.162+0000 7fad5fd93700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad58159a00 con 0x7fad5806a700 2024-03-30T01:41:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.162+0000 7fad5d32e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad5806b330 0x7fad581594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.162+0000 7fad5e330700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fad580c1da0 0x7fad580c2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46288/0 (socket says 172.21.15.31:46288) 2024-03-30T01:41:48.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.162+0000 7fad5e330700 1 -- 172.21.15.31:0/312926117 learned_addr learned my addr 172.21.15.31:0/312926117 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5db2f700 1 --2- 172.21.15.31:0/312926117 >> v2:172.21.15.59:3300/0 conn(0x7fad5806a700 0x7fad5806aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5d32e700 1 -- 172.21.15.31:0/312926117 >> v2:172.21.15.59:3300/0 conn(0x7fad5806a700 msgr2=0x7fad5806aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5d32e700 1 --2- 172.21.15.31:0/312926117 >> v2:172.21.15.59:3300/0 conn(0x7fad5806a700 0x7fad5806aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5d32e700 1 -- 172.21.15.31:0/312926117 >> v1:172.21.15.31:6789/0 conn(0x7fad580c1da0 legacy=0x7fad580c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5d32e700 1 -- 172.21.15.31:0/312926117 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580c2de0 con 0x7fad5806b330 2024-03-30T01:41:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5cb2d700 1 -- 172.21.15.31:0/312926117 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494112733 0 0) 0x7fad5806fb90 con 0x7fad580c1da0 2024-03-30T01:41:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5d32e700 1 --2- 172.21.15.31:0/312926117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5806b330 0x7fad581594c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fad5806c7f0 tx=0x7fad54009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f21f4869ff28815 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5cb2d700 1 -- 172.21.15.31:0/312926117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54011830 con 0x7fad5806b330 2024-03-30T01:41:48.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5cb2d700 1 -- 172.21.15.31:0/312926117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad54011990 con 0x7fad5806b330 2024-03-30T01:41:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.163+0000 7fad5cb2d700 1 -- 172.21.15.31:0/312926117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54011d20 con 0x7fad5806b330 2024-03-30T01:41:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.164+0000 7fad5fd93700 1 -- 172.21.15.31:0/312926117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5806b330 msgr2=0x7fad581594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.164+0000 7fad5fd93700 1 --2- 172.21.15.31:0/312926117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5806b330 0x7fad581594c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fad5806c7f0 tx=0x7fad54009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.164+0000 7fad5fd93700 1 -- 172.21.15.31:0/312926117 shutdown_connections 2024-03-30T01:41:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.164+0000 7fad5fd93700 1 --2- 172.21.15.31:0/312926117 >> v2:172.21.15.59:3300/0 conn(0x7fad5806a700 0x7fad5806aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.164+0000 7fad5fd93700 1 --2- 172.21.15.31:0/312926117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad5806b330 0x7fad581594c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.164+0000 7fad5fd93700 1 -- 172.21.15.31:0/312926117 >> 172.21.15.31:0/312926117 conn(0x7fad580c3980 msgr2=0x7fad580c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.164+0000 7fad5fd93700 1 -- 172.21.15.31:0/312926117 shutdown_connections 2024-03-30T01:41:48.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.164+0000 7fad5fd93700 1 -- 172.21.15.31:0/312926117 wait complete. 2024-03-30T01:41:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.165+0000 7fad5fd93700 1 Processor -- start 2024-03-30T01:41:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.165+0000 7fad5fd93700 1 -- start start 2024-03-30T01:41:48.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5fd93700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad5806a700 0x7fad58158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5fd93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad5806b330 0x7fad5815c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5fd93700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad580c1da0 0x7fad5814ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5fd93700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad581f1650 con 0x7fad580c1da0 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5fd93700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad581f1930 con 0x7fad5806b330 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5fd93700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad581f1c10 con 0x7fad5806a700 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5e330700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad580c1da0 0x7fad5814ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5e330700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad580c1da0 0x7fad5814ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55494/0 (socket says 172.21.15.31:55494) 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5e330700 1 -- 172.21.15.31:0/364617837 learned_addr learned my addr 172.21.15.31:0/364617837 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5d32e700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad5806b330 0x7fad5815c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5db2f700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad5806a700 0x7fad58158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5e330700 1 -- 172.21.15.31:0/364617837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad5806a700 msgr2=0x7fad58158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5e330700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad5806a700 0x7fad58158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5e330700 1 -- 172.21.15.31:0/364617837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad5806b330 msgr2=0x7fad5815c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5e330700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad5806b330 0x7fad5815c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.166+0000 7fad5e330700 1 -- 172.21.15.31:0/364617837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540094a0 con 0x7fad580c1da0 2024-03-30T01:41:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.167+0000 7fad5e330700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad580c1da0 0x7fad5814ea70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fad5000cba0 tx=0x7fad5000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.167+0000 7fad4a7fc700 1 -- 172.21.15.31:0/364617837 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5000aaa0 con 0x7fad580c1da0 2024-03-30T01:41:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.167+0000 7fad4a7fc700 1 -- 172.21.15.31:0/364617837 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5000ac20 con 0x7fad580c1da0 2024-03-30T01:41:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.167+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad581f1ef0 con 0x7fad580c1da0 2024-03-30T01:41:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.167+0000 7fad4a7fc700 1 -- 172.21.15.31:0/364617837 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500070d0 con 0x7fad580c1da0 2024-03-30T01:41:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.167+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad581585b0 con 0x7fad580c1da0 2024-03-30T01:41:48.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.168+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad581f2410 con 0x7fad580c1da0 2024-03-30T01:41:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.169+0000 7fad4a7fc700 1 -- 172.21.15.31:0/364617837 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fad50007230 con 0x7fad580c1da0 2024-03-30T01:41:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.169+0000 7fad4a7fc700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fad3c06ac90 0x7fad3c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.169+0000 7fad4a7fc700 1 -- 172.21.15.31:0/364617837 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fad50089370 con 0x7fad580c1da0 2024-03-30T01:41:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.169+0000 7fad5db2f700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fad3c06ac90 0x7fad3c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.170+0000 7fad5db2f700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fad3c06ac90 0x7fad3c06d150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fad4c006b20 tx=0x7fad4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:48.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.174+0000 7fad4a7fc700 1 -- 172.21.15.31:0/364617837 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fad581f2410 con 0x7fad580c1da0 2024-03-30T01:41:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.388+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad581f2410 con 0x7fad580c1da0 2024-03-30T01:41:48.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.389+0000 7fad4a7fc700 1 -- 172.21.15.31:0/364617837 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad581f2410 con 0x7fad580c1da0 2024-03-30T01:41:48.394 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fad3c06ac90 msgr2=0x7fad3c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fad3c06ac90 0x7fad3c06d150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fad4c006b20 tx=0x7fad4c008040 comp rx=0 tx=0).stop 2024-03-30T01:41:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad580c1da0 msgr2=0x7fad5814ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad580c1da0 0x7fad5814ea70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fad5000cba0 tx=0x7fad5000ceb0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 shutdown_connections 2024-03-30T01:41:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad580c1da0 0x7fad5814ea70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fad3c06ac90 0x7fad3c06d150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad5806a700 0x7fad58158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 --2- 172.21.15.31:0/364617837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad5806b330 0x7fad5815c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 >> 172.21.15.31:0/364617837 conn(0x7fad580c3980 msgr2=0x7fad581523c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 shutdown_connections 2024-03-30T01:41:48.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.391+0000 7fad5fd93700 1 -- 172.21.15.31:0/364617837 wait complete. 2024-03-30T01:41:48.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:48.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.544+0000 7fa9809b8700 1 Processor -- start 2024-03-30T01:41:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.544+0000 7fa9809b8700 1 -- start start 2024-03-30T01:41:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.545+0000 7fa9809b8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa97c0c49f0 0x7fa97c0a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.545+0000 7fa9809b8700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97c0c26f0 con 0x7fa97c0c3140 2024-03-30T01:41:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.545+0000 7fa9809b8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97c070740 con 0x7fa97c0c5d10 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.545+0000 7fa9809b8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97c0c7bb0 con 0x7fa97c0c49f0 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.545+0000 7fa97ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa97c0c3140 0x7fa97c1584f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46298/0 (socket says 172.21.15.31:46298) 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.545+0000 7fa97ad9d700 1 -- 172.21.15.31:0/1605498406 learned_addr learned my addr 172.21.15.31:0/1605498406 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.545+0000 7fa97a59c700 1 --2- 172.21.15.31:0/1605498406 >> v2:172.21.15.59:3300/0 conn(0x7fa97c0c49f0 0x7fa97c0a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97959a700 1 -- 172.21.15.31:0/1605498406 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835350972 0 0) 0x7fa97c0c26f0 con 0x7fa97c0c3140 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97959a700 1 -- 172.21.15.31:0/1605498406 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9680035b0 con 0x7fa97c0c3140 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97959a700 1 -- 172.21.15.31:0/1605498406 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631656906 0 0) 0x7fa97c070740 con 0x7fa97c0c5d10 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97959a700 1 -- 172.21.15.31:0/1605498406 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa97c0c26f0 con 0x7fa97c0c5d10 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97a59c700 1 -- 172.21.15.31:0/1605498406 >> v1:172.21.15.59:6789/0 conn(0x7fa97c0c5d10 legacy=0x7fa97c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97a59c700 1 -- 172.21.15.31:0/1605498406 >> v1:172.21.15.31:6789/0 conn(0x7fa97c0c3140 legacy=0x7fa97c1584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97959a700 1 -- 172.21.15.31:0/1605498406 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 183437899 0 0) 0x7fa9680035b0 con 0x7fa97c0c3140 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97a59c700 1 -- 172.21.15.31:0/1605498406 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c070a20 con 0x7fa97c0c49f0 2024-03-30T01:41:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97a59c700 1 --2- 172.21.15.31:0/1605498406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa97c0c49f0 0x7fa97c0a6d70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009840 tx=0x7fa96c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f0e7d7be47fb9a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97959a700 1 -- 172.21.15.31:0/1605498406 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740019f0 con 0x7fa97c0c49f0 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97959a700 1 -- 172.21.15.31:0/1605498406 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c004030 con 0x7fa97c0c49f0 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.546+0000 7fa97959a700 1 -- 172.21.15.31:0/1605498406 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0043c0 con 0x7fa97c0c49f0 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.547+0000 7fa9809b8700 1 -- 172.21.15.31:0/1605498406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa97c0c49f0 msgr2=0x7fa97c0a6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.547+0000 7fa9809b8700 1 --2- 172.21.15.31:0/1605498406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa97c0c49f0 0x7fa97c0a6d70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009840 tx=0x7fa96c009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.547+0000 7fa9809b8700 1 -- 172.21.15.31:0/1605498406 shutdown_connections 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.547+0000 7fa9809b8700 1 --2- 172.21.15.31:0/1605498406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa97c0c49f0 0x7fa97c0a6d70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.547+0000 7fa9809b8700 1 -- 172.21.15.31:0/1605498406 >> 172.21.15.31:0/1605498406 conn(0x7fa97c0c7cf0 msgr2=0x7fa97c0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.547+0000 7fa9809b8700 1 -- 172.21.15.31:0/1605498406 shutdown_connections 2024-03-30T01:41:48.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.547+0000 7fa9809b8700 1 -- 172.21.15.31:0/1605498406 wait complete. 2024-03-30T01:41:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.548+0000 7fa9809b8700 1 Processor -- start 2024-03-30T01:41:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.548+0000 7fa9809b8700 1 -- start start 2024-03-30T01:41:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.549+0000 7fa9809b8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa97c0c3140 0x7fa97c0a8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.549+0000 7fa9809b8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa97c0c49f0 0x7fa97c148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.549+0000 7fa9809b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c0c5d10 0x7fa97c1492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.549+0000 7fa9809b8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c14f3a0 con 0x7fa97c0c5d10 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.549+0000 7fa9809b8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c152d80 con 0x7fa97c0c3140 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.549+0000 7fa9809b8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c14d440 con 0x7fa97c0c49f0 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.549+0000 7fa97ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c0c5d10 0x7fa97c1492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.549+0000 7fa97ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c0c5d10 0x7fa97c1492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55496/0 (socket says 172.21.15.31:55496) 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.549+0000 7fa97ad9d700 1 -- 172.21.15.31:0/2376381519 learned_addr learned my addr 172.21.15.31:0/2376381519 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa97a59c700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa97c0c3140 0x7fa97c0a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa979d9b700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa97c0c49f0 0x7fa97c148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa97ad9d700 1 -- 172.21.15.31:0/2376381519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa97c0c49f0 msgr2=0x7fa97c148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa97ad9d700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa97c0c49f0 0x7fa97c148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa97ad9d700 1 -- 172.21.15.31:0/2376381519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa97c0c3140 msgr2=0x7fa97c0a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa97ad9d700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa97c0c3140 0x7fa97c0a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa97ad9d700 1 -- 172.21.15.31:0/2376381519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0094a0 con 0x7fa97c0c5d10 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa979d9b700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa97c0c49f0 0x7fa97c148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa97ad9d700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c0c5d10 0x7fa97c1492f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa97400cc50 tx=0x7fa97400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa972ffd700 1 -- 172.21.15.31:0/2376381519 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400ab80 con 0x7fa97c0c5d10 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa972ffd700 1 -- 172.21.15.31:0/2376381519 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97400ace0 con 0x7fa97c0c5d10 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97c06ef00 con 0x7fa97c0c5d10 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa972ffd700 1 -- 172.21.15.31:0/2376381519 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974007310 con 0x7fa97c0c5d10 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.550+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97c14d6e0 con 0x7fa97c0c5d10 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.551+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97c0747d0 con 0x7fa97c0c5d10 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.552+0000 7fa972ffd700 1 -- 172.21.15.31:0/2376381519 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa974007470 con 0x7fa97c0c5d10 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.552+0000 7fa972ffd700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa95c06ac90 0x7fa95c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.552+0000 7fa972ffd700 1 -- 172.21.15.31:0/2376381519 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa974089150 con 0x7fa97c0c5d10 2024-03-30T01:41:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.552+0000 7fa97a59c700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa95c06ac90 0x7fa95c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.553+0000 7fa97a59c700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa95c06ac90 0x7fa95c06d150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0090b0 tx=0x7fa96c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.557+0000 7fa972ffd700 1 -- 172.21.15.31:0/2376381519 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa974054000 con 0x7fa97c0c5d10 2024-03-30T01:41:48.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.769+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa97c0c5200 con 0x7fa97c0c5d10 2024-03-30T01:41:48.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.770+0000 7fa972ffd700 1 -- 172.21.15.31:0/2376381519 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa974057620 con 0x7fa97c0c5d10 2024-03-30T01:41:48.775 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa95c06ac90 msgr2=0x7fa95c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa95c06ac90 0x7fa95c06d150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0090b0 tx=0x7fa96c008030 comp rx=0 tx=0).stop 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c0c5d10 msgr2=0x7fa97c1492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c0c5d10 0x7fa97c1492f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa97400cc50 tx=0x7fa97400cf60 comp rx=0 tx=0).stop 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 shutdown_connections 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c0c5d10 0x7fa97c1492f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa95c06ac90 0x7fa95c06d150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa97c0c3140 0x7fa97c0a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 --2- 172.21.15.31:0/2376381519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa97c0c49f0 0x7fa97c148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 >> 172.21.15.31:0/2376381519 conn(0x7fa97c0c7cf0 msgr2=0x7fa97c06d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 shutdown_connections 2024-03-30T01:41:48.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.772+0000 7fa9809b8700 1 -- 172.21.15.31:0/2376381519 wait complete. 2024-03-30T01:41:48.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.924+0000 7f9d3cc46700 1 Processor -- start 2024-03-30T01:41:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.924+0000 7f9d3cc46700 1 -- start start 2024-03-30T01:41:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.925+0000 7f9d3cc46700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d380bfbf0 0x7f9d3806e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.925+0000 7f9d3cc46700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9d380c2b80 0x7f9d3806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.925+0000 7f9d3cc46700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d380c0ff0 con 0x7f9d380ab520 2024-03-30T01:41:48.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.925+0000 7f9d3cc46700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d381550b0 con 0x7f9d380c2b80 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.925+0000 7f9d3cc46700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380c2040 con 0x7f9d380bfbf0 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.925+0000 7f9d36d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9d380ab520 0x7f9d3814a980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46300/0 (socket says 172.21.15.31:46300) 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.925+0000 7f9d36d9d700 1 -- 172.21.15.31:0/256227830 learned_addr learned my addr 172.21.15.31:0/256227830 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d35d9b700 1 --2- 172.21.15.31:0/256227830 >> v2:172.21.15.59:3300/0 conn(0x7f9d380c2b80 0x7f9d3806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3659c700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d380bfbf0 0x7f9d3806e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347972956 0 0) 0x7f9d380c0ff0 con 0x7f9d380ab520 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d200035b0 con 0x7f9d380ab520 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132089152 0 0) 0x7f9d200035b0 con 0x7f9d380ab520 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d380bfbf0 msgr2=0x7f9d3806e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d380bfbf0 0x7f9d3806e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 >> v2:172.21.15.59:3300/0 conn(0x7f9d380c2b80 msgr2=0x7f9d3806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 --2- 172.21.15.31:0/256227830 >> v2:172.21.15.59:3300/0 conn(0x7f9d380c2b80 0x7f9d3806ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d38155290 con 0x7f9d380ab520 2024-03-30T01:41:48.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f9d300019f0 con 0x7f9d380ab520 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 >> v1:172.21.15.31:6789/0 conn(0x7f9d380ab520 legacy=0x7f9d3814a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20003ae0 0x7f9d20003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d20004850 0x7f9d20004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d380bfbf0 0x7f9d20005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d381550b0 con 0x7f9d20004850 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380c2040 con 0x7f9d20003ae0 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2000ba40 con 0x7f9d380bfbf0 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d35d9b700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d20004850 0x7f9d20004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d3659c700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20003ae0 0x7f9d20003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.926+0000 7f9d36d9d700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d380bfbf0 0x7f9d20005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d35d9b700 1 -- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d380bfbf0 msgr2=0x7f9d20005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d35d9b700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d380bfbf0 0x7f9d20005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d35d9b700 1 -- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20003ae0 msgr2=0x7f9d20003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d35d9b700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d20003ae0 0x7f9d20003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d35d9b700 1 -- 172.21.15.31:0/256227830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d200035b0 con 0x7f9d20004850 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d36d9d700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d380bfbf0 0x7f9d20005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d35d9b700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d20004850 0x7f9d20004cd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009b60 tx=0x7f9d20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55df3a980d84169e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800e6c0 con 0x7f9d20004850 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2800e820 con 0x7f9d20004850 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d3559a700 1 -- 172.21.15.31:0/256227830 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800eb90 con 0x7f9d20004850 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d3cc46700 1 -- 172.21.15.31:0/256227830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d20004850 msgr2=0x7f9d20004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d3cc46700 1 --2- 172.21.15.31:0/256227830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d20004850 0x7f9d20004cd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009b60 tx=0x7f9d20005880 comp rx=0 tx=0).stop 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.927+0000 7f9d36d9d700 1 -- 172.21.15.31:0/256227830 reap_dead start 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.928+0000 7f9d3cc46700 1 -- 172.21.15.31:0/256227830 shutdown_connections 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.928+0000 7f9d3cc46700 1 -- 172.21.15.31:0/256227830 >> 172.21.15.31:0/256227830 conn(0x7f9d380c6550 msgr2=0x7f9d380c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.928+0000 7f9d3cc46700 1 -- 172.21.15.31:0/256227830 shutdown_connections 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.928+0000 7f9d3cc46700 1 -- 172.21.15.31:0/256227830 wait complete. 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.929+0000 7f9d3cc46700 1 Processor -- start 2024-03-30T01:41:48.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.929+0000 7f9d3cc46700 1 -- start start 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d3cc46700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d20004850 0x7f9d3814cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d3cc46700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d38150940 0x7f9d3814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d3cc46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d38151040 0x7f9d3806b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d3cc46700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3814d2e0 con 0x7f9d38150940 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d3cc46700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3814e4d0 con 0x7f9d38151040 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d3cc46700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d381496f0 con 0x7f9d20004850 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d38150940 0x7f9d3814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d35d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d38150940 0x7f9d3814df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55512/0 (socket says 172.21.15.31:55512) 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d35d9b700 1 -- 172.21.15.31:0/735104700 learned_addr learned my addr 172.21.15.31:0/735104700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d36d9d700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d38151040 0x7f9d3806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d3659c700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d20004850 0x7f9d3814cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d35d9b700 1 -- 172.21.15.31:0/735104700 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d20004850 msgr2=0x7f9d3814cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d35d9b700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d20004850 0x7f9d3814cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d35d9b700 1 -- 172.21.15.31:0/735104700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d38151040 msgr2=0x7f9d3806b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d35d9b700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d38151040 0x7f9d3806b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.930+0000 7f9d35d9b700 1 -- 172.21.15.31:0/735104700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d38148fa0 con 0x7f9d38150940 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.931+0000 7f9d35d9b700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d38150940 0x7f9d3814df90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d280090b0 tx=0x7f9d280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.931+0000 7f9d26ffd700 1 -- 172.21.15.31:0/735104700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28021070 con 0x7f9d38150940 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.931+0000 7f9d26ffd700 1 -- 172.21.15.31:0/735104700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d28004510 con 0x7f9d38150940 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.931+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3814d2e0 con 0x7f9d38150940 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.931+0000 7f9d26ffd700 1 -- 172.21.15.31:0/735104700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800e6c0 con 0x7f9d38150940 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.931+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d381edc90 con 0x7f9d38150940 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.932+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d380747d0 con 0x7f9d38150940 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.932+0000 7f9d26ffd700 1 -- 172.21.15.31:0/735104700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9d28025070 con 0x7f9d38150940 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.933+0000 7f9d26ffd700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d1806ac90 0x7f9d1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.933+0000 7f9d26ffd700 1 -- 172.21.15.31:0/735104700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9d2808a6b0 con 0x7f9d38150940 2024-03-30T01:41:48.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.933+0000 7f9d3659c700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d1806ac90 0x7f9d1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:48.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.933+0000 7f9d3659c700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d1806ac90 0x7f9d1806d150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009b80 tx=0x7f9d2c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:48.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:48.938+0000 7f9d26ffd700 1 -- 172.21.15.31:0/735104700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9d28055740 con 0x7f9d38150940 2024-03-30T01:41:49.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.150+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d380c69d0 con 0x7f9d38150940 2024-03-30T01:41:49.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.151+0000 7f9d26ffd700 1 -- 172.21.15.31:0/735104700 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d28058d60 con 0x7f9d38150940 2024-03-30T01:41:49.156 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d1806ac90 msgr2=0x7f9d1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d1806ac90 0x7f9d1806d150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009b80 tx=0x7f9d2c009210 comp rx=0 tx=0).stop 2024-03-30T01:41:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d38150940 msgr2=0x7f9d3814df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d38150940 0x7f9d3814df90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d280090b0 tx=0x7f9d280047c0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 shutdown_connections 2024-03-30T01:41:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d1806ac90 0x7f9d1806d150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d38151040 0x7f9d3806b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d20004850 0x7f9d3814cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 --2- 172.21.15.31:0/735104700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d38150940 0x7f9d3814df90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 >> 172.21.15.31:0/735104700 conn(0x7f9d380c6550 msgr2=0x7f9d380a9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 shutdown_connections 2024-03-30T01:41:49.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.153+0000 7f9d3cc46700 1 -- 172.21.15.31:0/735104700 wait complete. 2024-03-30T01:41:49.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:49.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.306+0000 7fa8a8e9c700 1 Processor -- start 2024-03-30T01:41:49.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.306+0000 7fa8a8e9c700 1 -- start start 2024-03-30T01:41:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a8e9c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa8a414fb40 0x7fa8a40c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a8e9c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a40c4620 con 0x7fa8a40a7600 2024-03-30T01:41:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a8e9c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a4152ad0 con 0x7fa8a414ff20 2024-03-30T01:41:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a8e9c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a4152570 con 0x7fa8a414fb40 2024-03-30T01:41:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a1d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa8a40a7600 0x7fa8a4152f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46312/0 (socket says 172.21.15.31:46312) 2024-03-30T01:41:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a1d9b700 1 -- 172.21.15.31:0/601495234 learned_addr learned my addr 172.21.15.31:0/601495234 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:49.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a259c700 1 --2- 172.21.15.31:0/601495234 >> v2:172.21.15.59:3300/0 conn(0x7fa8a414fb40 0x7fa8a40c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a159a700 1 -- 172.21.15.31:0/601495234 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845795898 0 0) 0x7fa898001110 con 0x7fa8a40a7600 2024-03-30T01:41:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a159a700 1 -- 172.21.15.31:0/601495234 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa88c0035b0 con 0x7fa8a40a7600 2024-03-30T01:41:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a259c700 1 -- 172.21.15.31:0/601495234 >> v1:172.21.15.59:6789/0 conn(0x7fa8a414ff20 legacy=0x7fa8a40c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a259c700 1 -- 172.21.15.31:0/601495234 >> v1:172.21.15.31:6789/0 conn(0x7fa8a40a7600 legacy=0x7fa8a4152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.307+0000 7fa8a259c700 1 -- 172.21.15.31:0/601495234 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a40c2ca0 con 0x7fa8a414fb40 2024-03-30T01:41:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.308+0000 7fa8a259c700 1 --2- 172.21.15.31:0/601495234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a414fb40 0x7fa8a40c2760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa894009840 tx=0x7fa894009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b6def0d0f27e09 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.308+0000 7fa8a159a700 1 -- 172.21.15.31:0/601495234 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894011830 con 0x7fa8a414fb40 2024-03-30T01:41:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.308+0000 7fa8a159a700 1 -- 172.21.15.31:0/601495234 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa894011990 con 0x7fa8a414fb40 2024-03-30T01:41:49.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.308+0000 7fa8a159a700 1 -- 172.21.15.31:0/601495234 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894011d20 con 0x7fa8a414fb40 2024-03-30T01:41:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.308+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/601495234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a414fb40 msgr2=0x7fa8a40c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.308+0000 7fa8a8e9c700 1 --2- 172.21.15.31:0/601495234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a414fb40 0x7fa8a40c2760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa894009840 tx=0x7fa894009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.309+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/601495234 shutdown_connections 2024-03-30T01:41:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.309+0000 7fa8a8e9c700 1 --2- 172.21.15.31:0/601495234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a414fb40 0x7fa8a40c2760 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.309+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/601495234 >> 172.21.15.31:0/601495234 conn(0x7fa8a40c7d60 msgr2=0x7fa8a40c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.309+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/601495234 shutdown_connections 2024-03-30T01:41:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.309+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/601495234 wait complete. 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.310+0000 7fa8a8e9c700 1 Processor -- start 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.310+0000 7fa8a8e9c700 1 -- start start 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a8e9c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a40a7600 0x7fa8a40c0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a8e9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8a414fb40 0x7fa8a4145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a8e9c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a414ff20 0x7fa8a4146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a8e9c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40bde80 con 0x7fa8a414fb40 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a8e9c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40acd50 con 0x7fa8a40a7600 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a8e9c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40bf8c0 con 0x7fa8a414ff20 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8a414fb40 0x7fa8a4145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8a414fb40 0x7fa8a4145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55516/0 (socket says 172.21.15.31:55516) 2024-03-30T01:41:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a1d9b700 1 -- 172.21.15.31:0/708114669 learned_addr learned my addr 172.21.15.31:0/708114669 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a259c700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a40a7600 0x7fa8a40c0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a2d9d700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a414ff20 0x7fa8a4146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a1d9b700 1 -- 172.21.15.31:0/708114669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a414ff20 msgr2=0x7fa8a4146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a1d9b700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a414ff20 0x7fa8a4146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a1d9b700 1 -- 172.21.15.31:0/708114669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a40a7600 msgr2=0x7fa8a40c0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a1d9b700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a40a7600 0x7fa8a40c0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.311+0000 7fa8a1d9b700 1 -- 172.21.15.31:0/708114669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8940094a0 con 0x7fa8a414fb40 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.312+0000 7fa8a1d9b700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8a414fb40 0x7fa8a4145ff0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa89800c520 tx=0x7fa89800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.312+0000 7fa892ffd700 1 -- 172.21.15.31:0/708114669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898005d70 con 0x7fa8a414fb40 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.312+0000 7fa892ffd700 1 -- 172.21.15.31:0/708114669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8980070d0 con 0x7fa8a414fb40 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.312+0000 7fa892ffd700 1 -- 172.21.15.31:0/708114669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898007440 con 0x7fa8a414fb40 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.312+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a40bd830 con 0x7fa8a414fb40 2024-03-30T01:41:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.312+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a40bf5c0 con 0x7fa8a414fb40 2024-03-30T01:41:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.313+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a409a7d0 con 0x7fa8a414fb40 2024-03-30T01:41:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.313+0000 7fa892ffd700 1 -- 172.21.15.31:0/708114669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa8980075a0 con 0x7fa8a414fb40 2024-03-30T01:41:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.314+0000 7fa892ffd700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa88406ac90 0x7fa88406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.314+0000 7fa892ffd700 1 -- 172.21.15.31:0/708114669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa8980889a0 con 0x7fa8a414fb40 2024-03-30T01:41:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.314+0000 7fa8a259c700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa88406ac90 0x7fa88406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.314+0000 7fa8a259c700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa88406ac90 0x7fa88406d150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8940090b0 tx=0x7fa894008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:49.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.319+0000 7fa892ffd700 1 -- 172.21.15.31:0/708114669 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa8a409a7d0 con 0x7fa8a414fb40 2024-03-30T01:41:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.531+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa8a40c4620 con 0x7fa8a414fb40 2024-03-30T01:41:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.532+0000 7fa892ffd700 1 -- 172.21.15.31:0/708114669 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa8a40c4620 con 0x7fa8a414fb40 2024-03-30T01:41:49.538 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa88406ac90 msgr2=0x7fa88406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa88406ac90 0x7fa88406d150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8940090b0 tx=0x7fa894008030 comp rx=0 tx=0).stop 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8a414fb40 msgr2=0x7fa8a4145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8a414fb40 0x7fa8a4145ff0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa89800c520 tx=0x7fa89800c830 comp rx=0 tx=0).stop 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 shutdown_connections 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa88406ac90 0x7fa88406d150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a414ff20 0x7fa8a4146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a40a7600 0x7fa8a40c0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 --2- 172.21.15.31:0/708114669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8a414fb40 0x7fa8a4145ff0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 >> 172.21.15.31:0/708114669 conn(0x7fa8a40c7d60 msgr2=0x7fa8a41519c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 shutdown_connections 2024-03-30T01:41:49.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.534+0000 7fa8a8e9c700 1 -- 172.21.15.31:0/708114669 wait complete. 2024-03-30T01:41:49.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T01:41:49.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.685+0000 7fbd3d8f7700 1 Processor -- start 2024-03-30T01:41:49.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.685+0000 7fbd3d8f7700 1 -- start start 2024-03-30T01:41:49.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd3d8f7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbd380c7270 0x7fbd380c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd3d8f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c6330 0x7fbd38151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd3d8f7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3806bf10 con 0x7fbd38152000 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd3d8f7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd380c5990 con 0x7fbd380c7270 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd3d8f7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd38152490 con 0x7fbd380c6330 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd36ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbd380c7270 0x7fbd380c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd36ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbd380c7270 0x7fbd380c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55528/0 (socket says 172.21.15.31:55528) 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd36ffd700 1 -- 172.21.15.31:0/2923330986 learned_addr learned my addr 172.21.15.31:0/2923330986 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd367fc700 1 --2- 172.21.15.31:0/2923330986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c6330 0x7fbd38151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd36ffd700 1 -- 172.21.15.31:0/2923330986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c6330 msgr2=0x7fbd38151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd36ffd700 1 --2- 172.21.15.31:0/2923330986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c6330 0x7fbd38151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd36ffd700 1 -- 172.21.15.31:0/2923330986 >> v1:172.21.15.59:6789/0 conn(0x7fbd38152000 legacy=0x7fbd38157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd36ffd700 1 -- 172.21.15.31:0/2923330986 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3806b300 con 0x7fbd380c7270 2024-03-30T01:41:49.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd36ffd700 1 --2- 172.21.15.31:0/2923330986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd380c7270 0x7fbd380c5df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd200098f0 tx=0x7fbd20009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407cb633f54a4e21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd35ffb700 1 -- 172.21.15.31:0/2923330986 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000b8a0 con 0x7fbd380c7270 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd35ffb700 1 -- 172.21.15.31:0/2923330986 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2000ba00 con 0x7fbd380c7270 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.686+0000 7fbd35ffb700 1 -- 172.21.15.31:0/2923330986 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000bd90 con 0x7fbd380c7270 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.687+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/2923330986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd380c7270 msgr2=0x7fbd380c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.687+0000 7fbd3d8f7700 1 --2- 172.21.15.31:0/2923330986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd380c7270 0x7fbd380c5df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd200098f0 tx=0x7fbd20009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.687+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/2923330986 shutdown_connections 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.687+0000 7fbd3d8f7700 1 --2- 172.21.15.31:0/2923330986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd380c7270 0x7fbd380c5df0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.687+0000 7fbd3d8f7700 1 --2- 172.21.15.31:0/2923330986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c6330 0x7fbd38151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.687+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/2923330986 >> 172.21.15.31:0/2923330986 conn(0x7fbd380abf60 msgr2=0x7fbd380ac370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.687+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/2923330986 shutdown_connections 2024-03-30T01:41:49.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.687+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/2923330986 wait complete. 2024-03-30T01:41:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.688+0000 7fbd3d8f7700 1 Processor -- start 2024-03-30T01:41:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.689+0000 7fbd3d8f7700 1 -- start start 2024-03-30T01:41:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd3d8f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd380c6330 0x7fbd381500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd3d8f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c7270 0x7fbd380c3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd3d8f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd38152000 0x7fbd380c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd3d8f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd380c1860 con 0x7fbd38152000 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd3d8f7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd380c1ab0 con 0x7fbd380c6330 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd3d8f7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd380c1d90 con 0x7fbd380c7270 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd38152000 0x7fbd380c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd38152000 0x7fbd380c3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55540/0 (socket says 172.21.15.31:55540) 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd377fe700 1 -- 172.21.15.31:0/6873985 learned_addr learned my addr 172.21.15.31:0/6873985 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd367fc700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c7270 0x7fbd380c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd36ffd700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd380c6330 0x7fbd381500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd377fe700 1 -- 172.21.15.31:0/6873985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c7270 msgr2=0x7fbd380c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd377fe700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c7270 0x7fbd380c3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd377fe700 1 -- 172.21.15.31:0/6873985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd380c6330 msgr2=0x7fbd381500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd377fe700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd380c6330 0x7fbd381500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd377fe700 1 -- 172.21.15.31:0/6873985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd20009550 con 0x7fbd38152000 2024-03-30T01:41:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd377fe700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd38152000 0x7fbd380c3b80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000caa0 tx=0x7fbd3000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd2ffff700 1 -- 172.21.15.31:0/6873985 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd30003c00 con 0x7fbd38152000 2024-03-30T01:41:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd2ffff700 1 -- 172.21.15.31:0/6873985 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd30003d60 con 0x7fbd38152000 2024-03-30T01:41:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd380ab210 con 0x7fbd38152000 2024-03-30T01:41:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.690+0000 7fbd2ffff700 1 -- 172.21.15.31:0/6873985 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3000aac0 con 0x7fbd38152000 2024-03-30T01:41:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.691+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd380ab7d0 con 0x7fbd38152000 2024-03-30T01:41:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.692+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3804ea80 con 0x7fbd38152000 2024-03-30T01:41:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.692+0000 7fbd2ffff700 1 -- 172.21.15.31:0/6873985 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbd3000ac20 con 0x7fbd38152000 2024-03-30T01:41:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.693+0000 7fbd2ffff700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbd1806ac90 0x7fbd1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.693+0000 7fbd2ffff700 1 -- 172.21.15.31:0/6873985 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbd30089700 con 0x7fbd38152000 2024-03-30T01:41:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.693+0000 7fbd36ffd700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbd1806ac90 0x7fbd1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.693+0000 7fbd36ffd700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbd1806ac90 0x7fbd1806d150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009c40 tx=0x7fbd20008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.697+0000 7fbd2ffff700 1 -- 172.21.15.31:0/6873985 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbd30054510 con 0x7fbd38152000 2024-03-30T01:41:49.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.931+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbd3814c630 con 0x7fbd38152000 2024-03-30T01:41:49.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.931+0000 7fbd2ffff700 1 -- 172.21.15.31:0/6873985 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbd30057b30 con 0x7fbd38152000 2024-03-30T01:41:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.933+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbd1806ac90 msgr2=0x7fbd1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.933+0000 7fbd3d8f7700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbd1806ac90 0x7fbd1806d150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009c40 tx=0x7fbd20008030 comp rx=0 tx=0).stop 2024-03-30T01:41:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.933+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd38152000 msgr2=0x7fbd380c3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.933+0000 7fbd3d8f7700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd38152000 0x7fbd380c3b80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000caa0 tx=0x7fbd3000ce60 comp rx=0 tx=0).stop 2024-03-30T01:41:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.934+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 shutdown_connections 2024-03-30T01:41:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.934+0000 7fbd3d8f7700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd38152000 0x7fbd380c3b80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.934+0000 7fbd3d8f7700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbd1806ac90 0x7fbd1806d150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.934+0000 7fbd3d8f7700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd380c6330 0x7fbd381500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.934+0000 7fbd3d8f7700 1 --2- 172.21.15.31:0/6873985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd380c7270 0x7fbd380c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.934+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 >> 172.21.15.31:0/6873985 conn(0x7fbd380abf60 msgr2=0x7fbd3814efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.934+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 shutdown_connections 2024-03-30T01:41:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:49.934+0000 7fbd3d8f7700 1 -- 172.21.15.31:0/6873985 wait complete. 2024-03-30T01:41:49.941 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-03-30T01:41:49.954 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-30T01:41:49.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:49.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-30T01:41:49.994 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-03-30T01:41:49.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:49.994 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T01:41:50.055 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:41:50.055 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:50.078 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:41:50.078 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:50.140 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:41:50.140 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:41:50.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.340+0000 7f67e5adb700 1 Processor -- start 2024-03-30T01:41:50.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.340+0000 7f67e5adb700 1 -- start start 2024-03-30T01:41:50.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.341+0000 7f67e5adb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f67e00a86d0 0x7f67e0153040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.341+0000 7f67e5adb700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e0156680 con 0x7f67e00c6990 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.341+0000 7f67e5adb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e0156b10 con 0x7f67e006cf20 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.341+0000 7f67e5adb700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e00a9c10 con 0x7f67e00a86d0 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.341+0000 7f67df7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f67e00a86d0 0x7f67e0153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.341+0000 7f67deffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67e00c6990 0x7f67e014f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46318/0 (socket says 172.21.15.31:46318) 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.341+0000 7f67deffd700 1 -- 172.21.15.31:0/4057276076 learned_addr learned my addr 172.21.15.31:0/4057276076 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.342+0000 7f67df7fe700 1 -- 172.21.15.31:0/4057276076 >> v1:172.21.15.59:6789/0 conn(0x7f67e006cf20 legacy=0x7f67e006d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.342+0000 7f67df7fe700 1 -- 172.21.15.31:0/4057276076 >> v1:172.21.15.31:6789/0 conn(0x7f67e00c6990 legacy=0x7f67e014f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.342+0000 7f67de7fc700 1 -- 172.21.15.31:0/4057276076 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995588722 0 0) 0x7f67e0156680 con 0x7f67e00c6990 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.342+0000 7f67df7fe700 1 -- 172.21.15.31:0/4057276076 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0155d50 con 0x7f67e00a86d0 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.342+0000 7f67df7fe700 1 --2- 172.21.15.31:0/4057276076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00a86d0 0x7f67e0153040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009840 tx=0x7f67d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413d539d3d84cbe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.342+0000 7f67de7fc700 1 -- 172.21.15.31:0/4057276076 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0011830 con 0x7f67e00a86d0 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.342+0000 7f67de7fc700 1 -- 172.21.15.31:0/4057276076 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d0011990 con 0x7f67e00a86d0 2024-03-30T01:41:50.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.342+0000 7f67de7fc700 1 -- 172.21.15.31:0/4057276076 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0011d20 con 0x7f67e00a86d0 2024-03-30T01:41:50.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.343+0000 7f67e5adb700 1 -- 172.21.15.31:0/4057276076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00a86d0 msgr2=0x7f67e0153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.343+0000 7f67e5adb700 1 --2- 172.21.15.31:0/4057276076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00a86d0 0x7f67e0153040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009840 tx=0x7f67d0009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:50.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.343+0000 7f67e5adb700 1 -- 172.21.15.31:0/4057276076 shutdown_connections 2024-03-30T01:41:50.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.343+0000 7f67e5adb700 1 --2- 172.21.15.31:0/4057276076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00a86d0 0x7f67e0153040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.343+0000 7f67e5adb700 1 -- 172.21.15.31:0/4057276076 >> 172.21.15.31:0/4057276076 conn(0x7f67e006af80 msgr2=0x7f67e00c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:50.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.343+0000 7f67e5adb700 1 -- 172.21.15.31:0/4057276076 shutdown_connections 2024-03-30T01:41:50.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.343+0000 7f67e5adb700 1 -- 172.21.15.31:0/4057276076 wait complete. 2024-03-30T01:41:50.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.344+0000 7f67e5adb700 1 Processor -- start 2024-03-30T01:41:50.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.344+0000 7f67e5adb700 1 -- start start 2024-03-30T01:41:50.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67e5adb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e006cf20 0x7f67e01e8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67e5adb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e00a86d0 0x7f67e0152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67e5adb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67e00c6990 0x7f67e01431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67e5adb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0153620 con 0x7f67e006cf20 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67e5adb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0149b20 con 0x7f67e00c6990 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67e5adb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01e88b0 con 0x7f67e00a86d0 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e006cf20 0x7f67e01e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e006cf20 0x7f67e01e8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55560/0 (socket says 172.21.15.31:55560) 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67df7fe700 1 -- 172.21.15.31:0/657345364 learned_addr learned my addr 172.21.15.31:0/657345364 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67dffff700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67e00c6990 0x7f67e01431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.345+0000 7f67deffd700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e00a86d0 0x7f67e0152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67df7fe700 1 -- 172.21.15.31:0/657345364 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e00a86d0 msgr2=0x7f67e0152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67df7fe700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e00a86d0 0x7f67e0152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67df7fe700 1 -- 172.21.15.31:0/657345364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67e00c6990 msgr2=0x7f67e01431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67df7fe700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67e00c6990 0x7f67e01431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67df7fe700 1 -- 172.21.15.31:0/657345364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00094a0 con 0x7f67e006cf20 2024-03-30T01:41:50.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67df7fe700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e006cf20 0x7f67e01e8370 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f67d00090b0 tx=0x7f67d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:50.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67c7fff700 1 -- 172.21.15.31:0/657345364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d001a7e0 con 0x7f67e006cf20 2024-03-30T01:41:50.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67c7fff700 1 -- 172.21.15.31:0/657345364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d001a940 con 0x7f67e006cf20 2024-03-30T01:41:50.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e01510e0 con 0x7f67e006cf20 2024-03-30T01:41:50.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67c7fff700 1 -- 172.21.15.31:0/657345364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d001acb0 con 0x7f67e006cf20 2024-03-30T01:41:50.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.346+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e01507b0 con 0x7f67e006cf20 2024-03-30T01:41:50.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.347+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e004ea80 con 0x7f67e006cf20 2024-03-30T01:41:50.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.348+0000 7f67c7fff700 1 -- 172.21.15.31:0/657345364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f67d0022070 con 0x7f67e006cf20 2024-03-30T01:41:50.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.349+0000 7f67c7fff700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67cc06ac90 0x7f67cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.349+0000 7f67c7fff700 1 -- 172.21.15.31:0/657345364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f67d008aff0 con 0x7f67e006cf20 2024-03-30T01:41:50.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.349+0000 7f67deffd700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67cc06ac90 0x7f67cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.349+0000 7f67deffd700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67cc06ac90 0x7f67cc06d150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f67e00c20f0 tx=0x7f67d4007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.353+0000 7f67c7fff700 1 -- 172.21.15.31:0/657345364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f67d0055f30 con 0x7f67e006cf20 2024-03-30T01:41:50.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.567+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67e006c080 con 0x7f67e006cf20 2024-03-30T01:41:50.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.568+0000 7f67c7fff700 1 -- 172.21.15.31:0/657345364 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18219 (secure 0 0 0) 0x7f67d0029090 con 0x7f67e006cf20 2024-03-30T01:41:50.575 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:50.575 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:42.922828+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:50.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67cc06ac90 msgr2=0x7f67cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67cc06ac90 0x7f67cc06d150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f67e00c20f0 tx=0x7f67d4007a60 comp rx=0 tx=0).stop 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e006cf20 msgr2=0x7f67e01e8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e006cf20 0x7f67e01e8370 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f67d00090b0 tx=0x7f67d0004750 comp rx=0 tx=0).stop 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 shutdown_connections 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e006cf20 0x7f67e01e8370 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67cc06ac90 0x7f67cc06d150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67e00c6990 0x7f67e01431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 --2- 172.21.15.31:0/657345364 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e00a86d0 0x7f67e0152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 >> 172.21.15.31:0/657345364 conn(0x7f67e006af80 msgr2=0x7f67e0154200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 shutdown_connections 2024-03-30T01:41:50.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.571+0000 7f67e5adb700 1 -- 172.21.15.31:0/657345364 wait complete. 2024-03-30T01:41:50.592 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:50.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:50.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.725+0000 7fbbb9a53700 1 Processor -- start 2024-03-30T01:41:50.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.725+0000 7fbbb9a53700 1 -- start start 2024-03-30T01:41:50.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.726+0000 7fbbb9a53700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbbb41549a0 0x7fbbb4072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.726+0000 7fbbb9a53700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbbb406d040 0x7fbbb4154d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.726+0000 7fbbb9a53700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb414ee60 con 0x7fbbb406d900 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.726+0000 7fbbb9a53700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb40c2290 con 0x7fbbb406d040 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.726+0000 7fbbb9a53700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb4155720 con 0x7fbbb41549a0 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.726+0000 7fbbb27fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbbb406d040 0x7fbbb4154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.726+0000 7fbbb27fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbbb406d040 0x7fbbb4154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55570/0 (socket says 172.21.15.31:55570) 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.726+0000 7fbbb37fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbbb406d900 0x7fbbb406dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46324/0 (socket says 172.21.15.31:46324) 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.726+0000 7fbbb27fc700 1 -- 172.21.15.31:0/180682875 learned_addr learned my addr 172.21.15.31:0/180682875 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb2ffd700 1 --2- 172.21.15.31:0/180682875 >> v2:172.21.15.59:3300/0 conn(0x7fbbb41549a0 0x7fbbb4072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb27fc700 1 -- 172.21.15.31:0/180682875 >> v2:172.21.15.59:3300/0 conn(0x7fbbb41549a0 msgr2=0x7fbbb4072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb27fc700 1 --2- 172.21.15.31:0/180682875 >> v2:172.21.15.59:3300/0 conn(0x7fbbb41549a0 0x7fbbb4072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb27fc700 1 -- 172.21.15.31:0/180682875 >> v1:172.21.15.31:6789/0 conn(0x7fbbb406d900 legacy=0x7fbbb406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb27fc700 1 -- 172.21.15.31:0/180682875 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb40c24e0 con 0x7fbbb406d040 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb2ffd700 1 --2- 172.21.15.31:0/180682875 >> v2:172.21.15.59:3300/0 conn(0x7fbbb41549a0 0x7fbbb4072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:50.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb27fc700 1 --2- 172.21.15.31:0/180682875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d040 0x7fbbb4154d80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40733c0 tx=0x7fbba800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a66fc31d4d2d930 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb1ffb700 1 -- 172.21.15.31:0/180682875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800dd30 con 0x7fbbb406d040 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb1ffb700 1 -- 172.21.15.31:0/180682875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba800de90 con 0x7fbbb406d040 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.727+0000 7fbbb1ffb700 1 -- 172.21.15.31:0/180682875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8010690 con 0x7fbbb406d040 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.728+0000 7fbbb9a53700 1 -- 172.21.15.31:0/180682875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d040 msgr2=0x7fbbb4154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.728+0000 7fbbb9a53700 1 --2- 172.21.15.31:0/180682875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d040 0x7fbbb4154d80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40733c0 tx=0x7fbba800be70 comp rx=0 tx=0).stop 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.728+0000 7fbbb9a53700 1 -- 172.21.15.31:0/180682875 shutdown_connections 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.728+0000 7fbbb9a53700 1 --2- 172.21.15.31:0/180682875 >> v2:172.21.15.59:3300/0 conn(0x7fbbb41549a0 0x7fbbb4072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.728+0000 7fbbb9a53700 1 --2- 172.21.15.31:0/180682875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d040 0x7fbbb4154d80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.728+0000 7fbbb9a53700 1 -- 172.21.15.31:0/180682875 >> 172.21.15.31:0/180682875 conn(0x7fbbb40c3980 msgr2=0x7fbbb40c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.728+0000 7fbbb9a53700 1 -- 172.21.15.31:0/180682875 shutdown_connections 2024-03-30T01:41:50.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.728+0000 7fbbb9a53700 1 -- 172.21.15.31:0/180682875 wait complete. 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.729+0000 7fbbb9a53700 1 Processor -- start 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.729+0000 7fbbb9a53700 1 -- start start 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb9a53700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbb406d040 0x7fbbb40c0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb9a53700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d900 0x7fbbb41ea250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb9a53700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbb41549a0 0x7fbbb41ea790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb9a53700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb414bdb0 con 0x7fbbb406d900 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb9a53700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41489c0 con 0x7fbbb41549a0 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb9a53700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41ed940 con 0x7fbbb406d040 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d900 0x7fbbb41ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d900 0x7fbbb41ea250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55578/0 (socket says 172.21.15.31:55578) 2024-03-30T01:41:50.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb27fc700 1 -- 172.21.15.31:0/1005643551 learned_addr learned my addr 172.21.15.31:0/1005643551 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb2ffd700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbb406d040 0x7fbbb40c0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb37fe700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbb41549a0 0x7fbbb41ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb27fc700 1 -- 172.21.15.31:0/1005643551 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbb406d040 msgr2=0x7fbbb40c0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb27fc700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbb406d040 0x7fbbb40c0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb27fc700 1 -- 172.21.15.31:0/1005643551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbb41549a0 msgr2=0x7fbbb41ea790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb27fc700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbb41549a0 0x7fbbb41ea790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.730+0000 7fbbb27fc700 1 -- 172.21.15.31:0/1005643551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba800b7f0 con 0x7fbbb406d900 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.731+0000 7fbbb27fc700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d900 0x7fbbb41ea250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbba800b7c0 tx=0x7fbba8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.731+0000 7fbb9bfff700 1 -- 172.21.15.31:0/1005643551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8003da0 con 0x7fbbb406d900 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.731+0000 7fbb9bfff700 1 -- 172.21.15.31:0/1005643551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba8010470 con 0x7fbbb406d900 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.731+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb41edc20 con 0x7fbbb406d900 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.731+0000 7fbb9bfff700 1 -- 172.21.15.31:0/1005643551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8010800 con 0x7fbbb406d900 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.731+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb41486d0 con 0x7fbbb406d900 2024-03-30T01:41:50.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.732+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb40c4870 con 0x7fbbb406d900 2024-03-30T01:41:50.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.733+0000 7fbb9bfff700 1 -- 172.21.15.31:0/1005643551 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbba802d430 con 0x7fbbb406d900 2024-03-30T01:41:50.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.733+0000 7fbb9bfff700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbba006ac90 0x7fbba006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:50.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.733+0000 7fbb9bfff700 1 -- 172.21.15.31:0/1005643551 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbba808aeb0 con 0x7fbbb406d900 2024-03-30T01:41:50.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.733+0000 7fbbb2ffd700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbba006ac90 0x7fbba006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:50.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.734+0000 7fbbb2ffd700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbba006ac90 0x7fbba006d150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40a9c30 tx=0x7fbba4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.738+0000 7fbb9bfff700 1 -- 172.21.15.31:0/1005643551 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbba8055d40 con 0x7fbbb406d900 2024-03-30T01:41:50.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.954+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbb41ee140 con 0x7fbbb406d900 2024-03-30T01:41:50.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.955+0000 7fbb9bfff700 1 -- 172.21.15.31:0/1005643551 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbb41ee140 con 0x7fbbb406d900 2024-03-30T01:41:50.961 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:50.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbba006ac90 msgr2=0x7fbba006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbba006ac90 0x7fbba006d150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40a9c30 tx=0x7fbba4008040 comp rx=0 tx=0).stop 2024-03-30T01:41:50.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d900 msgr2=0x7fbbb41ea250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:50.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d900 0x7fbbb41ea250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbba800b7c0 tx=0x7fbba8004750 comp rx=0 tx=0).stop 2024-03-30T01:41:50.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 shutdown_connections 2024-03-30T01:41:50.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbba006ac90 0x7fbba006d150 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbb41549a0 0x7fbbb41ea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbb406d040 0x7fbbb40c0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 --2- 172.21.15.31:0/1005643551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb406d900 0x7fbbb41ea250 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:50.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.957+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 >> 172.21.15.31:0/1005643551 conn(0x7fbbb40c3980 msgr2=0x7fbbb406eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:50.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.958+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 shutdown_connections 2024-03-30T01:41:50.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:50.958+0000 7fbbb9a53700 1 -- 172.21.15.31:0/1005643551 wait complete. 2024-03-30T01:41:50.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.111+0000 7f0899b01700 1 Processor -- start 2024-03-30T01:41:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.111+0000 7f0899b01700 1 -- start start 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.111+0000 7f0899b01700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0894151ea0 0x7f08940c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f0899b01700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f08940721f0 0x7f089406a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f0899b01700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0894155000 con 0x7f08940c07f0 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f0899b01700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089406e930 con 0x7f0894151ea0 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f0899b01700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089406df40 con 0x7f08940721f0 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08937fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0894151ea0 0x7f08940c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f0893fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f08940c07f0 0x7f089406ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46334/0 (socket says 172.21.15.31:46334) 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f0893fff700 1 -- 172.21.15.31:0/447311515 learned_addr learned my addr 172.21.15.31:0/447311515 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f0892ffd700 1 --2- 172.21.15.31:0/447311515 >> v2:172.21.15.59:3300/0 conn(0x7f08940721f0 0x7f089406a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08937fe700 1 -- 172.21.15.31:0/447311515 >> v2:172.21.15.59:3300/0 conn(0x7f08940721f0 msgr2=0x7f089406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08937fe700 1 --2- 172.21.15.31:0/447311515 >> v2:172.21.15.59:3300/0 conn(0x7f08940721f0 0x7f089406a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08937fe700 1 -- 172.21.15.31:0/447311515 >> v1:172.21.15.31:6789/0 conn(0x7f08940c07f0 legacy=0x7f089406ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08937fe700 1 -- 172.21.15.31:0/447311515 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089406e650 con 0x7f0894151ea0 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08927fc700 1 -- 172.21.15.31:0/447311515 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62146335 0 0) 0x7f0894155000 con 0x7f08940c07f0 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08937fe700 1 --2- 172.21.15.31:0/447311515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0894151ea0 0x7f08940c02b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0894073420 tx=0x7f0884009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d9ddcafb7c74cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08927fc700 1 -- 172.21.15.31:0/447311515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884011830 con 0x7f0894151ea0 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08927fc700 1 -- 172.21.15.31:0/447311515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0884011990 con 0x7f0894151ea0 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.112+0000 7f08927fc700 1 -- 172.21.15.31:0/447311515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884011d20 con 0x7f0894151ea0 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.113+0000 7f0899b01700 1 -- 172.21.15.31:0/447311515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0894151ea0 msgr2=0x7f08940c02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.113+0000 7f0899b01700 1 --2- 172.21.15.31:0/447311515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0894151ea0 0x7f08940c02b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0894073420 tx=0x7f0884009b20 comp rx=0 tx=0).stop 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.113+0000 7f0899b01700 1 -- 172.21.15.31:0/447311515 shutdown_connections 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.113+0000 7f0899b01700 1 --2- 172.21.15.31:0/447311515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0894151ea0 0x7f08940c02b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.113+0000 7f0899b01700 1 --2- 172.21.15.31:0/447311515 >> v2:172.21.15.59:3300/0 conn(0x7f08940721f0 0x7f089406a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.113+0000 7f0899b01700 1 -- 172.21.15.31:0/447311515 >> 172.21.15.31:0/447311515 conn(0x7f08940c3980 msgr2=0x7f08940c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.113+0000 7f0899b01700 1 -- 172.21.15.31:0/447311515 shutdown_connections 2024-03-30T01:41:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.113+0000 7f0899b01700 1 -- 172.21.15.31:0/447311515 wait complete. 2024-03-30T01:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.114+0000 7f0899b01700 1 Processor -- start 2024-03-30T01:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.115+0000 7f0899b01700 1 -- start start 2024-03-30T01:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.115+0000 7f0899b01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08940721f0 0x7f089414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.115+0000 7f0899b01700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940c07f0 0x7f089414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.115+0000 7f0899b01700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0894151ea0 0x7f089414f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.115+0000 7f0899b01700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089414be40 con 0x7f08940721f0 2024-03-30T01:41:51.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.115+0000 7f0899b01700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0894147b80 con 0x7f08940c07f0 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.115+0000 7f0899b01700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089414b410 con 0x7f0894151ea0 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.115+0000 7f08937fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08940721f0 0x7f089414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f08937fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08940721f0 0x7f089414cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55596/0 (socket says 172.21.15.31:55596) 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f08937fe700 1 -- 172.21.15.31:0/2365377854 learned_addr learned my addr 172.21.15.31:0/2365377854 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f0892ffd700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940c07f0 0x7f089414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f0893fff700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0894151ea0 0x7f089414f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f08937fe700 1 -- 172.21.15.31:0/2365377854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0894151ea0 msgr2=0x7f089414f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f08937fe700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0894151ea0 0x7f089414f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f08937fe700 1 -- 172.21.15.31:0/2365377854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940c07f0 msgr2=0x7f089414aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f08937fe700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940c07f0 0x7f089414aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f08937fe700 1 -- 172.21.15.31:0/2365377854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840094a0 con 0x7f08940721f0 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f0893fff700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0894151ea0 0x7f089414f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f08937fe700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08940721f0 0x7f089414cc50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f08840090b0 tx=0x7f0884004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f087bfff700 1 -- 172.21.15.31:0/2365377854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840038a0 con 0x7f08940721f0 2024-03-30T01:41:51.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f087bfff700 1 -- 172.21.15.31:0/2365377854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088401e070 con 0x7f08940721f0 2024-03-30T01:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0894148770 con 0x7f08940721f0 2024-03-30T01:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f087bfff700 1 -- 172.21.15.31:0/2365377854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884003c20 con 0x7f08940721f0 2024-03-30T01:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.116+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0894148cf0 con 0x7f08940721f0 2024-03-30T01:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.117+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08940747d0 con 0x7f08940721f0 2024-03-30T01:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.118+0000 7f087bfff700 1 -- 172.21.15.31:0/2365377854 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f088402d450 con 0x7f08940721f0 2024-03-30T01:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.118+0000 7f087bfff700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f088006ac90 0x7f088006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.118+0000 7f087bfff700 1 -- 172.21.15.31:0/2365377854 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f088408a740 con 0x7f08940721f0 2024-03-30T01:41:51.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.118+0000 7f0892ffd700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f088006ac90 0x7f088006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.119+0000 7f0892ffd700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f088006ac90 0x7f088006d150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f089414d190 tx=0x7f0888008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:51.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.123+0000 7f087bfff700 1 -- 172.21.15.31:0/2365377854 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f08840555f0 con 0x7f08940721f0 2024-03-30T01:41:51.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.338+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08940c1520 con 0x7f08940721f0 2024-03-30T01:41:51.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.338+0000 7f087bfff700 1 -- 172.21.15.31:0/2365377854 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0884058c10 con 0x7f08940721f0 2024-03-30T01:41:51.345 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:51.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f088006ac90 msgr2=0x7f088006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f088006ac90 0x7f088006d150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f089414d190 tx=0x7f0888008040 comp rx=0 tx=0).stop 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08940721f0 msgr2=0x7f089414cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08940721f0 0x7f089414cc50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f08840090b0 tx=0x7f0884004750 comp rx=0 tx=0).stop 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 shutdown_connections 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08940721f0 0x7f089414cc50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f088006ac90 0x7f088006d150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0894151ea0 0x7f089414f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 --2- 172.21.15.31:0/2365377854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08940c07f0 0x7f089414aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 >> 172.21.15.31:0/2365377854 conn(0x7f08940c3980 msgr2=0x7f0894150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 shutdown_connections 2024-03-30T01:41:51.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.341+0000 7f0899b01700 1 -- 172.21.15.31:0/2365377854 wait complete. 2024-03-30T01:41:51.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:51.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.496+0000 7fd76f49d700 1 Processor -- start 2024-03-30T01:41:51.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.496+0000 7fd76f49d700 1 -- start start 2024-03-30T01:41:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.497+0000 7fd76f49d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd76806d340 0x7fd768154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.497+0000 7fd76f49d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd768152070 con 0x7fd7680c1250 2024-03-30T01:41:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.497+0000 7fd76f49d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7680acef0 con 0x7fd768151c90 2024-03-30T01:41:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.497+0000 7fd76f49d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680acaa0 con 0x7fd76806d340 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.497+0000 7fd76ca38700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd7680c1250 0x7fd768154680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46350/0 (socket says 172.21.15.31:46350) 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.497+0000 7fd76ca38700 1 -- 172.21.15.31:0/3176572360 learned_addr learned my addr 172.21.15.31:0/3176572360 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.497+0000 7fd76da3a700 1 --2- 172.21.15.31:0/3176572360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd76806d340 0x7fd768154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd75ffff700 1 -- 172.21.15.31:0/3176572360 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190146702 0 0) 0x7fd768152070 con 0x7fd7680c1250 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd75ffff700 1 -- 172.21.15.31:0/3176572360 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7540035b0 con 0x7fd7680c1250 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd75ffff700 1 -- 172.21.15.31:0/3176572360 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176321596 0 0) 0x7fd7680acef0 con 0x7fd768151c90 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd75ffff700 1 -- 172.21.15.31:0/3176572360 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd768152070 con 0x7fd768151c90 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd76da3a700 1 -- 172.21.15.31:0/3176572360 >> v1:172.21.15.59:6789/0 conn(0x7fd768151c90 legacy=0x7fd76806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd76da3a700 1 -- 172.21.15.31:0/3176572360 >> v1:172.21.15.31:6789/0 conn(0x7fd7680c1250 legacy=0x7fd768154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd76da3a700 1 -- 172.21.15.31:0/3176572360 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7680c31d0 con 0x7fd76806d340 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd76da3a700 1 --2- 172.21.15.31:0/3176572360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd76806d340 0x7fd768154d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd760009dc0 tx=0x7fd76000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=986eb9a3b902fd4f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd75ffff700 1 -- 172.21.15.31:0/3176572360 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76000e070 con 0x7fd76806d340 2024-03-30T01:41:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd75ffff700 1 -- 172.21.15.31:0/3176572360 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76000bb20 con 0x7fd76806d340 2024-03-30T01:41:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.498+0000 7fd75ffff700 1 -- 172.21.15.31:0/3176572360 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760012430 con 0x7fd76806d340 2024-03-30T01:41:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.499+0000 7fd76f49d700 1 -- 172.21.15.31:0/3176572360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd76806d340 msgr2=0x7fd768154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.499+0000 7fd76f49d700 1 --2- 172.21.15.31:0/3176572360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd76806d340 0x7fd768154d90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd760009dc0 tx=0x7fd76000b330 comp rx=0 tx=0).stop 2024-03-30T01:41:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.499+0000 7fd76f49d700 1 -- 172.21.15.31:0/3176572360 shutdown_connections 2024-03-30T01:41:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.499+0000 7fd76f49d700 1 --2- 172.21.15.31:0/3176572360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd76806d340 0x7fd768154d90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.499+0000 7fd76f49d700 1 -- 172.21.15.31:0/3176572360 >> 172.21.15.31:0/3176572360 conn(0x7fd7680c3910 msgr2=0x7fd7680c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.499+0000 7fd76f49d700 1 -- 172.21.15.31:0/3176572360 shutdown_connections 2024-03-30T01:41:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.499+0000 7fd76f49d700 1 -- 172.21.15.31:0/3176572360 wait complete. 2024-03-30T01:41:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.500+0000 7fd76f49d700 1 Processor -- start 2024-03-30T01:41:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.500+0000 7fd76f49d700 1 -- start start 2024-03-30T01:41:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76f49d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd76806d340 0x7fd7680bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76f49d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7680c1250 0x7fd7680c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76f49d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd768151c90 0x7fd76814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76f49d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680bda70 con 0x7fd768151c90 2024-03-30T01:41:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76f49d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7681ed650 con 0x7fd76806d340 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76f49d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680c0400 con 0x7fd7680c1250 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76da3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd768151c90 0x7fd76814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76da3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd768151c90 0x7fd76814b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55602/0 (socket says 172.21.15.31:55602) 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76da3a700 1 -- 172.21.15.31:0/519644960 learned_addr learned my addr 172.21.15.31:0/519644960 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76ca38700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7680c1250 0x7fd7680c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76d239700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd76806d340 0x7fd7680bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76da3a700 1 -- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7680c1250 msgr2=0x7fd7680c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76da3a700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7680c1250 0x7fd7680c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76da3a700 1 -- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd76806d340 msgr2=0x7fd7680bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76da3a700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd76806d340 0x7fd7680bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76da3a700 1 -- 172.21.15.31:0/519644960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd760009a70 con 0x7fd768151c90 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76d239700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd76806d340 0x7fd7680bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.501+0000 7fd76ca38700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7680c1250 0x7fd7680c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:51.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.502+0000 7fd76da3a700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd768151c90 0x7fd76814b790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd76000b370 tx=0x7fd760004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.502+0000 7fd75dffb700 1 -- 172.21.15.31:0/519644960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76001b810 con 0x7fd768151c90 2024-03-30T01:41:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.502+0000 7fd75dffb700 1 -- 172.21.15.31:0/519644960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd760020070 con 0x7fd768151c90 2024-03-30T01:41:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.502+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd768158640 con 0x7fd768151c90 2024-03-30T01:41:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.502+0000 7fd75dffb700 1 -- 172.21.15.31:0/519644960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76001bbf0 con 0x7fd768151c90 2024-03-30T01:41:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.502+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7681eccb0 con 0x7fd768151c90 2024-03-30T01:41:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.503+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd768066f90 con 0x7fd768151c90 2024-03-30T01:41:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.504+0000 7fd75dffb700 1 -- 172.21.15.31:0/519644960 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd76001bd50 con 0x7fd768151c90 2024-03-30T01:41:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.504+0000 7fd75dffb700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd74c06ac90 0x7fd74c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.504+0000 7fd75dffb700 1 -- 172.21.15.31:0/519644960 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd760022070 con 0x7fd768151c90 2024-03-30T01:41:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.504+0000 7fd76d239700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd74c06ac90 0x7fd74c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.505+0000 7fd76d239700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd74c06ac90 0x7fd74c06d150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7640011e0 tx=0x7fd764009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:51.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.509+0000 7fd75dffb700 1 -- 172.21.15.31:0/519644960 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd7600568a0 con 0x7fd768151c90 2024-03-30T01:41:51.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.726+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7681ed790 con 0x7fd768151c90 2024-03-30T01:41:51.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.726+0000 7fd75dffb700 1 -- 172.21.15.31:0/519644960 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7681ed790 con 0x7fd768151c90 2024-03-30T01:41:51.733 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd74c06ac90 msgr2=0x7fd74c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd74c06ac90 0x7fd74c06d150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7640011e0 tx=0x7fd764009040 comp rx=0 tx=0).stop 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd768151c90 msgr2=0x7fd76814b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd768151c90 0x7fd76814b790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd76000b370 tx=0x7fd760004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 shutdown_connections 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd768151c90 0x7fd76814b790 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd74c06ac90 0x7fd74c06d150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd76806d340 0x7fd7680bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 --2- 172.21.15.31:0/519644960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7680c1250 0x7fd7680c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 >> 172.21.15.31:0/519644960 conn(0x7fd7680c3910 msgr2=0x7fd76814f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 shutdown_connections 2024-03-30T01:41:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.729+0000 7fd76f49d700 1 -- 172.21.15.31:0/519644960 wait complete. 2024-03-30T01:41:51.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:51.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.881+0000 7fb934359700 1 Processor -- start 2024-03-30T01:41:51.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.881+0000 7fb934359700 1 -- start start 2024-03-30T01:41:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb934359700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c06a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb934359700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb92c06d340 0x7fb92c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb934359700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92c152070 con 0x7fb92c0c1250 2024-03-30T01:41:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb934359700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92c0acaa0 con 0x7fb92c06d340 2024-03-30T01:41:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb934359700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0c3790 con 0x7fb92c151c90 2024-03-30T01:41:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb9320f5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb9328f6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb92c06d340 0x7fb92c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb9318f4700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb92c0c1250 0x7fb92c154680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:36752/0 (socket says 172.21.15.31:36752) 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb9328f6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb92c06d340 0x7fb92c154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:47418/0 (socket says 172.21.15.31:47418) 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb9318f4700 1 -- 172.21.15.31:0/1336972111 learned_addr learned my addr 172.21.15.31:0/1336972111 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.882+0000 7fb9320f5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c06a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:54598/0 (socket says 172.21.15.31:54598) 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.883+0000 7fb9320f5700 1 -- 172.21.15.31:0/1336972111 >> v2:172.21.15.59:3300/0 conn(0x7fb92c06d340 msgr2=0x7fb92c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.883+0000 7fb9320f5700 1 --2- 172.21.15.31:0/1336972111 >> v2:172.21.15.59:3300/0 conn(0x7fb92c06d340 0x7fb92c154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.883+0000 7fb9320f5700 1 -- 172.21.15.31:0/1336972111 >> v1:172.21.15.59:6789/0 conn(0x7fb92c0c1250 legacy=0x7fb92c154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.883+0000 7fb9320f5700 1 -- 172.21.15.31:0/1336972111 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c0acef0 con 0x7fb92c151c90 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.883+0000 7fb9320f5700 1 --2- 172.21.15.31:0/1336972111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c06a700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb920009840 tx=0x7fb920009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aecff0e37062ccc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.883+0000 7fb9310f3700 1 -- 172.21.15.31:0/1336972111 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920011830 con 0x7fb92c151c90 2024-03-30T01:41:51.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.883+0000 7fb9310f3700 1 -- 172.21.15.31:0/1336972111 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920011990 con 0x7fb92c151c90 2024-03-30T01:41:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.883+0000 7fb9310f3700 1 -- 172.21.15.31:0/1336972111 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920011d20 con 0x7fb92c151c90 2024-03-30T01:41:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.884+0000 7fb934359700 1 -- 172.21.15.31:0/1336972111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 msgr2=0x7fb92c06a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.884+0000 7fb934359700 1 --2- 172.21.15.31:0/1336972111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c06a700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb920009840 tx=0x7fb920009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.884+0000 7fb934359700 1 -- 172.21.15.31:0/1336972111 shutdown_connections 2024-03-30T01:41:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.884+0000 7fb934359700 1 --2- 172.21.15.31:0/1336972111 >> v2:172.21.15.59:3300/0 conn(0x7fb92c06d340 0x7fb92c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.884+0000 7fb934359700 1 --2- 172.21.15.31:0/1336972111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c06a700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.884+0000 7fb934359700 1 -- 172.21.15.31:0/1336972111 >> 172.21.15.31:0/1336972111 conn(0x7fb92c0c3910 msgr2=0x7fb92c0c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.884+0000 7fb934359700 1 -- 172.21.15.31:0/1336972111 shutdown_connections 2024-03-30T01:41:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.884+0000 7fb934359700 1 -- 172.21.15.31:0/1336972111 wait complete. 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.885+0000 7fb934359700 1 Processor -- start 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.885+0000 7fb934359700 1 -- start start 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb934359700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c06d340 0x7fb92c1ed690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb934359700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb92c0c1250 0x7fb92c071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb934359700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c1e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb934359700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c1edbd0 con 0x7fb92c0c1250 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb934359700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0bea00 con 0x7fb92c06d340 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb934359700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c1e98b0 con 0x7fb92c151c90 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9318f4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb92c0c1250 0x7fb92c071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9320f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c06d340 0x7fb92c1ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9318f4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb92c0c1250 0x7fb92c071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55612/0 (socket says 172.21.15.31:55612) 2024-03-30T01:41:51.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9318f4700 1 -- 172.21.15.31:0/400974031 learned_addr learned my addr 172.21.15.31:0/400974031 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9328f6700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c1e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9318f4700 1 -- 172.21.15.31:0/400974031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 msgr2=0x7fb92c1e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9318f4700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c1e9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9318f4700 1 -- 172.21.15.31:0/400974031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c06d340 msgr2=0x7fb92c1ed690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9318f4700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c06d340 0x7fb92c1ed690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.886+0000 7fb9318f4700 1 -- 172.21.15.31:0/400974031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9200094a0 con 0x7fb92c0c1250 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.887+0000 7fb9318f4700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb92c0c1250 0x7fb92c071aa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c550 tx=0x7fb92800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.887+0000 7fb91effd700 1 -- 172.21.15.31:0/400974031 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928005d70 con 0x7fb92c0c1250 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.887+0000 7fb91effd700 1 -- 172.21.15.31:0/400974031 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9280070d0 con 0x7fb92c0c1250 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.887+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92c158450 con 0x7fb92c0c1250 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.887+0000 7fb91effd700 1 -- 172.21.15.31:0/400974031 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928007440 con 0x7fb92c0c1250 2024-03-30T01:41:51.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.887+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92c157b80 con 0x7fb92c0c1250 2024-03-30T01:41:51.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.888+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92c066f90 con 0x7fb92c0c1250 2024-03-30T01:41:51.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.888+0000 7fb91effd700 1 -- 172.21.15.31:0/400974031 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb9280075a0 con 0x7fb92c0c1250 2024-03-30T01:41:51.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.889+0000 7fb91effd700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb91006ac90 0x7fb91006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:51.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.889+0000 7fb91effd700 1 -- 172.21.15.31:0/400974031 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb928088910 con 0x7fb92c0c1250 2024-03-30T01:41:51.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.889+0000 7fb9320f5700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb91006ac90 0x7fb91006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:51.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.889+0000 7fb9320f5700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb91006ac90 0x7fb91006d150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb920000c00 tx=0x7fb92001d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:51.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:51.894+0000 7fb91effd700 1 -- 172.21.15.31:0/400974031 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb928053870 con 0x7fb92c0c1250 2024-03-30T01:41:52.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.109+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb92c0747d0 con 0x7fb92c0c1250 2024-03-30T01:41:52.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.109+0000 7fb91effd700 1 -- 172.21.15.31:0/400974031 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb928017020 con 0x7fb92c0c1250 2024-03-30T01:41:52.116 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb91006ac90 msgr2=0x7fb91006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb91006ac90 0x7fb91006d150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb920000c00 tx=0x7fb92001d040 comp rx=0 tx=0).stop 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb92c0c1250 msgr2=0x7fb92c071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb92c0c1250 0x7fb92c071aa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c550 tx=0x7fb92800c860 comp rx=0 tx=0).stop 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 shutdown_connections 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb91006ac90 0x7fb91006d150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb92c151c90 0x7fb92c1e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb92c06d340 0x7fb92c1ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 --2- 172.21.15.31:0/400974031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb92c0c1250 0x7fb92c071aa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 >> 172.21.15.31:0/400974031 conn(0x7fb92c0c3910 msgr2=0x7fb92c14fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 shutdown_connections 2024-03-30T01:41:52.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.112+0000 7fb934359700 1 -- 172.21.15.31:0/400974031 wait complete. 2024-03-30T01:41:52.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T01:41:52.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.263+0000 7ff91c6cd700 1 Processor -- start 2024-03-30T01:41:52.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.263+0000 7ff91c6cd700 1 -- start start 2024-03-30T01:41:52.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff91c6cd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91414fbf0 0x7ff91406a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff91c6cd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff91406d9e0 0x7ff9141563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff91c6cd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff9140a9560 0x7ff9140ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff91c6cd700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91406e4a0 con 0x7ff91406d9e0 2024-03-30T01:41:52.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff91c6cd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff914155bf0 con 0x7ff9140a9560 2024-03-30T01:41:52.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff91c6cd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91406dee0 con 0x7ff91414fbf0 2024-03-30T01:41:52.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff919c68700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff91406d9e0 0x7ff9141563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff919c68700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff91406d9e0 0x7ff9141563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55616/0 (socket says 172.21.15.31:55616) 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff919c68700 1 -- 172.21.15.31:0/3518893165 learned_addr learned my addr 172.21.15.31:0/3518893165 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff91ac6a700 1 --2- 172.21.15.31:0/3518893165 >> v2:172.21.15.59:3300/0 conn(0x7ff9140a9560 0x7ff9140ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff91a469700 1 --2- 172.21.15.31:0/3518893165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91414fbf0 0x7ff91406a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff919c68700 1 -- 172.21.15.31:0/3518893165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91414fbf0 msgr2=0x7ff91406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff919c68700 1 --2- 172.21.15.31:0/3518893165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91414fbf0 0x7ff91406a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.264+0000 7ff919c68700 1 -- 172.21.15.31:0/3518893165 >> v2:172.21.15.59:3300/0 conn(0x7ff9140a9560 msgr2=0x7ff9140ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.265+0000 7ff919c68700 1 --2- 172.21.15.31:0/3518893165 >> v2:172.21.15.59:3300/0 conn(0x7ff9140a9560 0x7ff9140ab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.265+0000 7ff919c68700 1 -- 172.21.15.31:0/3518893165 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91406e1c0 con 0x7ff91406d9e0 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.265+0000 7ff919c68700 1 --2- 172.21.15.31:0/3518893165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff91406d9e0 0x7ff9141563b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff9140c6440 tx=0x7ff908009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9673ceaac3eb08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.265+0000 7ff919467700 1 -- 172.21.15.31:0/3518893165 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908005e30 con 0x7ff91406d9e0 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.265+0000 7ff919467700 1 -- 172.21.15.31:0/3518893165 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff908004030 con 0x7ff91406d9e0 2024-03-30T01:41:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.265+0000 7ff919467700 1 -- 172.21.15.31:0/3518893165 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9080043e0 con 0x7ff91406d9e0 2024-03-30T01:41:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.265+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3518893165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff91406d9e0 msgr2=0x7ff9141563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.265+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3518893165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff91406d9e0 0x7ff9141563b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff9140c6440 tx=0x7ff908009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.266+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3518893165 shutdown_connections 2024-03-30T01:41:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.266+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3518893165 >> v2:172.21.15.59:3300/0 conn(0x7ff9140a9560 0x7ff9140ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.266+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3518893165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91414fbf0 0x7ff91406a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.266+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3518893165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff91406d9e0 0x7ff9141563b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.266+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3518893165 >> 172.21.15.31:0/3518893165 conn(0x7ff9140c3190 msgr2=0x7ff9140c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.266+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3518893165 shutdown_connections 2024-03-30T01:41:52.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.266+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3518893165 wait complete. 2024-03-30T01:41:52.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.267+0000 7ff91c6cd700 1 Processor -- start 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.267+0000 7ff91c6cd700 1 -- start start 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.268+0000 7ff91c6cd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91406d9e0 0x7ff914149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.268+0000 7ff91c6cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9140a9560 0x7ff9140a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.268+0000 7ff91c6cd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91414fbf0 0x7ff914147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.268+0000 7ff91c6cd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141e7890 con 0x7ff9140a9560 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.268+0000 7ff91c6cd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141ebd40 con 0x7ff91414fbf0 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.268+0000 7ff91c6cd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141eb810 con 0x7ff91406d9e0 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.268+0000 7ff919c68700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9140a9560 0x7ff9140a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff919c68700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9140a9560 0x7ff9140a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55620/0 (socket says 172.21.15.31:55620) 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff919c68700 1 -- 172.21.15.31:0/3623457608 learned_addr learned my addr 172.21.15.31:0/3623457608 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff91a469700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91406d9e0 0x7ff914149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff91ac6a700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91414fbf0 0x7ff914147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff919c68700 1 -- 172.21.15.31:0/3623457608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91406d9e0 msgr2=0x7ff914149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff919c68700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91406d9e0 0x7ff914149d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff919c68700 1 -- 172.21.15.31:0/3623457608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91414fbf0 msgr2=0x7ff914147a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff919c68700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91414fbf0 0x7ff914147a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff919c68700 1 -- 172.21.15.31:0/3623457608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908009500 con 0x7ff9140a9560 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff919c68700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9140a9560 0x7ff9140a67a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff908000c00 tx=0x7ff908004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff906ffd700 1 -- 172.21.15.31:0/3623457608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908003c30 con 0x7ff9140a9560 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff906ffd700 1 -- 172.21.15.31:0/3623457608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff908003d90 con 0x7ff9140a9560 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9141e80d0 con 0x7ff9140a9560 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff906ffd700 1 -- 172.21.15.31:0/3623457608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908011660 con 0x7ff9140a9560 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.269+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9141e7a80 con 0x7ff9140a9560 2024-03-30T01:41:52.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.270+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9141e8260 con 0x7ff9140a9560 2024-03-30T01:41:52.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.271+0000 7ff906ffd700 1 -- 172.21.15.31:0/3623457608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff9080117c0 con 0x7ff9140a9560 2024-03-30T01:41:52.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.271+0000 7ff906ffd700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8f806ac90 0x7ff8f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.271+0000 7ff906ffd700 1 -- 172.21.15.31:0/3623457608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff90808a790 con 0x7ff9140a9560 2024-03-30T01:41:52.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.271+0000 7ff91a469700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8f806ac90 0x7ff8f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.272+0000 7ff91a469700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8f806ac90 0x7ff8f806d150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff9140c6160 tx=0x7ff910008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:52.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.276+0000 7ff906ffd700 1 -- 172.21.15.31:0/3623457608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff908055640 con 0x7ff9140a9560 2024-03-30T01:41:52.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.507+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff91404ea80 con 0x7ff9140a9560 2024-03-30T01:41:52.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.507+0000 7ff906ffd700 1 -- 172.21.15.31:0/3623457608 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff908058c60 con 0x7ff9140a9560 2024-03-30T01:41:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8f806ac90 msgr2=0x7ff8f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8f806ac90 0x7ff8f806d150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff9140c6160 tx=0x7ff910008040 comp rx=0 tx=0).stop 2024-03-30T01:41:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9140a9560 msgr2=0x7ff9140a67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9140a9560 0x7ff9140a67a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff908000c00 tx=0x7ff908004750 comp rx=0 tx=0).stop 2024-03-30T01:41:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 shutdown_connections 2024-03-30T01:41:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8f806ac90 0x7ff8f806d150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91414fbf0 0x7ff914147a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91406d9e0 0x7ff914149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 --2- 172.21.15.31:0/3623457608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9140a9560 0x7ff9140a67a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 >> 172.21.15.31:0/3623457608 conn(0x7ff9140c3190 msgr2=0x7ff914154b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 shutdown_connections 2024-03-30T01:41:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.510+0000 7ff91c6cd700 1 -- 172.21.15.31:0/3623457608 wait complete. 2024-03-30T01:41:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-03-30T01:41:52.530 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-30T01:41:52.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:52.530 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-30T01:41:52.575 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:41:52.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:41:52.575 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T01:41:52.670 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-30T01:41:52.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.810+0000 7f551e5cf700 1 Processor -- start 2024-03-30T01:41:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.810+0000 7f551e5cf700 1 -- start start 2024-03-30T01:41:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.810+0000 7f551e5cf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f551814fc30 0x7f551806a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.810+0000 7f551e5cf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5518152650 0x7f5518152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.810+0000 7f551e5cf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55180c2f60 con 0x7f55180c1300 2024-03-30T01:41:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.810+0000 7f551e5cf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55180c2a40 con 0x7f551814fc30 2024-03-30T01:41:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.810+0000 7f551e5cf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551806eca0 con 0x7f5518152650 2024-03-30T01:41:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f551cb6c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5518152650 0x7f5518152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f5517fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f551814fc30 0x7f551806a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f551cb6c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5518152650 0x7f5518152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:54652/0 (socket says 172.21.15.31:54652) 2024-03-30T01:41:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f551cb6c700 1 -- 172.21.15.31:0/3701509469 learned_addr learned my addr 172.21.15.31:0/3701509469 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f55177fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f55180c1300 0x7f551806ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:36766/0 (socket says 172.21.15.31:36766) 2024-03-30T01:41:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f5517fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f551814fc30 0x7f551806a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:47462/0 (socket says 172.21.15.31:47462) 2024-03-30T01:41:52.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f5517fff700 1 -- 172.21.15.31:0/3701509469 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5518152650 msgr2=0x7f5518152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:52.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f5517fff700 1 --2- 172.21.15.31:0/3701509469 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5518152650 0x7f5518152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f5517fff700 1 -- 172.21.15.31:0/3701509469 >> v1:172.21.15.59:6789/0 conn(0x7f55180c1300 legacy=0x7f551806ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f5517fff700 1 -- 172.21.15.31:0/3701509469 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551806b350 con 0x7f551814fc30 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f551cb6c700 1 --2- 172.21.15.31:0/3701509469 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5518152650 0x7f5518152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.811+0000 7f5516ffd700 1 -- 172.21.15.31:0/3701509469 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079961114 0 0) 0x7f55180c2f60 con 0x7f55180c1300 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f5517fff700 1 --2- 172.21.15.31:0/3701509469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551814fc30 0x7f551806a700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5508009840 tx=0x7f5508009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=174f741d65630297 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f5516ffd700 1 -- 172.21.15.31:0/3701509469 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508011830 con 0x7f551814fc30 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f5516ffd700 1 -- 172.21.15.31:0/3701509469 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508011990 con 0x7f551814fc30 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f5516ffd700 1 -- 172.21.15.31:0/3701509469 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508011d20 con 0x7f551814fc30 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f551e5cf700 1 -- 172.21.15.31:0/3701509469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551814fc30 msgr2=0x7f551806a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f551e5cf700 1 --2- 172.21.15.31:0/3701509469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551814fc30 0x7f551806a700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5508009840 tx=0x7f5508009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f551e5cf700 1 -- 172.21.15.31:0/3701509469 shutdown_connections 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f551e5cf700 1 --2- 172.21.15.31:0/3701509469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f551814fc30 0x7f551806a700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f551e5cf700 1 --2- 172.21.15.31:0/3701509469 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5518152650 0x7f5518152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f551e5cf700 1 -- 172.21.15.31:0/3701509469 >> 172.21.15.31:0/3701509469 conn(0x7f55180c3a60 msgr2=0x7f55180c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f551e5cf700 1 -- 172.21.15.31:0/3701509469 shutdown_connections 2024-03-30T01:41:52.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.812+0000 7f551e5cf700 1 -- 172.21.15.31:0/3701509469 wait complete. 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.814+0000 7f551e5cf700 1 Processor -- start 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.814+0000 7f551e5cf700 1 -- start start 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.814+0000 7f551e5cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55180c1300 0x7f55180abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551e5cf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f551814fc30 0x7f55180c0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551e5cf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5518152650 0x7f55181ea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551e5cf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5518144270 con 0x7f5518152650 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551e5cf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55180ab8b0 con 0x7f55180c1300 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551e5cf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5518143cb0 con 0x7f551814fc30 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551cb6c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5518152650 0x7f55181ea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551cb6c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5518152650 0x7f55181ea6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55634/0 (socket says 172.21.15.31:55634) 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551cb6c700 1 -- 172.21.15.31:0/4052019852 learned_addr learned my addr 172.21.15.31:0/4052019852 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f55177fe700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f551814fc30 0x7f55180c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f5517fff700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55180c1300 0x7f55180abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551cb6c700 1 -- 172.21.15.31:0/4052019852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f551814fc30 msgr2=0x7f55180c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551cb6c700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f551814fc30 0x7f55180c0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551cb6c700 1 -- 172.21.15.31:0/4052019852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55180c1300 msgr2=0x7f55180abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551cb6c700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55180c1300 0x7f55180abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551cb6c700 1 -- 172.21.15.31:0/4052019852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510009b30 con 0x7f5518152650 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.815+0000 7f551cb6c700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5518152650 0x7f55181ea6e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f551000efc0 tx=0x7f551000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.816+0000 7f5514ff9700 1 -- 172.21.15.31:0/4052019852 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510018070 con 0x7f5518152650 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.816+0000 7f5514ff9700 1 -- 172.21.15.31:0/4052019852 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551000f040 con 0x7f5518152650 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.816+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55080094a0 con 0x7f5518152650 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.816+0000 7f5514ff9700 1 -- 172.21.15.31:0/4052019852 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510003c40 con 0x7f5518152650 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.816+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55181e9ee0 con 0x7f5518152650 2024-03-30T01:41:52.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.817+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55181e9d40 con 0x7f5518152650 2024-03-30T01:41:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.817+0000 7f5514ff9700 1 -- 172.21.15.31:0/4052019852 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f551000c880 con 0x7f5518152650 2024-03-30T01:41:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.818+0000 7f5514ff9700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f550406ac90 0x7f550406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.818+0000 7f5514ff9700 1 -- 172.21.15.31:0/4052019852 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5510089730 con 0x7f5518152650 2024-03-30T01:41:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.818+0000 7f5517fff700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f550406ac90 0x7f550406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:52.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.818+0000 7f5517fff700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f550406ac90 0x7f550406d150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f55080090b0 tx=0x7f5508008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:52.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:52.823+0000 7f5514ff9700 1 -- 172.21.15.31:0/4052019852 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f55181e9d40 con 0x7f5518152650 2024-03-30T01:41:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.088+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55181e9d40 con 0x7f5518152650 2024-03-30T01:41:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.088+0000 7f5514ff9700 1 -- 172.21.15.31:0/4052019852 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f55181e9d40 con 0x7f5518152650 2024-03-30T01:41:53.095 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:53.095 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:16.755934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T01:41:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f550406ac90 msgr2=0x7f550406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f550406ac90 0x7f550406d150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f55080090b0 tx=0x7f5508008030 comp rx=0 tx=0).stop 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5518152650 msgr2=0x7f55181ea6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5518152650 0x7f55181ea6e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f551000efc0 tx=0x7f551000c3a0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 shutdown_connections 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5518152650 0x7f55181ea6e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f550406ac90 0x7f550406d150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55180c1300 0x7f55180abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 --2- 172.21.15.31:0/4052019852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f551814fc30 0x7f55180c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 >> 172.21.15.31:0/4052019852 conn(0x7f55180c3a60 msgr2=0x7f5518152ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 shutdown_connections 2024-03-30T01:41:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.091+0000 7f551e5cf700 1 -- 172.21.15.31:0/4052019852 wait complete. 2024-03-30T01:41:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-03-30T01:41:53.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:53.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.241+0000 7ff21f4cd700 1 Processor -- start 2024-03-30T01:41:53.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.241+0000 7ff21f4cd700 1 -- start start 2024-03-30T01:41:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21f4cd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff218154cf0 0x7ff2180c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21f4cd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2180c46a0 0x7ff21806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21f4cd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff218072270 0x7ff21814df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21f4cd700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2180a8460 con 0x7ff2180c46a0 2024-03-30T01:41:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21f4cd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2180c2e40 con 0x7ff218072270 2024-03-30T01:41:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21f4cd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff218155630 con 0x7ff218154cf0 2024-03-30T01:41:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21ca68700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2180c46a0 0x7ff21806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21ca68700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff2180c46a0 0x7ff21806d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55650/0 (socket says 172.21.15.31:55650) 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21ca68700 1 -- 172.21.15.31:0/337250560 learned_addr learned my addr 172.21.15.31:0/337250560 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21d269700 1 --2- 172.21.15.31:0/337250560 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff218154cf0 0x7ff2180c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21da6a700 1 --2- 172.21.15.31:0/337250560 >> v2:172.21.15.59:3300/0 conn(0x7ff218072270 0x7ff21814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21ca68700 1 -- 172.21.15.31:0/337250560 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff218154cf0 msgr2=0x7ff2180c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21ca68700 1 --2- 172.21.15.31:0/337250560 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff218154cf0 0x7ff2180c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21ca68700 1 -- 172.21.15.31:0/337250560 >> v2:172.21.15.59:3300/0 conn(0x7ff218072270 msgr2=0x7ff21814df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21ca68700 1 --2- 172.21.15.31:0/337250560 >> v2:172.21.15.59:3300/0 conn(0x7ff218072270 0x7ff21814df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21ca68700 1 -- 172.21.15.31:0/337250560 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2180a86b0 con 0x7ff2180c46a0 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.242+0000 7ff21da6a700 1 --2- 172.21.15.31:0/337250560 >> v2:172.21.15.59:3300/0 conn(0x7ff218072270 0x7ff21814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21ca68700 1 --2- 172.21.15.31:0/337250560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2180c46a0 0x7ff21806d5a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff21814f2b0 tx=0x7ff214009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7bae93041870af server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff20bfff700 1 -- 172.21.15.31:0/337250560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21400b870 con 0x7ff2180c46a0 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff20bfff700 1 -- 172.21.15.31:0/337250560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21400b9d0 con 0x7ff2180c46a0 2024-03-30T01:41:53.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff20bfff700 1 -- 172.21.15.31:0/337250560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21400bd60 con 0x7ff2180c46a0 2024-03-30T01:41:53.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21f4cd700 1 -- 172.21.15.31:0/337250560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2180c46a0 msgr2=0x7ff21806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/337250560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2180c46a0 0x7ff21806d5a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff21814f2b0 tx=0x7ff214009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21f4cd700 1 -- 172.21.15.31:0/337250560 shutdown_connections 2024-03-30T01:41:53.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/337250560 >> v2:172.21.15.59:3300/0 conn(0x7ff218072270 0x7ff21814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/337250560 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff218154cf0 0x7ff2180c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/337250560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2180c46a0 0x7ff21806d5a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21f4cd700 1 -- 172.21.15.31:0/337250560 >> 172.21.15.31:0/337250560 conn(0x7ff2180c3a60 msgr2=0x7ff2180c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21f4cd700 1 -- 172.21.15.31:0/337250560 shutdown_connections 2024-03-30T01:41:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.243+0000 7ff21f4cd700 1 -- 172.21.15.31:0/337250560 wait complete. 2024-03-30T01:41:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.245+0000 7ff21f4cd700 1 Processor -- start 2024-03-30T01:41:53.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.245+0000 7ff21f4cd700 1 -- start start 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.245+0000 7ff21f4cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218072270 0x7ff218149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.245+0000 7ff21f4cd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2180c46a0 0x7ff218150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.245+0000 7ff21f4cd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218154cf0 0x7ff218150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.245+0000 7ff21f4cd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21814de90 con 0x7ff218072270 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.245+0000 7ff21f4cd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21815ca30 con 0x7ff218154cf0 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.245+0000 7ff21f4cd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21815d2c0 con 0x7ff2180c46a0 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21d269700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218072270 0x7ff218149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21d269700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218072270 0x7ff218149eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55658/0 (socket says 172.21.15.31:55658) 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21d269700 1 -- 172.21.15.31:0/3158477883 learned_addr learned my addr 172.21.15.31:0/3158477883 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21ca68700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2180c46a0 0x7ff218150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21d269700 1 -- 172.21.15.31:0/3158477883 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2180c46a0 msgr2=0x7ff218150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21d269700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2180c46a0 0x7ff218150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21da6a700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218154cf0 0x7ff218150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21d269700 1 -- 172.21.15.31:0/3158477883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218154cf0 msgr2=0x7ff218150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21d269700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218154cf0 0x7ff218150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21d269700 1 -- 172.21.15.31:0/3158477883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff214009550 con 0x7ff218072270 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21d269700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218072270 0x7ff218149eb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00d800 tx=0x7ff20c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff209ffb700 1 -- 172.21.15.31:0/3158477883 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0125f0 con 0x7ff218072270 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff209ffb700 1 -- 172.21.15.31:0/3158477883 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20c012750 con 0x7ff218072270 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2180c0850 con 0x7ff218072270 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.246+0000 7ff209ffb700 1 -- 172.21.15.31:0/3158477883 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c012b00 con 0x7ff218072270 2024-03-30T01:41:53.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.247+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21815d560 con 0x7ff218072270 2024-03-30T01:41:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.248+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21815d8d0 con 0x7ff218072270 2024-03-30T01:41:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.248+0000 7ff209ffb700 1 -- 172.21.15.31:0/3158477883 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff20c012c60 con 0x7ff218072270 2024-03-30T01:41:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.248+0000 7ff209ffb700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff1fc06ac90 0x7ff1fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.248+0000 7ff209ffb700 1 -- 172.21.15.31:0/3158477883 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff20c089a00 con 0x7ff218072270 2024-03-30T01:41:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.249+0000 7ff21ca68700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff1fc06ac90 0x7ff1fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.249+0000 7ff21ca68700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff1fc06ac90 0x7ff1fc06d150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2180c4aa0 tx=0x7ff214008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:53.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.253+0000 7ff209ffb700 1 -- 172.21.15.31:0/3158477883 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff20c0548b0 con 0x7ff218072270 2024-03-30T01:41:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.468+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff21804ea80 con 0x7ff218072270 2024-03-30T01:41:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.469+0000 7ff209ffb700 1 -- 172.21.15.31:0/3158477883 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18219 (secure 0 0 0) 0x7ff20c057ed0 con 0x7ff218072270 2024-03-30T01:41:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:53.476 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:42.922828+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff1fc06ac90 msgr2=0x7ff1fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff1fc06ac90 0x7ff1fc06d150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2180c4aa0 tx=0x7ff214008030 comp rx=0 tx=0).stop 2024-03-30T01:41:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218072270 msgr2=0x7ff218149eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218072270 0x7ff218149eb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00d800 tx=0x7ff20c00dbc0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 shutdown_connections 2024-03-30T01:41:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff218072270 0x7ff218149eb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff1fc06ac90 0x7ff1fc06d150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff218154cf0 0x7ff218150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 --2- 172.21.15.31:0/3158477883 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2180c46a0 0x7ff218150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 >> 172.21.15.31:0/3158477883 conn(0x7ff2180c3a60 msgr2=0x7ff21806e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 shutdown_connections 2024-03-30T01:41:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.471+0000 7ff21f4cd700 1 -- 172.21.15.31:0/3158477883 wait complete. 2024-03-30T01:41:53.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:40:56.325144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:40:57.374109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T01:41:00.796698+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T01:41:53.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:41:53.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.624+0000 7faf597aa700 1 Processor -- start 2024-03-30T01:41:53.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.624+0000 7faf597aa700 1 -- start start 2024-03-30T01:41:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.625+0000 7faf597aa700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf540c4aa0 0x7faf54070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.625+0000 7faf597aa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 0x7faf5406bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.625+0000 7faf597aa700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf540c37c0 con 0x7faf540c46c0 2024-03-30T01:41:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.625+0000 7faf597aa700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf54070c60 con 0x7faf540c4aa0 2024-03-30T01:41:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.625+0000 7faf597aa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf540c3090 con 0x7faf540a6e90 2024-03-30T01:41:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.625+0000 7faf527fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf540c4aa0 0x7faf54070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.625+0000 7faf527fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf540c4aa0 0x7faf54070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55666/0 (socket says 172.21.15.31:55666) 2024-03-30T01:41:53.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.625+0000 7faf52ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faf540c46c0 0x7faf540c5bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46360/0 (socket says 172.21.15.31:46360) 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.625+0000 7faf527fc700 1 -- 172.21.15.31:0/3638396086 learned_addr learned my addr 172.21.15.31:0/3638396086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf537fe700 1 --2- 172.21.15.31:0/3638396086 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 0x7faf5406bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf527fc700 1 -- 172.21.15.31:0/3638396086 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 msgr2=0x7faf5406bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf527fc700 1 --2- 172.21.15.31:0/3638396086 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 0x7faf5406bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf527fc700 1 -- 172.21.15.31:0/3638396086 >> v1:172.21.15.31:6789/0 conn(0x7faf540c46c0 legacy=0x7faf540c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf527fc700 1 -- 172.21.15.31:0/3638396086 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf540c2890 con 0x7faf540c4aa0 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf51ffb700 1 -- 172.21.15.31:0/3638396086 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337307767 0 0) 0x7faf540c37c0 con 0x7faf540c46c0 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf527fc700 1 --2- 172.21.15.31:0/3638396086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c4aa0 0x7faf54070680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faf480098f0 tx=0x7faf48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a7c8058be0ca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf51ffb700 1 -- 172.21.15.31:0/3638396086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48011830 con 0x7faf540c4aa0 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf51ffb700 1 -- 172.21.15.31:0/3638396086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf48011990 con 0x7faf540c4aa0 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.626+0000 7faf51ffb700 1 -- 172.21.15.31:0/3638396086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48011d20 con 0x7faf540c4aa0 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.627+0000 7faf597aa700 1 -- 172.21.15.31:0/3638396086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c4aa0 msgr2=0x7faf54070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.627+0000 7faf597aa700 1 --2- 172.21.15.31:0/3638396086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c4aa0 0x7faf54070680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faf480098f0 tx=0x7faf48009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:53.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.627+0000 7faf597aa700 1 -- 172.21.15.31:0/3638396086 shutdown_connections 2024-03-30T01:41:53.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.627+0000 7faf597aa700 1 --2- 172.21.15.31:0/3638396086 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 0x7faf5406bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.627+0000 7faf597aa700 1 --2- 172.21.15.31:0/3638396086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c4aa0 0x7faf54070680 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.627+0000 7faf597aa700 1 -- 172.21.15.31:0/3638396086 >> 172.21.15.31:0/3638396086 conn(0x7faf540c7cf0 msgr2=0x7faf540c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:53.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.627+0000 7faf597aa700 1 -- 172.21.15.31:0/3638396086 shutdown_connections 2024-03-30T01:41:53.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.627+0000 7faf597aa700 1 -- 172.21.15.31:0/3638396086 wait complete. 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.628+0000 7faf597aa700 1 Processor -- start 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.628+0000 7faf597aa700 1 -- start start 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf597aa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 0x7faf541e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf597aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c46c0 0x7faf541dec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf597aa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf540c4aa0 0x7faf541df190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf597aa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541ed770 con 0x7faf540c46c0 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf597aa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541edca0 con 0x7faf540c4aa0 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf597aa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf540a7bd0 con 0x7faf540a6e90 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf527fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c46c0 0x7faf541dec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf527fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c46c0 0x7faf541dec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55680/0 (socket says 172.21.15.31:55680) 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf527fc700 1 -- 172.21.15.31:0/1649082672 learned_addr learned my addr 172.21.15.31:0/1649082672 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf52ffd700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 0x7faf541e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf537fe700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf540c4aa0 0x7faf541df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf527fc700 1 -- 172.21.15.31:0/1649082672 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 msgr2=0x7faf541e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf527fc700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 0x7faf541e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf527fc700 1 -- 172.21.15.31:0/1649082672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf540c4aa0 msgr2=0x7faf541df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.629+0000 7faf527fc700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf540c4aa0 0x7faf541df190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.630+0000 7faf527fc700 1 -- 172.21.15.31:0/1649082672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf48009550 con 0x7faf540c46c0 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.630+0000 7faf537fe700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf540c4aa0 0x7faf541df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:53.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.630+0000 7faf527fc700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c46c0 0x7faf541dec50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faf48009130 tx=0x7faf48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.630+0000 7faf3b7fe700 1 -- 172.21.15.31:0/1649082672 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480036a0 con 0x7faf540c46c0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.630+0000 7faf3b7fe700 1 -- 172.21.15.31:0/1649082672 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4801e070 con 0x7faf540c46c0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.630+0000 7faf597aa700 1 -- 172.21.15.31:0/1649082672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf541ed9c0 con 0x7faf540c46c0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.630+0000 7faf3b7fe700 1 -- 172.21.15.31:0/1649082672 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48003a00 con 0x7faf540c46c0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.630+0000 7faf597aa700 1 -- 172.21.15.31:0/1649082672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf541e1600 con 0x7faf540c46c0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.631+0000 7faf397fa700 1 -- 172.21.15.31:0/1649082672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c0052f0 con 0x7faf540c46c0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.632+0000 7faf3b7fe700 1 -- 172.21.15.31:0/1649082672 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7faf48003b60 con 0x7faf540c46c0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.632+0000 7faf3b7fe700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf4006ac90 0x7faf4006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.632+0000 7faf3b7fe700 1 -- 172.21.15.31:0/1649082672 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faf4808a600 con 0x7faf540c46c0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.632+0000 7faf52ffd700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf4006ac90 0x7faf4006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:53.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.633+0000 7faf52ffd700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf4006ac90 0x7faf4006d150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faf44000f60 tx=0x7faf44007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.637+0000 7faf3b7fe700 1 -- 172.21.15.31:0/1649082672 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7faf480554b0 con 0x7faf540c46c0 2024-03-30T01:41:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.852+0000 7faf397fa700 1 -- 172.21.15.31:0/1649082672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf2c005b20 con 0x7faf540c46c0 2024-03-30T01:41:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.853+0000 7faf3b7fe700 1 -- 172.21.15.31:0/1649082672 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf48058ad0 con 0x7faf540c46c0 2024-03-30T01:41:53.860 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 -- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf4006ac90 msgr2=0x7faf4006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf4006ac90 0x7faf4006d150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faf44000f60 tx=0x7faf44007c00 comp rx=0 tx=0).stop 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 -- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c46c0 msgr2=0x7faf541dec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c46c0 0x7faf541dec50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faf48009130 tx=0x7faf48004750 comp rx=0 tx=0).stop 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 -- 172.21.15.31:0/1649082672 shutdown_connections 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7faf4006ac90 0x7faf4006d150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf540c4aa0 0x7faf541df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf540a6e90 0x7faf541e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 --2- 172.21.15.31:0/1649082672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf540c46c0 0x7faf541dec50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 -- 172.21.15.31:0/1649082672 >> 172.21.15.31:0/1649082672 conn(0x7faf540c7cf0 msgr2=0x7faf541579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 -- 172.21.15.31:0/1649082672 shutdown_connections 2024-03-30T01:41:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:53.855+0000 7faf397fa700 1 -- 172.21.15.31:0/1649082672 wait complete. 2024-03-30T01:41:53.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:41:54.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.011+0000 7f7d61e22700 1 Processor -- start 2024-03-30T01:41:54.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.012+0000 7f7d61e22700 1 -- start start 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.012+0000 7f7d61e22700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d5c155490 0x7f7d5c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.012+0000 7f7d61e22700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c155000 con 0x7f7d5c151ea0 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.012+0000 7f7d61e22700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c06e650 con 0x7f7d5c0721f0 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.012+0000 7f7d61e22700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c06e930 con 0x7f7d5c155490 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.012+0000 7f7d5b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7d5c151ea0 0x7f7d5c0c02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46376/0 (socket says 172.21.15.31:46376) 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5b7fe700 1 -- 172.21.15.31:0/1572939999 learned_addr learned my addr 172.21.15.31:0/1572939999 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5bfff700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d5c155490 0x7f7d5c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394417678 0 0) 0x7f7d5c155000 con 0x7f7d5c151ea0 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d440035b0 con 0x7f7d5c151ea0 2024-03-30T01:41:54.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104115801 0 0) 0x7f7d5c06e650 con 0x7f7d5c0721f0 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d5c155000 con 0x7f7d5c0721f0 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 276084875 0 0) 0x7f7d440035b0 con 0x7f7d5c151ea0 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d5c155490 msgr2=0x7f7d5c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d5c155490 0x7f7d5c06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 >> v1:172.21.15.59:6789/0 conn(0x7f7d5c0721f0 legacy=0x7f7d5c06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c1549b0 con 0x7f7d5c151ea0 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f7d4c001670 con 0x7f7d5c151ea0 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5bfff700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d5c155490 0x7f7d5c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 >> v1:172.21.15.31:6789/0 conn(0x7f7d5c151ea0 legacy=0x7f7d5c0c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d5c155490 0x7f7d44003ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d440041e0 0x7f7d44004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44004d90 0x7f7d44005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c06e930 con 0x7f7d440041e0 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4400bc50 con 0x7f7d44004d90 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4400be10 con 0x7f7d5c155490 2024-03-30T01:41:54.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5affd700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d440041e0 0x7f7d44004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5b7fe700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d5c155490 0x7f7d44003ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.013+0000 7f7d5bfff700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44004d90 0x7f7d44005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.014+0000 7f7d5affd700 1 -- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d5c155490 msgr2=0x7f7d44003ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.014+0000 7f7d5affd700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d5c155490 0x7f7d44003ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.014+0000 7f7d5affd700 1 -- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44004d90 msgr2=0x7f7d44005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.014+0000 7f7d5affd700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d44004d90 0x7f7d44005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.014+0000 7f7d5affd700 1 -- 172.21.15.31:0/1572939999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d54009a70 con 0x7f7d440041e0 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.014+0000 7f7d5affd700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d440041e0 0x7f7d44004850 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7d50001980 tx=0x7f7d44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d85731a729eafa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.014+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50001470 con 0x7f7d440041e0 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.014+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d500052d0 con 0x7f7d440041e0 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.014+0000 7f7d5a7fc700 1 -- 172.21.15.31:0/1572939999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50003680 con 0x7f7d440041e0 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.015+0000 7f7d61e22700 1 -- 172.21.15.31:0/1572939999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d440041e0 msgr2=0x7f7d44004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.015+0000 7f7d61e22700 1 --2- 172.21.15.31:0/1572939999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d440041e0 0x7f7d44004850 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7d50001980 tx=0x7f7d44005880 comp rx=0 tx=0).stop 2024-03-30T01:41:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.015+0000 7f7d5bfff700 1 -- 172.21.15.31:0/1572939999 reap_dead start 2024-03-30T01:41:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.015+0000 7f7d61e22700 1 -- 172.21.15.31:0/1572939999 shutdown_connections 2024-03-30T01:41:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.015+0000 7f7d61e22700 1 -- 172.21.15.31:0/1572939999 >> 172.21.15.31:0/1572939999 conn(0x7f7d5c0c3980 msgr2=0x7f7d5c0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.015+0000 7f7d61e22700 1 -- 172.21.15.31:0/1572939999 shutdown_connections 2024-03-30T01:41:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.015+0000 7f7d61e22700 1 -- 172.21.15.31:0/1572939999 wait complete. 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.016+0000 7f7d61e22700 1 Processor -- start 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.016+0000 7f7d61e22700 1 -- start start 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d61e22700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d440041e0 0x7f7d5c1ed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d61e22700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c0ac120 0x7f7d5c1ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d61e22700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c14e380 0x7f7d5c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d61e22700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1edec0 con 0x7f7d5c0ac120 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d61e22700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c155ee0 con 0x7f7d5c14e380 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d61e22700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1edc30 con 0x7f7d440041e0 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c0ac120 0x7f7d5c1ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c0ac120 0x7f7d5c1ed6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55696/0 (socket says 172.21.15.31:55696) 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5affd700 1 -- 172.21.15.31:0/2634188484 learned_addr learned my addr 172.21.15.31:0/2634188484 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5b7fe700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d440041e0 0x7f7d5c1ed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5bfff700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c14e380 0x7f7d5c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5affd700 1 -- 172.21.15.31:0/2634188484 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d440041e0 msgr2=0x7f7d5c1ed1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5affd700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d440041e0 0x7f7d5c1ed1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5affd700 1 -- 172.21.15.31:0/2634188484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c14e380 msgr2=0x7f7d5c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5affd700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c14e380 0x7f7d5c14e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.017+0000 7f7d5affd700 1 -- 172.21.15.31:0/2634188484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c155b40 con 0x7f7d5c0ac120 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.018+0000 7f7d5affd700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c0ac120 0x7f7d5c1ed6f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7d50006a90 tx=0x7f7d50001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.018+0000 7f7d43fff700 1 -- 172.21.15.31:0/2634188484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50001110 con 0x7f7d5c0ac120 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.018+0000 7f7d43fff700 1 -- 172.21.15.31:0/2634188484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d50001410 con 0x7f7d5c0ac120 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.018+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c150260 con 0x7f7d5c0ac120 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.018+0000 7f7d43fff700 1 -- 172.21.15.31:0/2634188484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50007280 con 0x7f7d5c0ac120 2024-03-30T01:41:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.018+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c14f430 con 0x7f7d5c0ac120 2024-03-30T01:41:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.019+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c0747d0 con 0x7f7d5c0ac120 2024-03-30T01:41:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.019+0000 7f7d43fff700 1 -- 172.21.15.31:0/2634188484 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7d500073e0 con 0x7f7d5c0ac120 2024-03-30T01:41:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.020+0000 7f7d43fff700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7d4806ac90 0x7f7d4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.020+0000 7f7d43fff700 1 -- 172.21.15.31:0/2634188484 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7d5008a600 con 0x7f7d5c0ac120 2024-03-30T01:41:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.020+0000 7f7d5b7fe700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7d4806ac90 0x7f7d4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.020+0000 7f7d5b7fe700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7d4806ac90 0x7f7d4806d150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c000c00 tx=0x7f7d4c001e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.025+0000 7f7d43fff700 1 -- 172.21.15.31:0/2634188484 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7d500554c0 con 0x7f7d5c0ac120 2024-03-30T01:41:54.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.236+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d5c150830 con 0x7f7d5c0ac120 2024-03-30T01:41:54.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.236+0000 7f7d43fff700 1 -- 172.21.15.31:0/2634188484 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d50058ae0 con 0x7f7d5c0ac120 2024-03-30T01:41:54.243 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:54.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7d4806ac90 msgr2=0x7f7d4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7d4806ac90 0x7f7d4806d150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c000c00 tx=0x7f7d4c001e00 comp rx=0 tx=0).stop 2024-03-30T01:41:54.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c0ac120 msgr2=0x7f7d5c1ed6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c0ac120 0x7f7d5c1ed6f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7d50006a90 tx=0x7f7d50001310 comp rx=0 tx=0).stop 2024-03-30T01:41:54.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 shutdown_connections 2024-03-30T01:41:54.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7d4806ac90 0x7f7d4806d150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d5c14e380 0x7f7d5c14e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d440041e0 0x7f7d5c1ed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 --2- 172.21.15.31:0/2634188484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5c0ac120 0x7f7d5c1ed6f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 >> 172.21.15.31:0/2634188484 conn(0x7f7d5c0c3980 msgr2=0x7f7d5c0c4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:54.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 shutdown_connections 2024-03-30T01:41:54.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.238+0000 7f7d61e22700 1 -- 172.21.15.31:0/2634188484 wait complete. 2024-03-30T01:41:54.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:41:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.391+0000 7f8956c27700 1 Processor -- start 2024-03-30T01:41:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.391+0000 7f8956c27700 1 -- start start 2024-03-30T01:41:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f8956c27700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f89500c7f10 0x7f89500c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f8956c27700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8950155240 con 0x7f8950151d80 2024-03-30T01:41:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f8956c27700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89501556d0 con 0x7f89500c8420 2024-03-30T01:41:54.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f8956c27700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895006cb20 con 0x7f89500c7f10 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f894ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f89500c7f10 0x7f89500c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f89549c3700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8950151d80 0x7f89500ca350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46384/0 (socket says 172.21.15.31:46384) 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f89549c3700 1 -- 172.21.15.31:0/2329383680 learned_addr learned my addr 172.21.15.31:0/2329383680 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f894ffff700 1 -- 172.21.15.31:0/2329383680 >> v1:172.21.15.59:6789/0 conn(0x7f89500c8420 legacy=0x7f89500c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f894ffff700 1 -- 172.21.15.31:0/2329383680 >> v1:172.21.15.31:6789/0 conn(0x7f8950151d80 legacy=0x7f89500ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f894ffff700 1 -- 172.21.15.31:0/2329383680 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950154910 con 0x7f89500c7f10 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.392+0000 7f894f7fe700 1 -- 172.21.15.31:0/2329383680 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201496405 0 0) 0x7f8950155240 con 0x7f8950151d80 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.393+0000 7f894ffff700 1 --2- 172.21.15.31:0/2329383680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c7f10 0x7f89500c6a20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8938009840 tx=0x7f8938009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76affe7c5e47a65 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.393+0000 7f894f7fe700 1 -- 172.21.15.31:0/2329383680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938011830 con 0x7f89500c7f10 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.393+0000 7f894f7fe700 1 -- 172.21.15.31:0/2329383680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8938011990 con 0x7f89500c7f10 2024-03-30T01:41:54.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.393+0000 7f894f7fe700 1 -- 172.21.15.31:0/2329383680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938011d20 con 0x7f89500c7f10 2024-03-30T01:41:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.393+0000 7f8956c27700 1 -- 172.21.15.31:0/2329383680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c7f10 msgr2=0x7f89500c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.393+0000 7f8956c27700 1 --2- 172.21.15.31:0/2329383680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c7f10 0x7f89500c6a20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8938009840 tx=0x7f8938009b50 comp rx=0 tx=0).stop 2024-03-30T01:41:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.393+0000 7f8956c27700 1 -- 172.21.15.31:0/2329383680 shutdown_connections 2024-03-30T01:41:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.393+0000 7f8956c27700 1 --2- 172.21.15.31:0/2329383680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c7f10 0x7f89500c6a20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.393+0000 7f8956c27700 1 -- 172.21.15.31:0/2329383680 >> 172.21.15.31:0/2329383680 conn(0x7f8950072010 msgr2=0x7f8950072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.394+0000 7f8956c27700 1 -- 172.21.15.31:0/2329383680 shutdown_connections 2024-03-30T01:41:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.394+0000 7f8956c27700 1 -- 172.21.15.31:0/2329383680 wait complete. 2024-03-30T01:41:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.395+0000 7f8956c27700 1 Processor -- start 2024-03-30T01:41:54.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.395+0000 7f8956c27700 1 -- start start 2024-03-30T01:41:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f8956c27700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89500c7f10 0x7f89500c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f8956c27700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c8420 0x7f89500aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f8956c27700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8950151d80 0x7f89501ed830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f8956c27700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89500a99d0 con 0x7f89500c8420 2024-03-30T01:41:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f8956c27700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89500a9c20 con 0x7f89500c7f10 2024-03-30T01:41:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f8956c27700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89501ed0c0 con 0x7f8950151d80 2024-03-30T01:41:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c8420 0x7f89500aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c8420 0x7f89500aa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55706/0 (socket says 172.21.15.31:55706) 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894ffff700 1 -- 172.21.15.31:0/1568937893 learned_addr learned my addr 172.21.15.31:0/1568937893 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f89549c3700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89500c7f10 0x7f89500c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f89551c4700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8950151d80 0x7f89501ed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894ffff700 1 -- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8950151d80 msgr2=0x7f89501ed830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894ffff700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8950151d80 0x7f89501ed830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894ffff700 1 -- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89500c7f10 msgr2=0x7f89500c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894ffff700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89500c7f10 0x7f89500c2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894ffff700 1 -- 172.21.15.31:0/1568937893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380094a0 con 0x7f89500c8420 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f89549c3700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89500c7f10 0x7f89500c2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f89551c4700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8950151d80 0x7f89501ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894ffff700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c8420 0x7f89500aa7f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f89500cafe0 tx=0x7f8938004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.396+0000 7f894d7fa700 1 -- 172.21.15.31:0/1568937893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89380036a0 con 0x7f89500c8420 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.397+0000 7f894d7fa700 1 -- 172.21.15.31:0/1568937893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893801e070 con 0x7f89500c8420 2024-03-30T01:41:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.397+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89501ed3a0 con 0x7f89500c8420 2024-03-30T01:41:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.397+0000 7f894d7fa700 1 -- 172.21.15.31:0/1568937893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938003a40 con 0x7f89500c8420 2024-03-30T01:41:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.397+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89500ac800 con 0x7f89500c8420 2024-03-30T01:41:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.398+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89501ed530 con 0x7f89500c8420 2024-03-30T01:41:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.398+0000 7f894d7fa700 1 -- 172.21.15.31:0/1568937893 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8938003ba0 con 0x7f89500c8420 2024-03-30T01:41:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.399+0000 7f894d7fa700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f893006ac90 0x7f893006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.399+0000 7f894d7fa700 1 -- 172.21.15.31:0/1568937893 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f893808a8a0 con 0x7f89500c8420 2024-03-30T01:41:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.399+0000 7f89549c3700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f893006ac90 0x7f893006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.399+0000 7f89549c3700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f893006ac90 0x7f893006d150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8940000f60 tx=0x7f8940009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:54.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.403+0000 7f894d7fa700 1 -- 172.21.15.31:0/1568937893 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8938055750 con 0x7f89500c8420 2024-03-30T01:41:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.617+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89500747d0 con 0x7f89500c8420 2024-03-30T01:41:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.618+0000 7f894d7fa700 1 -- 172.21.15.31:0/1568937893 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89380200a0 con 0x7f89500c8420 2024-03-30T01:41:54.625 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:41:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f893006ac90 msgr2=0x7f893006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f893006ac90 0x7f893006d150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8940000f60 tx=0x7f8940009040 comp rx=0 tx=0).stop 2024-03-30T01:41:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c8420 msgr2=0x7f89500aa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c8420 0x7f89500aa7f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f89500cafe0 tx=0x7f8938004750 comp rx=0 tx=0).stop 2024-03-30T01:41:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 shutdown_connections 2024-03-30T01:41:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f893006ac90 0x7f893006d150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8950151d80 0x7f89501ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89500c7f10 0x7f89500c2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 --2- 172.21.15.31:0/1568937893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89500c8420 0x7f89500aa7f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 >> 172.21.15.31:0/1568937893 conn(0x7f8950072010 msgr2=0x7f89500723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 shutdown_connections 2024-03-30T01:41:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.620+0000 7f8956c27700 1 -- 172.21.15.31:0/1568937893 wait complete. 2024-03-30T01:41:54.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T01:41:54.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.772+0000 7f0c2b868700 1 Processor -- start 2024-03-30T01:41:54.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.773+0000 7f0c2b868700 1 -- start start 2024-03-30T01:41:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.773+0000 7f0c2b868700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c24151b60 0x7f0c24070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c2b868700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c24151f40 0x7f0c240c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c2b868700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c24154990 con 0x7f0c240c5d40 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c2b868700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c24155610 con 0x7f0c24151f40 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c2b868700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c241547a0 con 0x7f0c24151b60 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c29e05700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c24151f40 0x7f0c240c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c28e03700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0c240c5d40 0x7f0c240c2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46392/0 (socket says 172.21.15.31:46392) 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c28e03700 1 -- 172.21.15.31:0/2099917902 learned_addr learned my addr 172.21.15.31:0/2099917902 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c29604700 1 --2- 172.21.15.31:0/2099917902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c24151b60 0x7f0c24070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c1bfff700 1 -- 172.21.15.31:0/2099917902 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987627912 0 0) 0x7f0c24154990 con 0x7f0c240c5d40 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c1bfff700 1 -- 172.21.15.31:0/2099917902 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c100035b0 con 0x7f0c240c5d40 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c29e05700 1 -- 172.21.15.31:0/2099917902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c24151b60 msgr2=0x7f0c24070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c29e05700 1 --2- 172.21.15.31:0/2099917902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c24151b60 0x7f0c24070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c29e05700 1 -- 172.21.15.31:0/2099917902 >> v1:172.21.15.31:6789/0 conn(0x7f0c240c5d40 legacy=0x7f0c240c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c29e05700 1 -- 172.21.15.31:0/2099917902 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24154c70 con 0x7f0c24151f40 2024-03-30T01:41:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.774+0000 7f0c29604700 1 --2- 172.21.15.31:0/2099917902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c24151b60 0x7f0c24070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c29e05700 1 --2- 172.21.15.31:0/2099917902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c24151f40 0x7f0c240c6610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009dc0 tx=0x7f0c1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d92834a65ae0e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c1bfff700 1 -- 172.21.15.31:0/2099917902 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00e070 con 0x7f0c24151f40 2024-03-30T01:41:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c1bfff700 1 -- 172.21.15.31:0/2099917902 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c00bae0 con 0x7f0c24151f40 2024-03-30T01:41:54.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c1bfff700 1 -- 172.21.15.31:0/2099917902 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00be50 con 0x7f0c24151f40 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c2b868700 1 -- 172.21.15.31:0/2099917902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c24151f40 msgr2=0x7f0c240c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c2b868700 1 --2- 172.21.15.31:0/2099917902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c24151f40 0x7f0c240c6610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009dc0 tx=0x7f0c1c00b330 comp rx=0 tx=0).stop 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c2b868700 1 -- 172.21.15.31:0/2099917902 shutdown_connections 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c2b868700 1 --2- 172.21.15.31:0/2099917902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c24151f40 0x7f0c240c6610 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c2b868700 1 --2- 172.21.15.31:0/2099917902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c24151b60 0x7f0c24070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.775+0000 7f0c2b868700 1 -- 172.21.15.31:0/2099917902 >> 172.21.15.31:0/2099917902 conn(0x7f0c240c7d60 msgr2=0x7f0c240c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.776+0000 7f0c2b868700 1 -- 172.21.15.31:0/2099917902 shutdown_connections 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.776+0000 7f0c2b868700 1 -- 172.21.15.31:0/2099917902 wait complete. 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.777+0000 7f0c2b868700 1 Processor -- start 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.777+0000 7f0c2b868700 1 -- start start 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c2b868700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c240c5d40 0x7f0c2414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c2b868700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c24151b60 0x7f0c240abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c2b868700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24151f40 0x7f0c240ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c2b868700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c24151750 con 0x7f0c24151f40 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c2b868700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c241e93a0 con 0x7f0c24151b60 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c2b868700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2414db00 con 0x7f0c240c5d40 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29e05700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24151f40 0x7f0c240ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29e05700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24151f40 0x7f0c240ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55718/0 (socket says 172.21.15.31:55718) 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29e05700 1 -- 172.21.15.31:0/2985423772 learned_addr learned my addr 172.21.15.31:0/2985423772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c28e03700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c24151b60 0x7f0c240abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29604700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c240c5d40 0x7f0c2414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29e05700 1 -- 172.21.15.31:0/2985423772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c240c5d40 msgr2=0x7f0c2414efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29e05700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c240c5d40 0x7f0c2414efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29e05700 1 -- 172.21.15.31:0/2985423772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c24151b60 msgr2=0x7f0c240abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29e05700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c24151b60 0x7f0c240abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29e05700 1 -- 172.21.15.31:0/2985423772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c009a70 con 0x7f0c24151f40 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c29e05700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24151f40 0x7f0c240ac160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00d010 tx=0x7f0c1c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:54.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c19ffb700 1 -- 172.21.15.31:0/2985423772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c027480 con 0x7f0c24151f40 2024-03-30T01:41:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c19ffb700 1 -- 172.21.15.31:0/2985423772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c020070 con 0x7f0c24151f40 2024-03-30T01:41:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.778+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c24151470 con 0x7f0c24151f40 2024-03-30T01:41:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.779+0000 7f0c19ffb700 1 -- 172.21.15.31:0/2985423772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c027820 con 0x7f0c24151f40 2024-03-30T01:41:54.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.779+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c241e9890 con 0x7f0c24151f40 2024-03-30T01:41:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.780+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c240747d0 con 0x7f0c24151f40 2024-03-30T01:41:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.780+0000 7f0c19ffb700 1 -- 172.21.15.31:0/2985423772 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0c1c012430 con 0x7f0c24151f40 2024-03-30T01:41:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.781+0000 7f0c19ffb700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0c0806ac90 0x7f0c0806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.781+0000 7f0c19ffb700 1 -- 172.21.15.31:0/2985423772 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0c1c022070 con 0x7f0c24151f40 2024-03-30T01:41:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.781+0000 7f0c29604700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0c0806ac90 0x7f0c0806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:54.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.781+0000 7f0c29604700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0c0806ac90 0x7f0c0806d150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0c2406f780 tx=0x7f0c20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:54.786+0000 7f0c19ffb700 1 -- 172.21.15.31:0/2985423772 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0c1c056740 con 0x7f0c24151f40 2024-03-30T01:41:55.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.000+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0c240c53b0 con 0x7f0c24151f40 2024-03-30T01:41:55.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.001+0000 7f0c19ffb700 1 -- 172.21.15.31:0/2985423772 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0c1c059d60 con 0x7f0c24151f40 2024-03-30T01:41:55.007 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:41:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.003+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0c0806ac90 msgr2=0x7f0c0806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.003+0000 7f0c2b868700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0c0806ac90 0x7f0c0806d150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0c2406f780 tx=0x7f0c20008040 comp rx=0 tx=0).stop 2024-03-30T01:41:55.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.003+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24151f40 msgr2=0x7f0c240ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.003+0000 7f0c2b868700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24151f40 0x7f0c240ac160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00d010 tx=0x7f0c1c004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.004+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 shutdown_connections 2024-03-30T01:41:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.004+0000 7f0c2b868700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c24151f40 0x7f0c240ac160 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.004+0000 7f0c2b868700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0c0806ac90 0x7f0c0806d150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.004+0000 7f0c2b868700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c240c5d40 0x7f0c2414efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.004+0000 7f0c2b868700 1 --2- 172.21.15.31:0/2985423772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c24151b60 0x7f0c240abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.004+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 >> 172.21.15.31:0/2985423772 conn(0x7f0c240c7d60 msgr2=0x7f0c240c8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.004+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 shutdown_connections 2024-03-30T01:41:55.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.004+0000 7f0c2b868700 1 -- 172.21.15.31:0/2985423772 wait complete. 2024-03-30T01:41:55.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:55.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.153+0000 7f10b4777700 1 Processor -- start 2024-03-30T01:41:55.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.154+0000 7f10b4777700 1 -- start start 2024-03-30T01:41:55.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.154+0000 7f10b4777700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10ac14fbf0 0x7f10ac06a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b4777700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac06e4a0 con 0x7f10ac0a9560 2024-03-30T01:41:55.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b4777700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac155bf0 con 0x7f10ac06d9e0 2024-03-30T01:41:55.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b4777700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac156ac0 con 0x7f10ac14fbf0 2024-03-30T01:41:55.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b2d14700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f10ac0a9560 0x7f10ac0ab0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46408/0 (socket says 172.21.15.31:46408) 2024-03-30T01:41:55.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b2d14700 1 -- 172.21.15.31:0/3901023018 learned_addr learned my addr 172.21.15.31:0/3901023018 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:55.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691711994 0 0) 0x7f10ac06e4a0 con 0x7f10ac0a9560 2024-03-30T01:41:55.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10980035b0 con 0x7f10ac0a9560 2024-03-30T01:41:55.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b2513700 1 --2- 172.21.15.31:0/3901023018 >> v2:172.21.15.59:3300/0 conn(0x7f10ac14fbf0 0x7f10ac06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3286367168 0 0) 0x7f10980035b0 con 0x7f10ac0a9560 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 >> v2:172.21.15.59:3300/0 conn(0x7f10ac14fbf0 msgr2=0x7f10ac06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 --2- 172.21.15.31:0/3901023018 >> v2:172.21.15.59:3300/0 conn(0x7f10ac14fbf0 0x7f10ac06a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 >> v1:172.21.15.59:6789/0 conn(0x7f10ac06d9e0 legacy=0x7f10ac1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac06dee0 con 0x7f10ac0a9560 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f10a40019f0 con 0x7f10ac0a9560 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 >> v1:172.21.15.31:6789/0 conn(0x7f10ac0a9560 legacy=0x7f10ac0ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1098003a50 0x7f1098003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1098004850 0x7f1098004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1098005260 0x7f109800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109800c180 con 0x7f1098005260 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109800c2f0 con 0x7f1098003a50 2024-03-30T01:41:55.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.155+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109800c4b0 con 0x7f1098004850 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b2d14700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1098005260 0x7f109800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b2513700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1098003a50 0x7f1098003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b1d12700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1098004850 0x7f1098004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b2d14700 1 -- 172.21.15.31:0/3901023018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1098004850 msgr2=0x7f1098004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b2d14700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1098004850 0x7f1098004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b2d14700 1 -- 172.21.15.31:0/3901023018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1098003a50 msgr2=0x7f1098003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b2d14700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1098003a50 0x7f1098003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b2d14700 1 -- 172.21.15.31:0/3901023018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980035b0 con 0x7f1098005260 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b2d14700 1 -- 172.21.15.31:0/3901023018 reap_dead start 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b2d14700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1098005260 0x7f109800ba10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f10a4002230 tx=0x7f1098005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f517b5e5fafb8a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a400a250 con 0x7f1098005260 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a40072d0 con 0x7f1098005260 2024-03-30T01:41:55.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.156+0000 7f10b1511700 1 -- 172.21.15.31:0/3901023018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4007680 con 0x7f1098005260 2024-03-30T01:41:55.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.157+0000 7f10b4777700 1 -- 172.21.15.31:0/3901023018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1098005260 msgr2=0x7f109800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.157+0000 7f10b4777700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1098005260 0x7f109800ba10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f10a4002230 tx=0x7f1098005880 comp rx=0 tx=0).stop 2024-03-30T01:41:55.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.157+0000 7f10b4777700 1 -- 172.21.15.31:0/3901023018 shutdown_connections 2024-03-30T01:41:55.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.157+0000 7f10b4777700 1 --2- 172.21.15.31:0/3901023018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1098005260 0x7f109800ba10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.157+0000 7f10b4777700 1 -- 172.21.15.31:0/3901023018 >> 172.21.15.31:0/3901023018 conn(0x7f10ac0c3190 msgr2=0x7f10ac0c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:55.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.157+0000 7f10b4777700 1 -- 172.21.15.31:0/3901023018 shutdown_connections 2024-03-30T01:41:55.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.157+0000 7f10b4777700 1 -- 172.21.15.31:0/3901023018 wait complete. 2024-03-30T01:41:55.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.158+0000 7f10b4777700 1 Processor -- start 2024-03-30T01:41:55.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.158+0000 7f10b4777700 1 -- start start 2024-03-30T01:41:55.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b4777700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1098005260 0x7f10ac1eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b4777700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10ac06a8e0 0x7f10ac1e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b4777700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10ac1e8180 0x7f10ac1db030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b4777700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac1dd6f0 con 0x7f10ac1e8180 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b4777700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac1dd130 con 0x7f1098005260 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b4777700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac1dd410 con 0x7f10ac06a8e0 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b2d14700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10ac1e8180 0x7f10ac1db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b2d14700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10ac1e8180 0x7f10ac1db030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55746/0 (socket says 172.21.15.31:55746) 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b2d14700 1 -- 172.21.15.31:0/202877854 learned_addr learned my addr 172.21.15.31:0/202877854 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b2513700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1098005260 0x7f10ac1eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b1d12700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10ac06a8e0 0x7f10ac1e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b2d14700 1 -- 172.21.15.31:0/202877854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10ac06a8e0 msgr2=0x7f10ac1e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b2d14700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10ac06a8e0 0x7f10ac1e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b2d14700 1 -- 172.21.15.31:0/202877854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1098005260 msgr2=0x7f10ac1eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b2d14700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1098005260 0x7f10ac1eb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.159+0000 7f10b2d14700 1 -- 172.21.15.31:0/202877854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac1e7720 con 0x7f10ac1e8180 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.160+0000 7f10b2d14700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10ac1e8180 0x7f10ac1db030 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0a92e0 tx=0x7f10a4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.160+0000 7f10a2ffd700 1 -- 172.21.15.31:0/202877854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4024030 con 0x7f10ac1e8180 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.160+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac1dc350 con 0x7f10ac1e8180 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.160+0000 7f10a2ffd700 1 -- 172.21.15.31:0/202877854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a4007a10 con 0x7f10ac1e8180 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.160+0000 7f10a2ffd700 1 -- 172.21.15.31:0/202877854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a400a250 con 0x7f10ac1e8180 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.160+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac1508d0 con 0x7f10ac1e8180 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.161+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac04ea80 con 0x7f10ac1e8180 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.161+0000 7f10a2ffd700 1 -- 172.21.15.31:0/202877854 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f10a4027070 con 0x7f10ac1e8180 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.162+0000 7f10a2ffd700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f109006ac90 0x7f109006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.162+0000 7f10a2ffd700 1 -- 172.21.15.31:0/202877854 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f10a4025070 con 0x7f10ac1e8180 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.162+0000 7f10b2513700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f109006ac90 0x7f109006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.162+0000 7f10b2513700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f109006ac90 0x7f109006d150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f10ac06bd90 tx=0x7f10a8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.166+0000 7f10a2ffd700 1 -- 172.21.15.31:0/202877854 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f10a4056040 con 0x7f10ac1e8180 2024-03-30T01:41:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.429+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10ac1dde40 con 0x7f10ac1e8180 2024-03-30T01:41:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.430+0000 7f10a2ffd700 1 -- 172.21.15.31:0/202877854 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f10a4059660 con 0x7f10ac1e8180 2024-03-30T01:41:55.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:55.437 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:16.755934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f109006ac90 msgr2=0x7f109006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f109006ac90 0x7f109006d150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f10ac06bd90 tx=0x7f10a8009210 comp rx=0 tx=0).stop 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10ac1e8180 msgr2=0x7f10ac1db030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10ac1e8180 0x7f10ac1db030 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0a92e0 tx=0x7f10a4001cb0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 shutdown_connections 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10ac1e8180 0x7f10ac1db030 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f109006ac90 0x7f109006d150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1098005260 0x7f10ac1eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 --2- 172.21.15.31:0/202877854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10ac06a8e0 0x7f10ac1e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 >> 172.21.15.31:0/202877854 conn(0x7f10ac0c3190 msgr2=0x7f10ac0a8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:55.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 shutdown_connections 2024-03-30T01:41:55.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.433+0000 7f10b4777700 1 -- 172.21.15.31:0/202877854 wait complete. 2024-03-30T01:41:55.441 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-03-30T01:41:55.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.582+0000 7f03d0597700 1 Processor -- start 2024-03-30T01:41:55.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.582+0000 7f03d0597700 1 -- start start 2024-03-30T01:41:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.583+0000 7f03d0597700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c5df0 0x7f03c80c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.583+0000 7f03d0597700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03c80c2870 0x7f03c80a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.583+0000 7f03d0597700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c81556d0 con 0x7f03c80c8310 2024-03-30T01:41:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.583+0000 7f03d0597700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c81559b0 con 0x7f03c80c2870 2024-03-30T01:41:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.583+0000 7f03d0597700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c80c1f70 con 0x7f03c80c5df0 2024-03-30T01:41:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.583+0000 7f03ceb34700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03c80c2870 0x7f03c80a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.583+0000 7f03ceb34700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03c80c2870 0x7f03c80a6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59588/0 (socket says 172.21.15.31:59588) 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.583+0000 7f03ceb34700 1 -- 172.21.15.31:0/1312533873 learned_addr learned my addr 172.21.15.31:0/1312533873 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.583+0000 7f03cdb32700 1 --2- 172.21.15.31:0/1312533873 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c5df0 0x7f03c80c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.584+0000 7f03ceb34700 1 -- 172.21.15.31:0/1312533873 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c5df0 msgr2=0x7f03c80c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.584+0000 7f03ceb34700 1 --2- 172.21.15.31:0/1312533873 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c5df0 0x7f03c80c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.584+0000 7f03ceb34700 1 -- 172.21.15.31:0/1312533873 >> v1:172.21.15.59:6789/0 conn(0x7f03c80c8310 legacy=0x7f03c8151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.584+0000 7f03ceb34700 1 -- 172.21.15.31:0/1312533873 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c80c2250 con 0x7f03c80c2870 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.584+0000 7f03ceb34700 1 --2- 172.21.15.31:0/1312533873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c2870 0x7f03c80a6ad0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009e50 tx=0x7f03c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8215dd1c4a798f12 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.584+0000 7f03cd331700 1 -- 172.21.15.31:0/1312533873 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c000e070 con 0x7f03c80c2870 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.584+0000 7f03cd331700 1 -- 172.21.15.31:0/1312533873 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c000b910 con 0x7f03c80c2870 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.584+0000 7f03cd331700 1 -- 172.21.15.31:0/1312533873 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c000bcc0 con 0x7f03c80c2870 2024-03-30T01:41:55.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.585+0000 7f03d0597700 1 -- 172.21.15.31:0/1312533873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c2870 msgr2=0x7f03c80a6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.585+0000 7f03d0597700 1 --2- 172.21.15.31:0/1312533873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c2870 0x7f03c80a6ad0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009e50 tx=0x7f03c000b330 comp rx=0 tx=0).stop 2024-03-30T01:41:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.585+0000 7f03d0597700 1 -- 172.21.15.31:0/1312533873 shutdown_connections 2024-03-30T01:41:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.585+0000 7f03d0597700 1 --2- 172.21.15.31:0/1312533873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c2870 0x7f03c80a6ad0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.585+0000 7f03d0597700 1 --2- 172.21.15.31:0/1312533873 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c5df0 0x7f03c80c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.585+0000 7f03d0597700 1 -- 172.21.15.31:0/1312533873 >> 172.21.15.31:0/1312533873 conn(0x7f03c806ab20 msgr2=0x7f03c806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.585+0000 7f03d0597700 1 -- 172.21.15.31:0/1312533873 shutdown_connections 2024-03-30T01:41:55.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.585+0000 7f03d0597700 1 -- 172.21.15.31:0/1312533873 wait complete. 2024-03-30T01:41:55.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.586+0000 7f03d0597700 1 Processor -- start 2024-03-30T01:41:55.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.586+0000 7f03d0597700 1 -- start start 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03d0597700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c2870 0x7f03c80ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03d0597700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03c80c5df0 0x7f03c814e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03d0597700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c8310 0x7f03c814e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03d0597700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8158b20 con 0x7f03c80c8310 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03d0597700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c80a85d0 con 0x7f03c80c5df0 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03d0597700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c80a88b0 con 0x7f03c80c2870 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ceb34700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c8310 0x7f03c814e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ceb34700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c8310 0x7f03c814e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59598/0 (socket says 172.21.15.31:59598) 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ceb34700 1 -- 172.21.15.31:0/2624296345 learned_addr learned my addr 172.21.15.31:0/2624296345 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ce333700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c2870 0x7f03c80ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03cdb32700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03c80c5df0 0x7f03c814e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ceb34700 1 -- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c2870 msgr2=0x7f03c80ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ceb34700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c2870 0x7f03c80ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ceb34700 1 -- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03c80c5df0 msgr2=0x7f03c814e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ceb34700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03c80c5df0 0x7f03c814e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ceb34700 1 -- 172.21.15.31:0/2624296345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0009b30 con 0x7f03c80c8310 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03cdb32700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03c80c5df0 0x7f03c814e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ce333700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c2870 0x7f03c80ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03ceb34700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c8310 0x7f03c814e840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f03c80c8ef0 tx=0x7f03c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03beffd700 1 -- 172.21.15.31:0/2624296345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0004960 con 0x7f03c80c8310 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03beffd700 1 -- 172.21.15.31:0/2624296345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c00099b0 con 0x7f03c80c8310 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.587+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c8158880 con 0x7f03c80c8310 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.588+0000 7f03beffd700 1 -- 172.21.15.31:0/2624296345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c001c680 con 0x7f03c80c8310 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.588+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c80ac920 con 0x7f03c80c8310 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.589+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c8073760 con 0x7f03c80c8310 2024-03-30T01:41:55.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.589+0000 7f03beffd700 1 -- 172.21.15.31:0/2624296345 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f03c001c7e0 con 0x7f03c80c8310 2024-03-30T01:41:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.590+0000 7f03beffd700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f03ac06ac90 0x7f03ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.590+0000 7f03ce333700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f03ac06ac90 0x7f03ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.590+0000 7f03beffd700 1 -- 172.21.15.31:0/2624296345 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f03c008b370 con 0x7f03c80c8310 2024-03-30T01:41:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.590+0000 7f03ce333700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f03ac06ac90 0x7f03ac06d150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f60 tx=0x7f03c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:55.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.594+0000 7f03beffd700 1 -- 172.21.15.31:0/2624296345 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f03c0056220 con 0x7f03c80c8310 2024-03-30T01:41:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.810+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03c80a8330 con 0x7f03c80c8310 2024-03-30T01:41:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.811+0000 7f03beffd700 1 -- 172.21.15.31:0/2624296345 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18219 (secure 0 0 0) 0x7f03c80a8330 con 0x7f03c80c8310 2024-03-30T01:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:55.818 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:42.922828+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:55.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f03ac06ac90 msgr2=0x7f03ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f03ac06ac90 0x7f03ac06d150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f60 tx=0x7f03c4009040 comp rx=0 tx=0).stop 2024-03-30T01:41:55.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c8310 msgr2=0x7f03c814e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c8310 0x7f03c814e840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f03c80c8ef0 tx=0x7f03c0004ad0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 shutdown_connections 2024-03-30T01:41:55.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c80c8310 0x7f03c814e840 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f03ac06ac90 0x7f03ac06d150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03c80c2870 0x7f03c80ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 --2- 172.21.15.31:0/2624296345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03c80c5df0 0x7f03c814e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.813+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 >> 172.21.15.31:0/2624296345 conn(0x7f03c806ab20 msgr2=0x7f03c80c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.814+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 shutdown_connections 2024-03-30T01:41:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.814+0000 7f03d0597700 1 -- 172.21.15.31:0/2624296345 wait complete. 2024-03-30T01:41:55.834 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T01:41:55.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:55.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.964+0000 7f280333b700 1 Processor -- start 2024-03-30T01:41:55.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.964+0000 7f280333b700 1 -- start start 2024-03-30T01:41:55.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f280333b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 0x7f27fc0c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f280333b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f27fc0c46a0 0x7f27fc06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f280333b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f27fc072270 0x7f27fc14df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f280333b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27fc0a8460 con 0x7f27fc072270 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f280333b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27fc0c2e40 con 0x7f27fc0c46a0 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f280333b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc155630 con 0x7f27fc154cf0 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f28018d8700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f27fc072270 0x7f27fc14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f28008d6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f27fc0c46a0 0x7f27fc06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f28010d7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 0x7f27fc0c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f28008d6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f27fc0c46a0 0x7f27fc06d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:47260/0 (socket says 172.21.15.31:47260) 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f28008d6700 1 -- 172.21.15.31:0/85984057 learned_addr learned my addr 172.21.15.31:0/85984057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.965+0000 7f28010d7700 1 --2- 172.21.15.31:0/85984057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 0x7f27fc0c32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:48734/0 (socket says 172.21.15.31:48734) 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.966+0000 7f28018d8700 1 -- 172.21.15.31:0/85984057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 msgr2=0x7f27fc0c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.966+0000 7f28018d8700 1 --2- 172.21.15.31:0/85984057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 0x7f27fc0c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.966+0000 7f28018d8700 1 -- 172.21.15.31:0/85984057 >> v2:172.21.15.59:3300/0 conn(0x7f27fc0c46a0 msgr2=0x7f27fc06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.966+0000 7f28018d8700 1 --2- 172.21.15.31:0/85984057 >> v2:172.21.15.59:3300/0 conn(0x7f27fc0c46a0 0x7f27fc06d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.966+0000 7f28018d8700 1 -- 172.21.15.31:0/85984057 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc0a86b0 con 0x7f27fc072270 2024-03-30T01:41:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.966+0000 7f28008d6700 1 --2- 172.21.15.31:0/85984057 >> v2:172.21.15.59:3300/0 conn(0x7f27fc0c46a0 0x7f27fc06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:41:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.966+0000 7f28018d8700 1 --2- 172.21.15.31:0/85984057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc072270 0x7f27fc14df70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f27fc06f550 tx=0x7f27f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d44a4f4564e60c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.968+0000 7f27f3fff700 1 -- 172.21.15.31:0/85984057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f400e070 con 0x7f27fc072270 2024-03-30T01:41:55.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.968+0000 7f27f3fff700 1 -- 172.21.15.31:0/85984057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f400bae0 con 0x7f27fc072270 2024-03-30T01:41:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.968+0000 7f27f3fff700 1 -- 172.21.15.31:0/85984057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f400be90 con 0x7f27fc072270 2024-03-30T01:41:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.969+0000 7f280333b700 1 -- 172.21.15.31:0/85984057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc072270 msgr2=0x7f27fc14df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.969+0000 7f280333b700 1 --2- 172.21.15.31:0/85984057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc072270 0x7f27fc14df70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f27fc06f550 tx=0x7f27f400b330 comp rx=0 tx=0).stop 2024-03-30T01:41:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.969+0000 7f280333b700 1 -- 172.21.15.31:0/85984057 shutdown_connections 2024-03-30T01:41:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.969+0000 7f280333b700 1 --2- 172.21.15.31:0/85984057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc072270 0x7f27fc14df70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.969+0000 7f280333b700 1 --2- 172.21.15.31:0/85984057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 0x7f27fc0c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.969+0000 7f280333b700 1 --2- 172.21.15.31:0/85984057 >> v2:172.21.15.59:3300/0 conn(0x7f27fc0c46a0 0x7f27fc06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.969+0000 7f280333b700 1 -- 172.21.15.31:0/85984057 >> 172.21.15.31:0/85984057 conn(0x7f27fc0c3a60 msgr2=0x7f27fc0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.969+0000 7f280333b700 1 -- 172.21.15.31:0/85984057 shutdown_connections 2024-03-30T01:41:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.969+0000 7f280333b700 1 -- 172.21.15.31:0/85984057 wait complete. 2024-03-30T01:41:55.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.970+0000 7f280333b700 1 Processor -- start 2024-03-30T01:41:55.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.971+0000 7f280333b700 1 -- start start 2024-03-30T01:41:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.971+0000 7f280333b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27fc072270 0x7f27fc0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.971+0000 7f280333b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc0c46a0 0x7f27fc149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f280333b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 0x7f27fc14f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f280333b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc142730 con 0x7f27fc0c46a0 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f280333b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc0ab0b0 con 0x7f27fc072270 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f280333b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc0aae10 con 0x7f27fc154cf0 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28008d6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc0c46a0 0x7f27fc149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28008d6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc0c46a0 0x7f27fc149f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59618/0 (socket says 172.21.15.31:59618) 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28008d6700 1 -- 172.21.15.31:0/1153344495 learned_addr learned my addr 172.21.15.31:0/1153344495 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28010d7700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27fc072270 0x7f27fc0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28018d8700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 0x7f27fc14f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28008d6700 1 -- 172.21.15.31:0/1153344495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 msgr2=0x7f27fc14f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28008d6700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 0x7f27fc14f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28008d6700 1 -- 172.21.15.31:0/1153344495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27fc072270 msgr2=0x7f27fc0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28008d6700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27fc072270 0x7f27fc0ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28008d6700 1 -- 172.21.15.31:0/1153344495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec009550 con 0x7f27fc0c46a0 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f28008d6700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc0c46a0 0x7f27fc149f40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f27ec006e90 tx=0x7f27ec00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f27f1ffb700 1 -- 172.21.15.31:0/1153344495 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec004030 con 0x7f27fc0c46a0 2024-03-30T01:41:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f27f1ffb700 1 -- 172.21.15.31:0/1153344495 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ec004190 con 0x7f27fc0c46a0 2024-03-30T01:41:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f4009a70 con 0x7f27fc0c46a0 2024-03-30T01:41:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.972+0000 7f27f1ffb700 1 -- 172.21.15.31:0/1153344495 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec004520 con 0x7f27fc0c46a0 2024-03-30T01:41:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.973+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc0ac360 con 0x7f27fc0c46a0 2024-03-30T01:41:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.974+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27fc0abeb0 con 0x7f27fc0c46a0 2024-03-30T01:41:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.974+0000 7f27f1ffb700 1 -- 172.21.15.31:0/1153344495 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f27ec00c850 con 0x7f27fc0c46a0 2024-03-30T01:41:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.974+0000 7f27f1ffb700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f27e006acc0 0x7f27e006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.975+0000 7f27f1ffb700 1 -- 172.21.15.31:0/1153344495 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f27ec0893e0 con 0x7f27fc0c46a0 2024-03-30T01:41:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.975+0000 7f28010d7700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f27e006acc0 0x7f27e006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.975+0000 7f28010d7700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f27e006acc0 0x7f27e006d180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009b40 tx=0x7f27f8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:55.979+0000 7f27f1ffb700 1 -- 172.21.15.31:0/1153344495 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f27ec054290 con 0x7f27fc0c46a0 2024-03-30T01:41:56.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.244+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27fc0ac550 con 0x7f27fc0c46a0 2024-03-30T01:41:56.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.244+0000 7f27f1ffb700 1 -- 172.21.15.31:0/1153344495 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f27ec0578b0 con 0x7f27fc0c46a0 2024-03-30T01:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:56.251 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:16.755934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f27e006acc0 msgr2=0x7f27e006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f27e006acc0 0x7f27e006d180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009b40 tx=0x7f27f8009210 comp rx=0 tx=0).stop 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc0c46a0 msgr2=0x7f27fc149f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc0c46a0 0x7f27fc149f40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f27ec006e90 tx=0x7f27ec00ee90 comp rx=0 tx=0).stop 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 shutdown_connections 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f27e006acc0 0x7f27e006d180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27fc154cf0 0x7f27fc14f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27fc072270 0x7f27fc0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 --2- 172.21.15.31:0/1153344495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27fc0c46a0 0x7f27fc149f40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 >> 172.21.15.31:0/1153344495 conn(0x7f27fc0c3a60 msgr2=0x7f27fc147bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:56.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 shutdown_connections 2024-03-30T01:41:56.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.247+0000 7f280333b700 1 -- 172.21.15.31:0/1153344495 wait complete. 2024-03-30T01:41:56.255 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-03-30T01:41:56.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:41:56.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.398+0000 7f08f996e700 1 Processor -- start 2024-03-30T01:41:56.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.398+0000 7f08f996e700 1 -- start start 2024-03-30T01:41:56.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.399+0000 7f08f996e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08f414e5f0 0x7f08f40c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.399+0000 7f08f996e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f4071240 0x7f08f4154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.399+0000 7f08f996e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f08f40c4100 0x7f08f4154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.399+0000 7f08f996e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f4072160 con 0x7f08f414e5f0 2024-03-30T01:41:56.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.399+0000 7f08f996e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40c2bd0 con 0x7f08f40c4100 2024-03-30T01:41:56.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.399+0000 7f08f996e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40c2e90 con 0x7f08f4071240 2024-03-30T01:41:56.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.399+0000 7f08f2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08f414e5f0 0x7f08f40c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.399+0000 7f08f2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08f414e5f0 0x7f08f40c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59628/0 (socket says 172.21.15.31:59628) 2024-03-30T01:41:56.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.399+0000 7f08f2ffd700 1 -- 172.21.15.31:0/2253645357 learned_addr learned my addr 172.21.15.31:0/2253645357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:56.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f37fe700 1 --2- 172.21.15.31:0/2253645357 >> v2:172.21.15.59:3300/0 conn(0x7f08f40c4100 0x7f08f4154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f27fc700 1 --2- 172.21.15.31:0/2253645357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f4071240 0x7f08f4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f2ffd700 1 -- 172.21.15.31:0/2253645357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f4071240 msgr2=0x7f08f4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:56.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f2ffd700 1 --2- 172.21.15.31:0/2253645357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f4071240 0x7f08f4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f2ffd700 1 -- 172.21.15.31:0/2253645357 >> v2:172.21.15.59:3300/0 conn(0x7f08f40c4100 msgr2=0x7f08f4154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f2ffd700 1 --2- 172.21.15.31:0/2253645357 >> v2:172.21.15.59:3300/0 conn(0x7f08f40c4100 0x7f08f4154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f2ffd700 1 -- 172.21.15.31:0/2253645357 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40723b0 con 0x7f08f414e5f0 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f37fe700 1 --2- 172.21.15.31:0/2253645357 >> v2:172.21.15.59:3300/0 conn(0x7f08f40c4100 0x7f08f4154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f2ffd700 1 --2- 172.21.15.31:0/2253645357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 0x7f08f40c3bc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f08f40c0e50 tx=0x7f08dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aab15bf21965966 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f1ffb700 1 -- 172.21.15.31:0/2253645357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00b870 con 0x7f08f414e5f0 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f1ffb700 1 -- 172.21.15.31:0/2253645357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc00b9d0 con 0x7f08f414e5f0 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.400+0000 7f08f1ffb700 1 -- 172.21.15.31:0/2253645357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00bd60 con 0x7f08f414e5f0 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.401+0000 7f08f996e700 1 -- 172.21.15.31:0/2253645357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 msgr2=0x7f08f40c3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.401+0000 7f08f996e700 1 --2- 172.21.15.31:0/2253645357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 0x7f08f40c3bc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f08f40c0e50 tx=0x7f08dc009bd0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.401+0000 7f08f996e700 1 -- 172.21.15.31:0/2253645357 shutdown_connections 2024-03-30T01:41:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.401+0000 7f08f996e700 1 --2- 172.21.15.31:0/2253645357 >> v2:172.21.15.59:3300/0 conn(0x7f08f40c4100 0x7f08f4154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.401+0000 7f08f996e700 1 --2- 172.21.15.31:0/2253645357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 0x7f08f40c3bc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.401+0000 7f08f996e700 1 --2- 172.21.15.31:0/2253645357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f4071240 0x7f08f4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.401+0000 7f08f996e700 1 -- 172.21.15.31:0/2253645357 >> 172.21.15.31:0/2253645357 conn(0x7f08f406ba20 msgr2=0x7f08f406be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.401+0000 7f08f996e700 1 -- 172.21.15.31:0/2253645357 shutdown_connections 2024-03-30T01:41:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.401+0000 7f08f996e700 1 -- 172.21.15.31:0/2253645357 wait complete. 2024-03-30T01:41:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.402+0000 7f08f996e700 1 Processor -- start 2024-03-30T01:41:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.402+0000 7f08f996e700 1 -- start start 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f996e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08f4071240 0x7f08f40c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f996e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f40c4100 0x7f08f41ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f996e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 0x7f08f4153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f996e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4148b20 con 0x7f08f414e5f0 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f996e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f41ead30 con 0x7f08f4071240 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f996e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40c6b80 con 0x7f08f40c4100 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 0x7f08f4153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 0x7f08f4153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59634/0 (socket says 172.21.15.31:59634) 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f37fe700 1 -- 172.21.15.31:0/4058996190 learned_addr learned my addr 172.21.15.31:0/4058996190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f2ffd700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08f4071240 0x7f08f40c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.403+0000 7f08f27fc700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f40c4100 0x7f08f41ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08f37fe700 1 -- 172.21.15.31:0/4058996190 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f40c4100 msgr2=0x7f08f41ea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08f37fe700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f40c4100 0x7f08f41ea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08f37fe700 1 -- 172.21.15.31:0/4058996190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08f4071240 msgr2=0x7f08f40c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08f37fe700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08f4071240 0x7f08f40c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08f37fe700 1 -- 172.21.15.31:0/4058996190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc009550 con 0x7f08f414e5f0 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08f37fe700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 0x7f08f4153a40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f08ec007ef0 tx=0x7f08ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08ebfff700 1 -- 172.21.15.31:0/4058996190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00dcc0 con 0x7f08f414e5f0 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08ebfff700 1 -- 172.21.15.31:0/4058996190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec00f040 con 0x7f08f414e5f0 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f40c4b10 con 0x7f08f414e5f0 2024-03-30T01:41:56.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08ebfff700 1 -- 172.21.15.31:0/4058996190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec003c40 con 0x7f08f414e5f0 2024-03-30T01:41:56.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.404+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f40c50d0 con 0x7f08f414e5f0 2024-03-30T01:41:56.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.405+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f404ea80 con 0x7f08f414e5f0 2024-03-30T01:41:56.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.406+0000 7f08ebfff700 1 -- 172.21.15.31:0/4058996190 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f08ec004030 con 0x7f08f414e5f0 2024-03-30T01:41:56.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.406+0000 7f08ebfff700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f08d406ac90 0x7f08d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.406+0000 7f08ebfff700 1 -- 172.21.15.31:0/4058996190 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f08ec089650 con 0x7f08f414e5f0 2024-03-30T01:41:56.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.406+0000 7f08f2ffd700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f08d406ac90 0x7f08d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.407+0000 7f08f2ffd700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f08d406ac90 0x7f08d406d150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f08dc014040 tx=0x7f08dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:56.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.411+0000 7f08ebfff700 1 -- 172.21.15.31:0/4058996190 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f08ec054500 con 0x7f08f414e5f0 2024-03-30T01:41:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.671+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08f406d7a0 con 0x7f08f414e5f0 2024-03-30T01:41:56.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.672+0000 7f08ebfff700 1 -- 172.21.15.31:0/4058996190 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4727 (secure 0 0 0) 0x7f08ec057b20 con 0x7f08f414e5f0 2024-03-30T01:41:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:56.679 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3893326599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3893326599},{"type":"v1","addr":"172.21.15.59:6835","nonce":3893326599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/821540524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":821540524},{"type":"v1","addr":"172.21.15.59:6834","nonce":821540524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3476643406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3476643406},{"type":"v1","addr":"172.21.15.31:6836","nonce":3476643406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:40:58.401733+0000","modified":"2024-03-30T01:41:16.755934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6837/3724657244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3724657244},{"type":"v1","addr":"172.21.15.31:6837","nonce":3724657244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T01:41:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f08d406ac90 msgr2=0x7f08d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f08d406ac90 0x7f08d406d150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f08dc014040 tx=0x7f08dc008030 comp rx=0 tx=0).stop 2024-03-30T01:41:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 msgr2=0x7f08f4153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 0x7f08f4153a40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f08ec007ef0 tx=0x7f08ec00d3b0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 shutdown_connections 2024-03-30T01:41:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08f414e5f0 0x7f08f4153a40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f08d406ac90 0x7f08d406d150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08f4071240 0x7f08f40c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 --2- 172.21.15.31:0/4058996190 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08f40c4100 0x7f08f41ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 >> 172.21.15.31:0/4058996190 conn(0x7f08f406ba20 msgr2=0x7f08f40c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:56.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 shutdown_connections 2024-03-30T01:41:56.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.674+0000 7f08f996e700 1 -- 172.21.15.31:0/4058996190 wait complete. 2024-03-30T01:41:56.682 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-03-30T01:41:56.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:41:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.825+0000 7f4f88038700 1 Processor -- start 2024-03-30T01:41:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.825+0000 7f4f88038700 1 -- start start 2024-03-30T01:41:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f88038700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c5df0 0x7f4f800c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f88038700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f801556d0 con 0x7f4f800c8310 2024-03-30T01:41:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f88038700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f800c1f70 con 0x7f4f800c2870 2024-03-30T01:41:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f88038700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f801559b0 con 0x7f4f800c5df0 2024-03-30T01:41:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f85dd4700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4f800c8310 0x7f4f80151d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39584/0 (socket says 172.21.15.31:39584) 2024-03-30T01:41:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f85dd4700 1 -- 172.21.15.31:0/3832350143 learned_addr learned my addr 172.21.15.31:0/3832350143 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:41:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f855d3700 1 --2- 172.21.15.31:0/3832350143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c5df0 0x7f4f800c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f84dd2700 1 -- 172.21.15.31:0/3832350143 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722465989 0 0) 0x7f4f801556d0 con 0x7f4f800c8310 2024-03-30T01:41:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f84dd2700 1 -- 172.21.15.31:0/3832350143 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f6c0035b0 con 0x7f4f800c8310 2024-03-30T01:41:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f84dd2700 1 -- 172.21.15.31:0/3832350143 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529632212 0 0) 0x7f4f800c1f70 con 0x7f4f800c2870 2024-03-30T01:41:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f84dd2700 1 -- 172.21.15.31:0/3832350143 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f801556d0 con 0x7f4f800c2870 2024-03-30T01:41:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f855d3700 1 -- 172.21.15.31:0/3832350143 >> v1:172.21.15.59:6789/0 conn(0x7f4f800c2870 legacy=0x7f4f800a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f855d3700 1 -- 172.21.15.31:0/3832350143 >> v1:172.21.15.31:6789/0 conn(0x7f4f800c8310 legacy=0x7f4f80151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.826+0000 7f4f855d3700 1 -- 172.21.15.31:0/3832350143 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800c2250 con 0x7f4f800c5df0 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.827+0000 7f4f855d3700 1 --2- 172.21.15.31:0/3832350143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c5df0 0x7f4f800c86f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009870 tx=0x7f4f70009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d841cc2b2e82b564 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.827+0000 7f4f84dd2700 1 -- 172.21.15.31:0/3832350143 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70011830 con 0x7f4f800c5df0 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.827+0000 7f4f84dd2700 1 -- 172.21.15.31:0/3832350143 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f70011990 con 0x7f4f800c5df0 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.827+0000 7f4f84dd2700 1 -- 172.21.15.31:0/3832350143 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70011d20 con 0x7f4f800c5df0 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.827+0000 7f4f88038700 1 -- 172.21.15.31:0/3832350143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c5df0 msgr2=0x7f4f800c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.827+0000 7f4f88038700 1 --2- 172.21.15.31:0/3832350143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c5df0 0x7f4f800c86f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009870 tx=0x7f4f70009b80 comp rx=0 tx=0).stop 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.827+0000 7f4f88038700 1 -- 172.21.15.31:0/3832350143 shutdown_connections 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.827+0000 7f4f88038700 1 --2- 172.21.15.31:0/3832350143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c5df0 0x7f4f800c86f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.827+0000 7f4f88038700 1 -- 172.21.15.31:0/3832350143 >> 172.21.15.31:0/3832350143 conn(0x7f4f8006ab20 msgr2=0x7f4f8006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.828+0000 7f4f88038700 1 -- 172.21.15.31:0/3832350143 shutdown_connections 2024-03-30T01:41:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.828+0000 7f4f88038700 1 -- 172.21.15.31:0/3832350143 wait complete. 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.829+0000 7f4f88038700 1 Processor -- start 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.829+0000 7f4f88038700 1 -- start start 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.829+0000 7f4f88038700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f800c2870 0x7f4f800ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f88038700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f800c5df0 0x7f4f80151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f88038700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c8310 0x7f4f80151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f88038700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8009a2f0 con 0x7f4f800c2870 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f88038700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80151ba0 con 0x7f4f800c5df0 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f88038700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800a6750 con 0x7f4f800c8310 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f85dd4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f800c2870 0x7f4f800ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f85dd4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f800c2870 0x7f4f800ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59640/0 (socket says 172.21.15.31:59640) 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f85dd4700 1 -- 172.21.15.31:0/593692064 learned_addr learned my addr 172.21.15.31:0/593692064 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f865d5700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c8310 0x7f4f80151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f855d3700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f800c5df0 0x7f4f80151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f85dd4700 1 -- 172.21.15.31:0/593692064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c8310 msgr2=0x7f4f80151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f85dd4700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c8310 0x7f4f80151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f85dd4700 1 -- 172.21.15.31:0/593692064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f800c5df0 msgr2=0x7f4f80151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f85dd4700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f800c5df0 0x7f4f80151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f85dd4700 1 -- 172.21.15.31:0/593692064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f700094a0 con 0x7f4f800c2870 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f865d5700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c8310 0x7f4f80151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f85dd4700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f800c2870 0x7f4f800ac140 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00c4d0 tx=0x7f4f7c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f767fc700 1 -- 172.21.15.31:0/593692064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c005d70 con 0x7f4f800c2870 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f767fc700 1 -- 172.21.15.31:0/593692064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c0070d0 con 0x7f4f800c2870 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f800a6c80 con 0x7f4f800c2870 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f767fc700 1 -- 172.21.15.31:0/593692064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c007480 con 0x7f4f800c2870 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.830+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f800ad090 con 0x7f4f800c2870 2024-03-30T01:41:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.831+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f80073760 con 0x7f4f800c2870 2024-03-30T01:41:56.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.832+0000 7f4f767fc700 1 -- 172.21.15.31:0/593692064 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4f7c0075e0 con 0x7f4f800c2870 2024-03-30T01:41:56.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.833+0000 7f4f767fc700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4f6406ac90 0x7f4f6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:56.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.833+0000 7f4f855d3700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4f6406ac90 0x7f4f6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:56.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.833+0000 7f4f767fc700 1 -- 172.21.15.31:0/593692064 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4f7c088c10 con 0x7f4f800c2870 2024-03-30T01:41:56.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.833+0000 7f4f855d3700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4f6406ac90 0x7f4f6406d150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f800c7820 tx=0x7f4f7001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:56.837+0000 7f4f767fc700 1 -- 172.21.15.31:0/593692064 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4f7c053ac0 con 0x7f4f800c2870 2024-03-30T01:41:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.056+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f800ad220 con 0x7f4f800c2870 2024-03-30T01:41:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.058+0000 7f4f767fc700 1 -- 172.21.15.31:0/593692064 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18219 (secure 0 0 0) 0x7f4f7c017090 con 0x7f4f800c2870 2024-03-30T01:41:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:41:57.064 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:41:42.922828+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:40:56.325144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T01:40:57.374109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T01:41:00.796698+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:41:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4f6406ac90 msgr2=0x7f4f6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4f6406ac90 0x7f4f6406d150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f800c7820 tx=0x7f4f7001c040 comp rx=0 tx=0).stop 2024-03-30T01:41:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f800c2870 msgr2=0x7f4f800ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f800c2870 0x7f4f800ac140 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00c4d0 tx=0x7f4f7c00c890 comp rx=0 tx=0).stop 2024-03-30T01:41:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 shutdown_connections 2024-03-30T01:41:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f800c2870 0x7f4f800ac140 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4f6406ac90 0x7f4f6406d150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4f800c8310 0x7f4f80151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 --2- 172.21.15.31:0/593692064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f800c5df0 0x7f4f80151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 >> 172.21.15.31:0/593692064 conn(0x7f4f8006ab20 msgr2=0x7f4f8006c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:57.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 shutdown_connections 2024-03-30T01:41:57.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.060+0000 7f4f88038700 1 -- 172.21.15.31:0/593692064 wait complete. 2024-03-30T01:41:57.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T01:41:57.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.211+0000 7f7756bd4700 1 Processor -- start 2024-03-30T01:41:57.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.211+0000 7f7756bd4700 1 -- start start 2024-03-30T01:41:57.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.212+0000 7f7756bd4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 0x7f77500c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.212+0000 7f7756bd4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f775006d5c0 0x7f7750155300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.212+0000 7f7756bd4700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f775006e0d0 0x7f77500c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7756bd4700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77501522e0 con 0x7f775006e0d0 2024-03-30T01:41:57.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7756bd4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7750152000 con 0x7f775006d5c0 2024-03-30T01:41:57.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7756bd4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7750154b00 con 0x7f775014e910 2024-03-30T01:41:57.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7755171700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f775006e0d0 0x7f77500c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7755171700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f775006e0d0 0x7f77500c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59646/0 (socket says 172.21.15.31:59646) 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7755171700 1 -- 172.21.15.31:0/4279602381 learned_addr learned my addr 172.21.15.31:0/4279602381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7754970700 1 --2- 172.21.15.31:0/4279602381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 0x7f77500c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f774ffff700 1 --2- 172.21.15.31:0/4279602381 >> v2:172.21.15.59:3300/0 conn(0x7f775006d5c0 0x7f7750155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7755171700 1 -- 172.21.15.31:0/4279602381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 msgr2=0x7f77500c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7755171700 1 --2- 172.21.15.31:0/4279602381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 0x7f77500c1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7755171700 1 -- 172.21.15.31:0/4279602381 >> v2:172.21.15.59:3300/0 conn(0x7f775006d5c0 msgr2=0x7f7750155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7755171700 1 --2- 172.21.15.31:0/4279602381 >> v2:172.21.15.59:3300/0 conn(0x7f775006d5c0 0x7f7750155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7755171700 1 -- 172.21.15.31:0/4279602381 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750154de0 con 0x7f775006e0d0 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f7755171700 1 --2- 172.21.15.31:0/4279602381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 0x7f77500c3c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f77500c7f40 tx=0x7f774800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0a4564507bd333 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:57.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.213+0000 7f774f7fe700 1 -- 172.21.15.31:0/4279602381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774800e070 con 0x7f775006e0d0 2024-03-30T01:41:57.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f774f7fe700 1 -- 172.21.15.31:0/4279602381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774800b910 con 0x7f775006e0d0 2024-03-30T01:41:57.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f774f7fe700 1 -- 172.21.15.31:0/4279602381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774800bcc0 con 0x7f775006e0d0 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f7756bd4700 1 -- 172.21.15.31:0/4279602381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 msgr2=0x7f77500c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f7756bd4700 1 --2- 172.21.15.31:0/4279602381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 0x7f77500c3c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f77500c7f40 tx=0x7f774800b330 comp rx=0 tx=0).stop 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f7756bd4700 1 -- 172.21.15.31:0/4279602381 shutdown_connections 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f7756bd4700 1 --2- 172.21.15.31:0/4279602381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 0x7f77500c3c90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f7756bd4700 1 --2- 172.21.15.31:0/4279602381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 0x7f77500c1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f7756bd4700 1 --2- 172.21.15.31:0/4279602381 >> v2:172.21.15.59:3300/0 conn(0x7f775006d5c0 0x7f7750155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f7756bd4700 1 -- 172.21.15.31:0/4279602381 >> 172.21.15.31:0/4279602381 conn(0x7f775006ba20 msgr2=0x7f775006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f7756bd4700 1 -- 172.21.15.31:0/4279602381 shutdown_connections 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.214+0000 7f7756bd4700 1 -- 172.21.15.31:0/4279602381 wait complete. 2024-03-30T01:41:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.216+0000 7f7756bd4700 1 Processor -- start 2024-03-30T01:41:57.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.216+0000 7f7756bd4700 1 -- start start 2024-03-30T01:41:57.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f7756bd4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f775006d5c0 0x7f7750149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f7756bd4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 0x7f77501ead90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f7756bd4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 0x7f77501eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f7756bd4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77500a9420 con 0x7f775006e0d0 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f7756bd4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77501e7720 con 0x7f775006d5c0 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f7756bd4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775014ab80 con 0x7f775014e910 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f774ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 0x7f77501ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f774ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 0x7f77501ead90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59660/0 (socket says 172.21.15.31:59660) 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f774ffff700 1 -- 172.21.15.31:0/146099598 learned_addr learned my addr 172.21.15.31:0/146099598 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f7755171700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 0x7f77501eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.217+0000 7f7754970700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f775006d5c0 0x7f7750149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f774ffff700 1 -- 172.21.15.31:0/146099598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 msgr2=0x7f77501eb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f774ffff700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 0x7f77501eb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f774ffff700 1 -- 172.21.15.31:0/146099598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f775006d5c0 msgr2=0x7f7750149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f774ffff700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f775006d5c0 0x7f7750149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f774ffff700 1 -- 172.21.15.31:0/146099598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7748009b30 con 0x7f775006e0d0 2024-03-30T01:41:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f774ffff700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 0x7f77501ead90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f773800e870 tx=0x7f773800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f774d7fa700 1 -- 172.21.15.31:0/146099598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738013680 con 0x7f775006e0d0 2024-03-30T01:41:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f774d7fa700 1 -- 172.21.15.31:0/146099598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77380137e0 con 0x7f775006e0d0 2024-03-30T01:41:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77501e7930 con 0x7f775006e0d0 2024-03-30T01:41:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f774d7fa700 1 -- 172.21.15.31:0/146099598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738013b90 con 0x7f775006e0d0 2024-03-30T01:41:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.218+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77501e7eb0 con 0x7f775006e0d0 2024-03-30T01:41:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.219+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775014b770 con 0x7f775006e0d0 2024-03-30T01:41:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.220+0000 7f774d7fa700 1 -- 172.21.15.31:0/146099598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7738013cf0 con 0x7f775006e0d0 2024-03-30T01:41:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.220+0000 7f774d7fa700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f773006ac90 0x7f773006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.220+0000 7f774d7fa700 1 -- 172.21.15.31:0/146099598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f77380893f0 con 0x7f775006e0d0 2024-03-30T01:41:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.220+0000 7f7754970700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f773006ac90 0x7f773006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.221+0000 7f7754970700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f773006ac90 0x7f773006d150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f77501e5560 tx=0x7f7740009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:57.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.225+0000 7f774d7fa700 1 -- 172.21.15.31:0/146099598 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f77380542a0 con 0x7f775006e0d0 2024-03-30T01:41:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.487+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f775004ea80 con 0x7f775006e0d0 2024-03-30T01:41:57.495 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-03-30T01:41:57.488+0000 7fd8b60ee700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T01:41:57.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.496+0000 7f774d7fa700 1 -- 172.21.15.31:0/146099598 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f77380578c0 con 0x7f775006e0d0 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f773006ac90 msgr2=0x7f773006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f773006ac90 0x7f773006d150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f77501e5560 tx=0x7f7740009210 comp rx=0 tx=0).stop 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 msgr2=0x7f77501ead90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 0x7f77501ead90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f773800e870 tx=0x7f773800eb80 comp rx=0 tx=0).stop 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 shutdown_connections 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f773006ac90 0x7f773006d150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f775014e910 0x7f77501eb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f775006d5c0 0x7f7750149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 --2- 172.21.15.31:0/146099598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f775006e0d0 0x7f77501ead90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 >> 172.21.15.31:0/146099598 conn(0x7f775006ba20 msgr2=0x7f775006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 shutdown_connections 2024-03-30T01:41:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.499+0000 7f7756bd4700 1 -- 172.21.15.31:0/146099598 wait complete. 2024-03-30T01:41:57.507 INFO:tasks.ceph.mds.a.smithi031.stderr: -4409> 2024-03-30T01:41:03.669+0000 7f0491f37700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T01:41:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T01:41:57.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T01:41:57.540 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-03-30T01:41:57.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.652+0000 7f2521255700 1 Processor -- start 2024-03-30T01:41:57.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.652+0000 7f2521255700 1 -- start start 2024-03-30T01:41:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.653+0000 7f2521255700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f251c14ea10 0x7f251c06d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.653+0000 7f2521255700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 0x7f251c06dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.653+0000 7f2521255700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251c0c1e20 con 0x7f251c0aa110 2024-03-30T01:41:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.653+0000 7f2521255700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251c06e1c0 con 0x7f251c14ea10 2024-03-30T01:41:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.653+0000 7f2521255700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c0c1b80 con 0x7f251c06bd30 2024-03-30T01:41:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.653+0000 7f251ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f251c14ea10 0x7f251c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.653+0000 7f251ad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f251c14ea10 0x7f251c06d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59686/0 (socket says 172.21.15.31:59686) 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.653+0000 7f251ad9d700 1 -- 172.21.15.31:0/2469347988 learned_addr learned my addr 172.21.15.31:0/2469347988 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.653+0000 7f251b59e700 1 --2- 172.21.15.31:0/2469347988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 0x7f251c06dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f251ad9d700 1 -- 172.21.15.31:0/2469347988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 msgr2=0x7f251c06dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f251ad9d700 1 --2- 172.21.15.31:0/2469347988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 0x7f251c06dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f251ad9d700 1 -- 172.21.15.31:0/2469347988 >> v1:172.21.15.59:6789/0 conn(0x7f251c0aa110 legacy=0x7f251c154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f251ad9d700 1 -- 172.21.15.31:0/2469347988 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c0c2700 con 0x7f251c14ea10 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f251ad9d700 1 --2- 172.21.15.31:0/2469347988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c14ea10 0x7f251c06d740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f250c0098f0 tx=0x7f250c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bb62f329cea6ea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f2519d9b700 1 -- 172.21.15.31:0/2469347988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00f8a0 con 0x7f251c14ea10 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f2519d9b700 1 -- 172.21.15.31:0/2469347988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c00fa00 con 0x7f251c14ea10 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f2519d9b700 1 -- 172.21.15.31:0/2469347988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00fd90 con 0x7f251c14ea10 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f2521255700 1 -- 172.21.15.31:0/2469347988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c14ea10 msgr2=0x7f251c06d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f2521255700 1 --2- 172.21.15.31:0/2469347988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c14ea10 0x7f251c06d740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f250c0098f0 tx=0x7f250c009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:57.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f2521255700 1 -- 172.21.15.31:0/2469347988 shutdown_connections 2024-03-30T01:41:57.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f2521255700 1 --2- 172.21.15.31:0/2469347988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c14ea10 0x7f251c06d740 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f2521255700 1 --2- 172.21.15.31:0/2469347988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 0x7f251c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.654+0000 7f2521255700 1 -- 172.21.15.31:0/2469347988 >> 172.21.15.31:0/2469347988 conn(0x7f251c0c6550 msgr2=0x7f251c0c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:57.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.655+0000 7f2521255700 1 -- 172.21.15.31:0/2469347988 shutdown_connections 2024-03-30T01:41:57.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.655+0000 7f2521255700 1 -- 172.21.15.31:0/2469347988 wait complete. 2024-03-30T01:41:57.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.656+0000 7f2521255700 1 Processor -- start 2024-03-30T01:41:57.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.656+0000 7f2521255700 1 -- start start 2024-03-30T01:41:57.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f2521255700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 0x7f251c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f2521255700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c0aa110 0x7f251c1e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f2521255700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c14ea10 0x7f251c1e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f2521255700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c072930 con 0x7f251c0aa110 2024-03-30T01:41:57.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f2521255700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c152ce0 con 0x7f251c14ea10 2024-03-30T01:41:57.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f2521255700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c072d50 con 0x7f251c06bd30 2024-03-30T01:41:57.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c0aa110 0x7f251c1e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c0aa110 0x7f251c1e58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59688/0 (socket says 172.21.15.31:59688) 2024-03-30T01:41:57.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251a59c700 1 -- 172.21.15.31:0/3282093366 learned_addr learned my addr 172.21.15.31:0/3282093366 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:57.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251ad9d700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 0x7f251c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251b59e700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c14ea10 0x7f251c1e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251a59c700 1 -- 172.21.15.31:0/3282093366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 msgr2=0x7f251c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251a59c700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 0x7f251c070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251a59c700 1 -- 172.21.15.31:0/3282093366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c14ea10 msgr2=0x7f251c1e5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:57.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251a59c700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c14ea10 0x7f251c1e5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:57.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.657+0000 7f251a59c700 1 -- 172.21.15.31:0/3282093366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c009550 con 0x7f251c0aa110 2024-03-30T01:41:57.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.658+0000 7f251a59c700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c0aa110 0x7f251c1e58a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f251000c4e0 tx=0x7f251000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:57.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.658+0000 7f25037fe700 1 -- 172.21.15.31:0/3282093366 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100052d0 con 0x7f251c0aa110 2024-03-30T01:41:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.658+0000 7f25037fe700 1 -- 172.21.15.31:0/3282093366 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2510005430 con 0x7f251c0aa110 2024-03-30T01:41:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.658+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c152a00 con 0x7f251c0aa110 2024-03-30T01:41:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.658+0000 7f25037fe700 1 -- 172.21.15.31:0/3282093366 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100038c0 con 0x7f251c0aa110 2024-03-30T01:41:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.658+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c1534e0 con 0x7f251c0aa110 2024-03-30T01:41:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.659+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c0c2290 con 0x7f251c0aa110 2024-03-30T01:41:57.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.659+0000 7f25037fe700 1 -- 172.21.15.31:0/3282093366 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2510003a20 con 0x7f251c0aa110 2024-03-30T01:41:57.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.660+0000 7f25037fe700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f250806ac90 0x7f250806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:57.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.660+0000 7f251ad9d700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f250806ac90 0x7f250806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:57.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.660+0000 7f25037fe700 1 -- 172.21.15.31:0/3282093366 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f25100889f0 con 0x7f251c0aa110 2024-03-30T01:41:57.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.661+0000 7f251ad9d700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f250806ac90 0x7f250806d150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f250c009c40 tx=0x7f250c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:57.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.665+0000 7f25037fe700 1 -- 172.21.15.31:0/3282093366 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f251c0c2290 con 0x7f251c0aa110 2024-03-30T01:41:57.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:57.930+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f251c072a70 con 0x7f251c0aa110 2024-03-30T01:41:58.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:41:58.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:41:58.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.501+0000 7f25037fe700 1 -- 172.21.15.31:0/3282093366 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f251c072a70 con 0x7f251c0aa110 2024-03-30T01:41:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f250806ac90 msgr2=0x7f250806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f250806ac90 0x7f250806d150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f250c009c40 tx=0x7f250c008030 comp rx=0 tx=0).stop 2024-03-30T01:41:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c0aa110 msgr2=0x7f251c1e58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c0aa110 0x7f251c1e58a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f251000c4e0 tx=0x7f251000c7f0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 shutdown_connections 2024-03-30T01:41:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f250806ac90 0x7f250806d150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251c14ea10 0x7f251c1e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f251c06bd30 0x7f251c070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 --2- 172.21.15.31:0/3282093366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c0aa110 0x7f251c1e58a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 >> 172.21.15.31:0/3282093366 conn(0x7f251c0c6550 msgr2=0x7f251c14c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 shutdown_connections 2024-03-30T01:41:58.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.504+0000 7f2521255700 1 -- 172.21.15.31:0/3282093366 wait complete. 2024-03-30T01:41:58.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T01:41:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.664+0000 7f47bc02d700 1 Processor -- start 2024-03-30T01:41:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.665+0000 7f47bc02d700 1 -- start start 2024-03-30T01:41:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.665+0000 7f47bc02d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f47b413dba0 0x7f47b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.665+0000 7f47bc02d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b4158540 0x7f47b415f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.665+0000 7f47bc02d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b40c0c50 con 0x7f47b406ecb0 2024-03-30T01:41:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.665+0000 7f47bc02d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b41556a0 con 0x7f47b413dba0 2024-03-30T01:41:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.665+0000 7f47bc02d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b40c0320 con 0x7f47b4158540 2024-03-30T01:41:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b9dc9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f47b413dba0 0x7f47b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b9dc9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f47b413dba0 0x7f47b406e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59704/0 (socket says 172.21.15.31:59704) 2024-03-30T01:41:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b95c8700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f47b406ecb0 0x7f47b4157e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39602/0 (socket says 172.21.15.31:39602) 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b9dc9700 1 -- 172.21.15.31:0/1993891568 learned_addr learned my addr 172.21.15.31:0/1993891568 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47ba5ca700 1 --2- 172.21.15.31:0/1993891568 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b4158540 0x7f47b415f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b9dc9700 1 -- 172.21.15.31:0/1993891568 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b4158540 msgr2=0x7f47b415f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b9dc9700 1 --2- 172.21.15.31:0/1993891568 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b4158540 0x7f47b415f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b9dc9700 1 -- 172.21.15.31:0/1993891568 >> v1:172.21.15.31:6789/0 conn(0x7f47b406ecb0 legacy=0x7f47b4157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b9dc9700 1 -- 172.21.15.31:0/1993891568 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b40c0600 con 0x7f47b413dba0 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b8dc7700 1 -- 172.21.15.31:0/1993891568 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303146369 0 0) 0x7f47b40c0c50 con 0x7f47b406ecb0 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b9dc9700 1 --2- 172.21.15.31:0/1993891568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b413dba0 0x7f47b406e770 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f47b00098f0 tx=0x7f47b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d5c7625cd36620 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b8dc7700 1 -- 172.21.15.31:0/1993891568 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000b8a0 con 0x7f47b413dba0 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b8dc7700 1 -- 172.21.15.31:0/1993891568 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b000ba00 con 0x7f47b413dba0 2024-03-30T01:41:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.666+0000 7f47b8dc7700 1 -- 172.21.15.31:0/1993891568 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000bd90 con 0x7f47b413dba0 2024-03-30T01:41:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.667+0000 7f47bc02d700 1 -- 172.21.15.31:0/1993891568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b413dba0 msgr2=0x7f47b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.667+0000 7f47bc02d700 1 --2- 172.21.15.31:0/1993891568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b413dba0 0x7f47b406e770 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f47b00098f0 tx=0x7f47b0009c00 comp rx=0 tx=0).stop 2024-03-30T01:41:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.667+0000 7f47bc02d700 1 -- 172.21.15.31:0/1993891568 shutdown_connections 2024-03-30T01:41:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.667+0000 7f47bc02d700 1 --2- 172.21.15.31:0/1993891568 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b4158540 0x7f47b415f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.667+0000 7f47bc02d700 1 --2- 172.21.15.31:0/1993891568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b413dba0 0x7f47b406e770 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.667+0000 7f47bc02d700 1 -- 172.21.15.31:0/1993891568 >> 172.21.15.31:0/1993891568 conn(0x7f47b40c39f0 msgr2=0x7f47b40c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.667+0000 7f47bc02d700 1 -- 172.21.15.31:0/1993891568 shutdown_connections 2024-03-30T01:41:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.667+0000 7f47bc02d700 1 -- 172.21.15.31:0/1993891568 wait complete. 2024-03-30T01:41:58.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.668+0000 7f47bc02d700 1 Processor -- start 2024-03-30T01:41:58.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.668+0000 7f47bc02d700 1 -- start start 2024-03-30T01:41:58.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.669+0000 7f47bc02d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b406ecb0 0x7f47b41ed850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:58.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.669+0000 7f47bc02d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b413dba0 0x7f47b41576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:58.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.669+0000 7f47bc02d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b4158540 0x7f47b40abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:58.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.669+0000 7f47bc02d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b4157f20 con 0x7f47b4158540 2024-03-30T01:41:58.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.669+0000 7f47bc02d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b4158200 con 0x7f47b413dba0 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.669+0000 7f47bc02d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b41ecd70 con 0x7f47b406ecb0 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.669+0000 7f47ba5ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b4158540 0x7f47b40abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.669+0000 7f47ba5ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b4158540 0x7f47b40abf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59706/0 (socket says 172.21.15.31:59706) 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.669+0000 7f47ba5ca700 1 -- 172.21.15.31:0/2468294854 learned_addr learned my addr 172.21.15.31:0/2468294854 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47b95c8700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b413dba0 0x7f47b41576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47b9dc9700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b406ecb0 0x7f47b41ed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47ba5ca700 1 -- 172.21.15.31:0/2468294854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b406ecb0 msgr2=0x7f47b41ed850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47ba5ca700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b406ecb0 0x7f47b41ed850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47ba5ca700 1 -- 172.21.15.31:0/2468294854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b413dba0 msgr2=0x7f47b41576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47ba5ca700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b413dba0 0x7f47b41576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47ba5ca700 1 -- 172.21.15.31:0/2468294854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b0009550 con 0x7f47b4158540 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47ba5ca700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b4158540 0x7f47b40abf00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00cc30 tx=0x7f47ac00a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47a67fc700 1 -- 172.21.15.31:0/2468294854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac00ab70 con 0x7f47b4158540 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47a67fc700 1 -- 172.21.15.31:0/2468294854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac00acd0 con 0x7f47b4158540 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47a67fc700 1 -- 172.21.15.31:0/2468294854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac003c40 con 0x7f47b4158540 2024-03-30T01:41:58.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b41ed050 con 0x7f47b4158540 2024-03-30T01:41:58.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.670+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b4157c30 con 0x7f47b4158540 2024-03-30T01:41:58.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.671+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b41ed570 con 0x7f47b4158540 2024-03-30T01:41:58.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.672+0000 7f47a67fc700 1 -- 172.21.15.31:0/2468294854 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f47ac00f3f0 con 0x7f47b4158540 2024-03-30T01:41:58.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.672+0000 7f47a67fc700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f479806ac90 0x7f479806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:58.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.672+0000 7f47a67fc700 1 -- 172.21.15.31:0/2468294854 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f47ac089890 con 0x7f47b4158540 2024-03-30T01:41:58.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.672+0000 7f47b9dc9700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f479806ac90 0x7f479806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:58.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.673+0000 7f47b9dc9700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f479806ac90 0x7f479806d150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47b0000c00 tx=0x7f47b0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:58.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.677+0000 7f47a67fc700 1 -- 172.21.15.31:0/2468294854 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f47b41ed570 con 0x7f47b4158540 2024-03-30T01:41:58.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:58.894+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f47b41ed570 con 0x7f47b4158540 2024-03-30T01:41:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.506+0000 7f47a67fc700 1 -- 172.21.15.31:0/2468294854 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f47b41ed570 con 0x7f47b4158540 2024-03-30T01:41:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f479806ac90 msgr2=0x7f479806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f479806ac90 0x7f479806d150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47b0000c00 tx=0x7f47b0008030 comp rx=0 tx=0).stop 2024-03-30T01:41:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b4158540 msgr2=0x7f47b40abf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b4158540 0x7f47b40abf00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00cc30 tx=0x7f47ac00a1f0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 shutdown_connections 2024-03-30T01:41:59.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47b4158540 0x7f47b40abf00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f479806ac90 0x7f479806d150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b406ecb0 0x7f47b41ed850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 --2- 172.21.15.31:0/2468294854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b413dba0 0x7f47b41576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 >> 172.21.15.31:0/2468294854 conn(0x7f47b40c39f0 msgr2=0x7f47b40c2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:59.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 shutdown_connections 2024-03-30T01:41:59.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.511+0000 7f47bc02d700 1 -- 172.21.15.31:0/2468294854 wait complete. 2024-03-30T01:41:59.519 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-03-30T01:41:59.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T01:41:59.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.675+0000 7ff911ac4700 1 Processor -- start 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.675+0000 7ff911ac4700 1 -- start start 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.676+0000 7ff911ac4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 0x7ff90c0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.676+0000 7ff911ac4700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff90c0c2840 0x7ff90c0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.676+0000 7ff911ac4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c0c47a0 con 0x7ff90c155640 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.676+0000 7ff911ac4700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90c06eec0 con 0x7ff90c0c2840 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.676+0000 7ff911ac4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0c7dc0 con 0x7ff90c0c6a20 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.676+0000 7ff90bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff90c0c2840 0x7ff90c0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.676+0000 7ff90bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff90c0c2840 0x7ff90c0c5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59710/0 (socket says 172.21.15.31:59710) 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.676+0000 7ff90bfff700 1 -- 172.21.15.31:0/1900836539 learned_addr learned my addr 172.21.15.31:0/1900836539 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.677+0000 7ff90affd700 1 --2- 172.21.15.31:0/1900836539 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 0x7ff90c0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.677+0000 7ff90bfff700 1 -- 172.21.15.31:0/1900836539 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 msgr2=0x7ff90c0c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.677+0000 7ff90bfff700 1 --2- 172.21.15.31:0/1900836539 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 0x7ff90c0c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.677+0000 7ff90bfff700 1 -- 172.21.15.31:0/1900836539 >> v1:172.21.15.59:6789/0 conn(0x7ff90c155640 legacy=0x7ff90c0c37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.677+0000 7ff90bfff700 1 -- 172.21.15.31:0/1900836539 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c0c80a0 con 0x7ff90c0c2840 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.677+0000 7ff90bfff700 1 --2- 172.21.15.31:0/1900836539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c0c2840 0x7ff90c0c5170 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff904009e50 tx=0x7ff90400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca8bce3017017d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.677+0000 7ff90a7fc700 1 -- 172.21.15.31:0/1900836539 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90400e070 con 0x7ff90c0c2840 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.677+0000 7ff90a7fc700 1 -- 172.21.15.31:0/1900836539 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff904003a20 con 0x7ff90c0c2840 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.677+0000 7ff90a7fc700 1 -- 172.21.15.31:0/1900836539 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904003dd0 con 0x7ff90c0c2840 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.678+0000 7ff911ac4700 1 -- 172.21.15.31:0/1900836539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c0c2840 msgr2=0x7ff90c0c5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.678+0000 7ff911ac4700 1 --2- 172.21.15.31:0/1900836539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c0c2840 0x7ff90c0c5170 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff904009e50 tx=0x7ff90400b330 comp rx=0 tx=0).stop 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.678+0000 7ff911ac4700 1 -- 172.21.15.31:0/1900836539 shutdown_connections 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.678+0000 7ff911ac4700 1 --2- 172.21.15.31:0/1900836539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c0c2840 0x7ff90c0c5170 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.678+0000 7ff911ac4700 1 --2- 172.21.15.31:0/1900836539 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 0x7ff90c0c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.678+0000 7ff911ac4700 1 -- 172.21.15.31:0/1900836539 >> 172.21.15.31:0/1900836539 conn(0x7ff90c06a700 msgr2=0x7ff90c06aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.678+0000 7ff911ac4700 1 -- 172.21.15.31:0/1900836539 shutdown_connections 2024-03-30T01:41:59.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.678+0000 7ff911ac4700 1 -- 172.21.15.31:0/1900836539 wait complete. 2024-03-30T01:41:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.679+0000 7ff911ac4700 1 Processor -- start 2024-03-30T01:41:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.679+0000 7ff911ac4700 1 -- start start 2024-03-30T01:41:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff911ac4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c0c2840 0x7ff90c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff911ac4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 0x7ff90c155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff911ac4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c155640 0x7ff90c14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:59.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff911ac4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c1ea4a0 con 0x7ff90c155640 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff911ac4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c1e9930 con 0x7ff90c0c2840 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff911ac4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0a6a50 con 0x7ff90c0c6a20 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff90bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c155640 0x7ff90c14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff90bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c155640 0x7ff90c14ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59722/0 (socket says 172.21.15.31:59722) 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff90bfff700 1 -- 172.21.15.31:0/703229349 learned_addr learned my addr 172.21.15.31:0/703229349 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff90affd700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 0x7ff90c155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff90b7fe700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c0c2840 0x7ff90c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff90bfff700 1 -- 172.21.15.31:0/703229349 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 msgr2=0x7ff90c155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff90bfff700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 0x7ff90c155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff90bfff700 1 -- 172.21.15.31:0/703229349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c0c2840 msgr2=0x7ff90c0a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.680+0000 7ff90bfff700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c0c2840 0x7ff90c0a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.681+0000 7ff90bfff700 1 -- 172.21.15.31:0/703229349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff904009b30 con 0x7ff90c155640 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.681+0000 7ff90bfff700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c155640 0x7ff90c14ef20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff90400b730 tx=0x7ff904004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:59.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.681+0000 7ff903fff700 1 -- 172.21.15.31:0/703229349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9040092e0 con 0x7ff90c155640 2024-03-30T01:41:59.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.681+0000 7ff903fff700 1 -- 172.21.15.31:0/703229349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9040094d0 con 0x7ff90c155640 2024-03-30T01:41:59.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.681+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c1ea8a0 con 0x7ff90c155640 2024-03-30T01:41:59.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.681+0000 7ff903fff700 1 -- 172.21.15.31:0/703229349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9040044a0 con 0x7ff90c155640 2024-03-30T01:41:59.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.681+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90c1e9de0 con 0x7ff90c155640 2024-03-30T01:41:59.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.682+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90c1ea710 con 0x7ff90c155640 2024-03-30T01:41:59.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.682+0000 7ff903fff700 1 -- 172.21.15.31:0/703229349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff904019070 con 0x7ff90c155640 2024-03-30T01:41:59.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.683+0000 7ff903fff700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8ec06ac90 0x7ff8ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:41:59.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.683+0000 7ff90b7fe700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8ec06ac90 0x7ff8ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:41:59.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.683+0000 7ff903fff700 1 -- 172.21.15.31:0/703229349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6311+0+0 (secure 0 0 0) 0x7ff90408a420 con 0x7ff90c155640 2024-03-30T01:41:59.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.684+0000 7ff90b7fe700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8ec06ac90 0x7ff8ec06d150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc007fd0 tx=0x7ff8fc007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:41:59.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.688+0000 7ff903fff700 1 -- 172.21.15.31:0/703229349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff904058ec0 con 0x7ff90c155640 2024-03-30T01:41:59.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:41:59.904+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff90c1ea8a0 con 0x7ff90c155640 2024-03-30T01:42:00.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.769+0000 7ff903fff700 1 -- 172.21.15.31:0/703229349 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff90c1ea8a0 con 0x7ff90c155640 2024-03-30T01:42:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8ec06ac90 msgr2=0x7ff8ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8ec06ac90 0x7ff8ec06d150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc007fd0 tx=0x7ff8fc007ec0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c155640 msgr2=0x7ff90c14ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c155640 0x7ff90c14ef20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff90400b730 tx=0x7ff904004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 shutdown_connections 2024-03-30T01:42:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff90c155640 0x7ff90c14ef20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8ec06ac90 0x7ff8ec06d150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c0c2840 0x7ff90c0a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 --2- 172.21.15.31:0/703229349 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c0c6a20 0x7ff90c155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.773+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 >> 172.21.15.31:0/703229349 conn(0x7ff90c06a700 msgr2=0x7ff90c0a9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.774+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 shutdown_connections 2024-03-30T01:42:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.774+0000 7ff911ac4700 1 -- 172.21.15.31:0/703229349 wait complete. 2024-03-30T01:42:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-03-30T01:42:00.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-30T01:42:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.939+0000 7fe0a476c700 1 Processor -- start 2024-03-30T01:42:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.939+0000 7fe0a476c700 1 -- start start 2024-03-30T01:42:00.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.940+0000 7fe0a476c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c06d140 0x7fe09c06e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:00.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.940+0000 7fe0a476c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe09c154cc0 0x7fe09c155150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.940+0000 7fe0a476c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe09c0c3260 con 0x7fe09c13ab50 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.940+0000 7fe0a476c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c0a85d0 con 0x7fe09c154cc0 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.940+0000 7fe0a476c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0c22b0 con 0x7fe09c06d140 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.940+0000 7fe0a2d09700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe09c154cc0 0x7fe09c155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.940+0000 7fe0a2508700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe09c13ab50 0x7fe09c06da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39610/0 (socket says 172.21.15.31:39610) 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.940+0000 7fe0a2d09700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe09c154cc0 0x7fe09c155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59730/0 (socket says 172.21.15.31:59730) 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.940+0000 7fe0a2508700 1 -- 172.21.15.31:0/1611549943 learned_addr learned my addr 172.21.15.31:0/1611549943 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a1d07700 1 --2- 172.21.15.31:0/1611549943 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c06d140 0x7fe09c06e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a2d09700 1 -- 172.21.15.31:0/1611549943 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c06d140 msgr2=0x7fe09c06e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a2d09700 1 --2- 172.21.15.31:0/1611549943 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c06d140 0x7fe09c06e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a2d09700 1 -- 172.21.15.31:0/1611549943 >> v1:172.21.15.31:6789/0 conn(0x7fe09c13ab50 legacy=0x7fe09c06da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a2d09700 1 -- 172.21.15.31:0/1611549943 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c06d700 con 0x7fe09c154cc0 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a1506700 1 -- 172.21.15.31:0/1611549943 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302365057 0 0) 0x7fe09c0c3260 con 0x7fe09c13ab50 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a2d09700 1 --2- 172.21.15.31:0/1611549943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c154cc0 0x7fe09c155150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe094009d90 tx=0x7fe09400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a2f74b5baa88ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a1506700 1 -- 172.21.15.31:0/1611549943 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09400e070 con 0x7fe09c154cc0 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a1506700 1 -- 172.21.15.31:0/1611549943 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09400bb20 con 0x7fe09c154cc0 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.941+0000 7fe0a1506700 1 -- 172.21.15.31:0/1611549943 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094012430 con 0x7fe09c154cc0 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.942+0000 7fe0a476c700 1 -- 172.21.15.31:0/1611549943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c154cc0 msgr2=0x7fe09c155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.942+0000 7fe0a476c700 1 --2- 172.21.15.31:0/1611549943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c154cc0 0x7fe09c155150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe094009d90 tx=0x7fe09400b330 comp rx=0 tx=0).stop 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.942+0000 7fe0a476c700 1 -- 172.21.15.31:0/1611549943 shutdown_connections 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.942+0000 7fe0a476c700 1 --2- 172.21.15.31:0/1611549943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c154cc0 0x7fe09c155150 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.942+0000 7fe0a476c700 1 --2- 172.21.15.31:0/1611549943 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c06d140 0x7fe09c06e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.942+0000 7fe0a476c700 1 -- 172.21.15.31:0/1611549943 >> 172.21.15.31:0/1611549943 conn(0x7fe09c0c39f0 msgr2=0x7fe09c0c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.942+0000 7fe0a476c700 1 -- 172.21.15.31:0/1611549943 shutdown_connections 2024-03-30T01:42:00.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.942+0000 7fe0a476c700 1 -- 172.21.15.31:0/1611549943 wait complete. 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.943+0000 7fe0a476c700 1 Processor -- start 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.943+0000 7fe0a476c700 1 -- start start 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a476c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe09c06d140 0x7fe09c0c0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a476c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c13ab50 0x7fe09c0abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a476c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c154cc0 0x7fe09c1e9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a476c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c14f720 con 0x7fe09c13ab50 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a476c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c150470 con 0x7fe09c06d140 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a476c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c14f990 con 0x7fe09c154cc0 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a1d07700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c13ab50 0x7fe09c0abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a1d07700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c13ab50 0x7fe09c0abb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59740/0 (socket says 172.21.15.31:59740) 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a1d07700 1 -- 172.21.15.31:0/1342408036 learned_addr learned my addr 172.21.15.31:0/1342408036 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a2d09700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c154cc0 0x7fe09c1e9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a2508700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe09c06d140 0x7fe09c0c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a1d07700 1 -- 172.21.15.31:0/1342408036 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c154cc0 msgr2=0x7fe09c1e9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a1d07700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c154cc0 0x7fe09c1e9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a1d07700 1 -- 172.21.15.31:0/1342408036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe09c06d140 msgr2=0x7fe09c0c0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a1d07700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe09c06d140 0x7fe09c0c0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a1d07700 1 -- 172.21.15.31:0/1342408036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe094009a70 con 0x7fe09c13ab50 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.944+0000 7fe0a2508700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe09c06d140 0x7fe09c0c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.945+0000 7fe0a1d07700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c13ab50 0x7fe09c0abb20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe09800e9b0 tx=0x7fe09800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.945+0000 7fe08effd700 1 -- 172.21.15.31:0/1342408036 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098009710 con 0x7fe09c13ab50 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.945+0000 7fe08effd700 1 -- 172.21.15.31:0/1342408036 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09800c850 con 0x7fe09c13ab50 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.945+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c14fc70 con 0x7fe09c13ab50 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.945+0000 7fe08effd700 1 -- 172.21.15.31:0/1342408036 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09800cbc0 con 0x7fe09c13ab50 2024-03-30T01:42:00.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.945+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c0c4b50 con 0x7fe09c13ab50 2024-03-30T01:42:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.946+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c150190 con 0x7fe09c13ab50 2024-03-30T01:42:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.946+0000 7fe08effd700 1 -- 172.21.15.31:0/1342408036 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe09800cd20 con 0x7fe09c13ab50 2024-03-30T01:42:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.947+0000 7fe08effd700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.947+0000 7fe08effd700 1 -- 172.21.15.31:0/1342408036 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fe098089520 con 0x7fe09c13ab50 2024-03-30T01:42:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.947+0000 7fe0a2508700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:00.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.947+0000 7fe0a2508700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe090000f60 tx=0x7fe090009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:00.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:00.952+0000 7fe08effd700 1 -- 172.21.15.31:0/1342408036 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe09c150190 con 0x7fe09c13ab50 2024-03-30T01:42:01.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.168+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe09c150190 con 0x7fe09c13ab50 2024-03-30T01:42:01.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.774+0000 7fe08effd700 1 -- 172.21.15.31:0/1342408036 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7fe09c150190 con 0x7fe09c13ab50 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 msgr2=0x7fe08006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe090000f60 tx=0x7fe090009040 comp rx=0 tx=0).stop 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c13ab50 msgr2=0x7fe09c0abb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c13ab50 0x7fe09c0abb20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe09800e9b0 tx=0x7fe09800ed70 comp rx=0 tx=0).stop 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 shutdown_connections 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe08006ac90 0x7fe08006d150 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe090000f60 tx=0x7fe090009040 comp rx=0 tx=0).stop 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09c154cc0 0x7fe09c1e9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe09c06d140 0x7fe09c0c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 --2- 172.21.15.31:0/1342408036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe09c13ab50 0x7fe09c0abb20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.779+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 >> 172.21.15.31:0/1342408036 conn(0x7fe09c0c39f0 msgr2=0x7fe09c137010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:01.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.780+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 shutdown_connections 2024-03-30T01:42:01.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:01.782+0000 7fe0a476c700 1 -- 172.21.15.31:0/1342408036 wait complete. 2024-03-30T01:42:01.789 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data_ec' removed 2024-03-30T01:42:01.807 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T01:42:01.807 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T01:42:01.807 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T01:42:01.807 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T01:42:01.807 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T01:42:01.807 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T01:42:01.808 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T01:42:01.808 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T01:42:01.808 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T01:42:01.808 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T01:42:01.808 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T01:42:01.808 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T01:42:01.808 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T01:42:01.808 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T01:42:01.809 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T01:42:01.809 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T01:42:01.819 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T01:42:01.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T01:42:01.820 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T01:42:01.820 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T01:42:01.821 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T01:42:01.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T01:42:01.822 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T01:42:01.822 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T01:42:01.861 INFO:tasks.ceph.mds.c:Started 2024-03-30T01:42:01.862 DEBUG:teuthology.parallel:result is None 2024-03-30T01:42:01.862 INFO:tasks.ceph.mds.a:Started 2024-03-30T01:42:01.862 DEBUG:teuthology.parallel:result is None 2024-03-30T01:42:01.865 INFO:tasks.ceph.mds.b:Started 2024-03-30T01:42:01.865 DEBUG:teuthology.parallel:result is None 2024-03-30T01:42:01.865 INFO:tasks.ceph.mds.d:Started 2024-03-30T01:42:01.865 DEBUG:teuthology.parallel:result is None 2024-03-30T01:42:01.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T01:42:01.957 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2024-03-30T01:42:01.958 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-03-30T01:42:01.961 INFO:tasks.ceph.mds.b.smithi059.stdout:starting mds.b at 2024-03-30T01:42:01.961 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2024-03-30T01:42:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.016+0000 7f2aea648700 1 Processor -- start 2024-03-30T01:42:02.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.016+0000 7f2aea648700 1 -- start start 2024-03-30T01:42:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2aea648700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ae406b400 0x7f2ae4151a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2aea648700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae40c4f60 0x7f2ae40c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2aea648700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ae4151f40 0x7f2ae40c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:02.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2aea648700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae4154830 con 0x7f2ae4151f40 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2aea648700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae40c5380 con 0x7f2ae406b400 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2aea648700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40c7fd0 con 0x7f2ae40c4f60 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2ae8be5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ae4151f40 0x7f2ae40c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2ae8be5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2ae4151f40 0x7f2ae40c62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59762/0 (socket says 172.21.15.31:59762) 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2ae8be5700 1 -- 172.21.15.31:0/895726116 learned_addr learned my addr 172.21.15.31:0/895726116 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2ae37fe700 1 --2- 172.21.15.31:0/895726116 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae40c4f60 0x7f2ae40c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.017+0000 7f2ae3fff700 1 --2- 172.21.15.31:0/895726116 >> v2:172.21.15.59:3300/0 conn(0x7f2ae406b400 0x7f2ae4151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.018+0000 7f2ae8be5700 1 -- 172.21.15.31:0/895726116 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae40c4f60 msgr2=0x7f2ae40c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.018+0000 7f2ae8be5700 1 --2- 172.21.15.31:0/895726116 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae40c4f60 0x7f2ae40c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.018+0000 7f2ae8be5700 1 -- 172.21.15.31:0/895726116 >> v2:172.21.15.59:3300/0 conn(0x7f2ae406b400 msgr2=0x7f2ae4151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.018+0000 7f2ae8be5700 1 --2- 172.21.15.31:0/895726116 >> v2:172.21.15.59:3300/0 conn(0x7f2ae406b400 0x7f2ae4151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.018+0000 7f2ae8be5700 1 -- 172.21.15.31:0/895726116 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae4154a20 con 0x7f2ae4151f40 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.018+0000 7f2ae8be5700 1 --2- 172.21.15.31:0/895726116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 0x7f2ae40c62b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2ae408b840 tx=0x7f2adc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8987558701e526a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:02.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.018+0000 7f2ae2ffd700 1 -- 172.21.15.31:0/895726116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00e070 con 0x7f2ae4151f40 2024-03-30T01:42:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.018+0000 7f2ae2ffd700 1 -- 172.21.15.31:0/895726116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc00b910 con 0x7f2ae4151f40 2024-03-30T01:42:02.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.018+0000 7f2ae2ffd700 1 -- 172.21.15.31:0/895726116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00bcc0 con 0x7f2ae4151f40 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.019+0000 7f2aea648700 1 -- 172.21.15.31:0/895726116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 msgr2=0x7f2ae40c62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.019+0000 7f2aea648700 1 --2- 172.21.15.31:0/895726116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 0x7f2ae40c62b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2ae408b840 tx=0x7f2adc00b330 comp rx=0 tx=0).stop 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.019+0000 7f2aea648700 1 -- 172.21.15.31:0/895726116 shutdown_connections 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.019+0000 7f2aea648700 1 --2- 172.21.15.31:0/895726116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 0x7f2ae40c62b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.019+0000 7f2aea648700 1 --2- 172.21.15.31:0/895726116 >> v2:172.21.15.59:3300/0 conn(0x7f2ae406b400 0x7f2ae4151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.019+0000 7f2aea648700 1 --2- 172.21.15.31:0/895726116 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae40c4f60 0x7f2ae40c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.019+0000 7f2aea648700 1 -- 172.21.15.31:0/895726116 >> 172.21.15.31:0/895726116 conn(0x7f2ae40ca0d0 msgr2=0x7f2ae40ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.019+0000 7f2aea648700 1 -- 172.21.15.31:0/895726116 shutdown_connections 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.019+0000 7f2aea648700 1 -- 172.21.15.31:0/895726116 wait complete. 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.020+0000 7f2aea648700 1 Processor -- start 2024-03-30T01:42:02.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.020+0000 7f2aea648700 1 -- start start 2024-03-30T01:42:02.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2aea648700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae406b400 0x7f2ae41580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:02.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2aea648700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ae40c4f60 0x7f2ae414b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:02.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2aea648700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 0x7f2ae414b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:02.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2aea648700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae40bff30 con 0x7f2ae4151f40 2024-03-30T01:42:02.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2aea648700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae414bce0 con 0x7f2ae40c4f60 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2aea648700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae4155be0 con 0x7f2ae406b400 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae8be5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 0x7f2ae414b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ae40c4f60 0x7f2ae414b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae3fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae406b400 0x7f2ae41580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae8be5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 0x7f2ae414b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59778/0 (socket says 172.21.15.31:59778) 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae8be5700 1 -- 172.21.15.31:0/2029487528 learned_addr learned my addr 172.21.15.31:0/2029487528 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae8be5700 1 -- 172.21.15.31:0/2029487528 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae406b400 msgr2=0x7f2ae41580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae8be5700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae406b400 0x7f2ae41580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae8be5700 1 -- 172.21.15.31:0/2029487528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ae40c4f60 msgr2=0x7f2ae414b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae8be5700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ae40c4f60 0x7f2ae414b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.021+0000 7f2ae8be5700 1 -- 172.21.15.31:0/2029487528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc009b30 con 0x7f2ae4151f40 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.022+0000 7f2ae8be5700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 0x7f2ae414b7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00b680 tx=0x7f2adc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.022+0000 7f2ae0ff9700 1 -- 172.21.15.31:0/2029487528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0099b0 con 0x7f2ae4151f40 2024-03-30T01:42:02.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.022+0000 7f2ae0ff9700 1 -- 172.21.15.31:0/2029487528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc00ce80 con 0x7f2ae4151f40 2024-03-30T01:42:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.022+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae41ea5d0 con 0x7f2ae4151f40 2024-03-30T01:42:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.022+0000 7f2ae0ff9700 1 -- 172.21.15.31:0/2029487528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc004490 con 0x7f2ae4151f40 2024-03-30T01:42:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.022+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae4156160 con 0x7f2ae4151f40 2024-03-30T01:42:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.023+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae404ea80 con 0x7f2ae4151f40 2024-03-30T01:42:02.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.023+0000 7f2ae0ff9700 1 -- 172.21.15.31:0/2029487528 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2adc0045f0 con 0x7f2ae4151f40 2024-03-30T01:42:02.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.024+0000 7f2ae0ff9700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ad006ac90 0x7f2ad006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:02.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.024+0000 7f2ae0ff9700 1 -- 172.21.15.31:0/2029487528 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2adc08a360 con 0x7f2ae4151f40 2024-03-30T01:42:02.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.024+0000 7f2ae3fff700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ad006ac90 0x7f2ad006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:02.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.024+0000 7f2ae3fff700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ad006ac90 0x7f2ad006d150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009b60 tx=0x7f2ad4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:02.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.028+0000 7f2ae0ff9700 1 -- 172.21.15.31:0/2029487528 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2adc055840 con 0x7f2ae4151f40 2024-03-30T01:42:02.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:02.261+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2ae40c9b30 con 0x7f2ae4151f40 2024-03-30T01:42:03.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.002+0000 7f2ae0ff9700 1 -- 172.21.15.31:0/2029487528 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2adc058e60 con 0x7f2ae4151f40 2024-03-30T01:42:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ad006ac90 msgr2=0x7f2ad006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ad006ac90 0x7f2ad006d150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009b60 tx=0x7f2ad4009210 comp rx=0 tx=0).stop 2024-03-30T01:42:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 msgr2=0x7f2ae414b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 0x7f2ae414b7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00b680 tx=0x7f2adc004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 shutdown_connections 2024-03-30T01:42:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae4151f40 0x7f2ae414b7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2ad006ac90 0x7f2ad006d150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ae406b400 0x7f2ae41580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 --2- 172.21.15.31:0/2029487528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ae40c4f60 0x7f2ae414b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 >> 172.21.15.31:0/2029487528 conn(0x7f2ae40ca0d0 msgr2=0x7f2ae40ca670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:03.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 shutdown_connections 2024-03-30T01:42:03.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.005+0000 7f2aea648700 1 -- 172.21.15.31:0/2029487528 wait complete. 2024-03-30T01:42:03.013 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2024-03-30T01:42:03.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T01:42:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.161+0000 7ff026685700 1 Processor -- start 2024-03-30T01:42:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.161+0000 7ff026685700 1 -- start start 2024-03-30T01:42:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff026685700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014fd30 0x7ff020156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff026685700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff020150110 0x7ff0200c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff026685700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0200c4d60 con 0x7ff0200c88c0 2024-03-30T01:42:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff026685700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0200c5040 con 0x7ff020150110 2024-03-30T01:42:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff026685700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200c5320 con 0x7ff02014fd30 2024-03-30T01:42:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff024c22700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff0200c88c0 0x7ff0200ac630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39624/0 (socket says 172.21.15.31:39624) 2024-03-30T01:42:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff024c22700 1 -- 172.21.15.31:0/1406104256 learned_addr learned my addr 172.21.15.31:0/1406104256 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01f7fe700 1 --2- 172.21.15.31:0/1406104256 >> v2:172.21.15.59:3300/0 conn(0x7ff020150110 0x7ff0200c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01ffff700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014fd30 0x7ff020156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971970129 0 0) 0x7ff0200c4d60 con 0x7ff0200c88c0 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0080035b0 con 0x7ff0200c88c0 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 574996112 0 0) 0x7ff0080035b0 con 0x7ff0200c88c0 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014fd30 msgr2=0x7ff020156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014fd30 0x7ff020156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 >> v2:172.21.15.59:3300/0 conn(0x7ff020150110 msgr2=0x7ff0200c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 --2- 172.21.15.31:0/1406104256 >> v2:172.21.15.59:3300/0 conn(0x7ff020150110 0x7ff0200c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0200c5600 con 0x7ff0200c88c0 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7ff0180019f0 con 0x7ff0200c88c0 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01f7fe700 1 --2- 172.21.15.31:0/1406104256 >> v2:172.21.15.59:3300/0 conn(0x7ff020150110 0x7ff0200c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 >> v1:172.21.15.31:6789/0 conn(0x7ff0200c88c0 legacy=0x7ff0200ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014fd30 0x7ff008003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.162+0000 7ff01effd700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff008003f80 0x7ff008004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff01effd700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff008004d90 0x7ff008005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200c5040 con 0x7ff008004d90 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200c5320 con 0x7ff008003f80 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00800ba60 con 0x7ff02014fd30 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff024c22700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff008004d90 0x7ff008005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff01f7fe700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff008003f80 0x7ff008004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff01ffff700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014fd30 0x7ff008003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff024c22700 1 -- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014fd30 msgr2=0x7ff008003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff024c22700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014fd30 0x7ff008003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff024c22700 1 -- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff008003f80 msgr2=0x7ff008004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff024c22700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff008003f80 0x7ff008004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff024c22700 1 -- 172.21.15.31:0/1406104256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080035b0 con 0x7ff008004d90 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff024c22700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff008004d90 0x7ff008005210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff01800a890 tx=0x7ff008005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fca757bcaa6d94b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:03.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018012600 con 0x7ff008004d90 2024-03-30T01:42:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018012780 con 0x7ff008004d90 2024-03-30T01:42:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.163+0000 7ff01effd700 1 -- 172.21.15.31:0/1406104256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018012b30 con 0x7ff008004d90 2024-03-30T01:42:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.164+0000 7ff026685700 1 -- 172.21.15.31:0/1406104256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff008004d90 msgr2=0x7ff008005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.164+0000 7ff026685700 1 --2- 172.21.15.31:0/1406104256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff008004d90 0x7ff008005210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff01800a890 tx=0x7ff008005880 comp rx=0 tx=0).stop 2024-03-30T01:42:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.164+0000 7ff024c22700 1 -- 172.21.15.31:0/1406104256 reap_dead start 2024-03-30T01:42:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.164+0000 7ff026685700 1 -- 172.21.15.31:0/1406104256 shutdown_connections 2024-03-30T01:42:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.164+0000 7ff026685700 1 -- 172.21.15.31:0/1406104256 >> 172.21.15.31:0/1406104256 conn(0x7ff02006ab20 msgr2=0x7ff02006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.164+0000 7ff026685700 1 -- 172.21.15.31:0/1406104256 shutdown_connections 2024-03-30T01:42:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.164+0000 7ff026685700 1 -- 172.21.15.31:0/1406104256 wait complete. 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.165+0000 7ff026685700 1 Processor -- start 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.165+0000 7ff026685700 1 -- start start 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.166+0000 7ff026685700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff008004d90 0x7ff0201e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.166+0000 7ff026685700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014dbc0 0x7ff0200bf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.166+0000 7ff026685700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff02014cfc0 0x7ff02014d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.166+0000 7ff026685700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0200bd8c0 con 0x7ff02014cfc0 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.166+0000 7ff026685700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02014d980 con 0x7ff008004d90 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.166+0000 7ff026685700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02014e110 con 0x7ff02014dbc0 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.166+0000 7ff024c22700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff02014cfc0 0x7ff02014d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.166+0000 7ff024c22700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff02014cfc0 0x7ff02014d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59788/0 (socket says 172.21.15.31:59788) 2024-03-30T01:42:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.166+0000 7ff024c22700 1 -- 172.21.15.31:0/1752804016 learned_addr learned my addr 172.21.15.31:0/1752804016 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff01ffff700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff008004d90 0x7ff0201e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff01f7fe700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014dbc0 0x7ff0200bf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff024c22700 1 -- 172.21.15.31:0/1752804016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014dbc0 msgr2=0x7ff0200bf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff024c22700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014dbc0 0x7ff0200bf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff024c22700 1 -- 172.21.15.31:0/1752804016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff008004d90 msgr2=0x7ff0201e7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff024c22700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff008004d90 0x7ff0201e7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff024c22700 1 -- 172.21.15.31:0/1752804016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0200bcf90 con 0x7ff02014cfc0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff024c22700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff02014cfc0 0x7ff02014d440 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff018001cb0 tx=0x7ff018001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff01cff9700 1 -- 172.21.15.31:0/1752804016 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0180128f0 con 0x7ff02014cfc0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff01cff9700 1 -- 172.21.15.31:0/1752804016 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01801e610 con 0x7ff02014cfc0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0200bd270 con 0x7ff02014cfc0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff01cff9700 1 -- 172.21.15.31:0/1752804016 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01801e9c0 con 0x7ff02014cfc0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.167+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0200bf600 con 0x7ff02014cfc0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.168+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff020073760 con 0x7ff02014cfc0 2024-03-30T01:42:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.169+0000 7ff01cff9700 1 -- 172.21.15.31:0/1752804016 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff018019070 con 0x7ff02014cfc0 2024-03-30T01:42:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.169+0000 7ff01cff9700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00c06ac90 0x7ff00c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.169+0000 7ff01cff9700 1 -- 172.21.15.31:0/1752804016 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff01808b020 con 0x7ff02014cfc0 2024-03-30T01:42:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.169+0000 7ff01ffff700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00c06ac90 0x7ff00c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.170+0000 7ff01ffff700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00c06ac90 0x7ff00c06d150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff010009ba0 tx=0x7ff010008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.174+0000 7ff01cff9700 1 -- 172.21.15.31:0/1752804016 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff0180564a0 con 0x7ff02014cfc0 2024-03-30T01:42:03.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.477+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff02004ea80 con 0x7ff02014cfc0 2024-03-30T01:42:03.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.478+0000 7ff01cff9700 1 -- 172.21.15.31:0/1752804016 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6065 (secure 0 0 0) 0x7ff018059ac0 con 0x7ff02014cfc0 2024-03-30T01:42:03.484 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:03.484 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-03-30T01:42:03.484 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2024-03-30T01:42:03.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.484 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbQdmKS4dOBAAoFJ9d9VJCzRRA6dvzqlvdA==", 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdmttL5ChAABt7Prk29JiHASSSErpnSgg==", 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:42:03.485 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdmy2PsBBAAmQgxB6hsqmzkMOlsiyA2DQ==", 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdm5o/FGRAAuUz1WBZZuhrB8lrBXkgrzg==", 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB0bQdmv2kRMBAA+Z/39yEKUrYr3hMcRw6UtQ==", 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:42:03.487 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB3bQdmgRSYLxAA9DUas+VdQgEmd2nF77YNmw==", 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:42:03.488 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB6bQdmjqwOMBAASDQNMVbnD3xzqGdxNYtxTg==", 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.489 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB9bQdmM61yLxAAAeBjdPHisSW+6+y6yUMYfw==", 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCDbQdmAeb7NhAAo/xdDmOPniSMNAWGqXprzg==", 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:42:03.490 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCGbQdmsibcNhAAYJXjCRMNskc5suFV3ylOqw==", 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCJbQdmCi5wNhAAoFVc2HDSZQULOOhYt7lBDA==", 2024-03-30T01:42:03.491 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCMbQdm2DocNxAAUPu6LwRv6gIMMr801Jsjhw==", 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdmickSHxAAowVfYqM7JYAqRJqYG0fQBw==", 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdmijhSIxAAbia5PslTYSh6u9Xvg/PF2g==", 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:42:03.493 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbQdmrbpUFhAAJIkJomJO7Vhgwhow68CKsg==", 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mds", 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCSbQdmpw/1OBAArTzg7GhJe3Ei6+0oap3dzQ==", 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mds" 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mgr", 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCSbQdmlBn1OBAAlzloYcpJ/zT5zjxFdtZfyQ==", 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-osd", 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCSbQdmvCL1OBAAlj0Tp7zTY4A4hP1G7nrysw==", 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-osd" 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd", 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCSbQdmpSv1OBAA0YvUK7+0hlH+rFWOgz7O6A==", 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-30T01:42:03.496 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCSbQdm9zb1OBAAohxooRms0kCOjlp+8cFc9w==", 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rgw", 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCSbQdmMED1OBAAlVlj8MuKuoXOKGF+gAM0LQ==", 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbQdmVEftMxAA1lDaA5SAWgLRPojMSwq70g==", 2024-03-30T01:42:03.497 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbQdmSyDMLhAAIk3a6lqgYJDDIrak1TE3Og==", 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:03.498 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-03-30T01:42:03.499 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-03-30T01:42:03.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00c06ac90 msgr2=0x7ff00c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00c06ac90 0x7ff00c06d150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff010009ba0 tx=0x7ff010008040 comp rx=0 tx=0).stop 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff02014cfc0 msgr2=0x7ff02014d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff02014cfc0 0x7ff02014d440 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff018001cb0 tx=0x7ff018001d80 comp rx=0 tx=0).stop 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 shutdown_connections 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff02014cfc0 0x7ff02014d440 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00c06ac90 0x7ff00c06d150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff008004d90 0x7ff0201e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 --2- 172.21.15.31:0/1752804016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff02014dbc0 0x7ff0200bf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 >> 172.21.15.31:0/1752804016 conn(0x7ff02006ab20 msgr2=0x7ff0200acd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 shutdown_connections 2024-03-30T01:42:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.480+0000 7ff026685700 1 -- 172.21.15.31:0/1752804016 wait complete. 2024-03-30T01:42:03.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-30T01:42:03.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.632+0000 7f6f6811f700 1 Processor -- start 2024-03-30T01:42:03.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.632+0000 7f6f6811f700 1 -- start start 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.633+0000 7f6f6811f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f60152650 0x7f6f60152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.633+0000 7f6f6811f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f600c2f60 con 0x7f6f600c1300 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.633+0000 7f6f6811f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f600c2a40 con 0x7f6f6014fc30 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.633+0000 7f6f6811f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6006eca0 con 0x7f6f60152650 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.633+0000 7f6f666bc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f60152650 0x7f6f60152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.633+0000 7f6f656ba700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6f600c1300 0x7f6f6006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39640/0 (socket says 172.21.15.31:39640) 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.633+0000 7f6f666bc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f60152650 0x7f6f60152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59796/0 (socket says 172.21.15.31:59796) 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.633+0000 7f6f656ba700 1 -- 172.21.15.31:0/1374089894 learned_addr learned my addr 172.21.15.31:0/1374089894 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.634+0000 7f6f666bc700 1 -- 172.21.15.31:0/1374089894 >> v1:172.21.15.59:6789/0 conn(0x7f6f6014fc30 legacy=0x7f6f6006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.634+0000 7f6f666bc700 1 -- 172.21.15.31:0/1374089894 >> v1:172.21.15.31:6789/0 conn(0x7f6f600c1300 legacy=0x7f6f6006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.634+0000 7f6f666bc700 1 -- 172.21.15.31:0/1374089894 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600c1c90 con 0x7f6f60152650 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.634+0000 7f6f64eb9700 1 -- 172.21.15.31:0/1374089894 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624929305 0 0) 0x7f6f600c2f60 con 0x7f6f600c1300 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.634+0000 7f6f666bc700 1 --2- 172.21.15.31:0/1374089894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f60152650 0x7f6f60152ac0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009d90 tx=0x7f6f5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ed9acd4ce7fb2d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.634+0000 7f6f64eb9700 1 -- 172.21.15.31:0/1374089894 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800e070 con 0x7f6f60152650 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.634+0000 7f6f64eb9700 1 -- 172.21.15.31:0/1374089894 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5800bbd0 con 0x7f6f60152650 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.634+0000 7f6f64eb9700 1 -- 172.21.15.31:0/1374089894 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58012430 con 0x7f6f60152650 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.635+0000 7f6f6811f700 1 -- 172.21.15.31:0/1374089894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f60152650 msgr2=0x7f6f60152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.635+0000 7f6f6811f700 1 --2- 172.21.15.31:0/1374089894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f60152650 0x7f6f60152ac0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009d90 tx=0x7f6f5800b330 comp rx=0 tx=0).stop 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.635+0000 7f6f6811f700 1 -- 172.21.15.31:0/1374089894 shutdown_connections 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.635+0000 7f6f6811f700 1 --2- 172.21.15.31:0/1374089894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f60152650 0x7f6f60152ac0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.635+0000 7f6f6811f700 1 -- 172.21.15.31:0/1374089894 >> 172.21.15.31:0/1374089894 conn(0x7f6f600c3a60 msgr2=0x7f6f600c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:03.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.635+0000 7f6f6811f700 1 -- 172.21.15.31:0/1374089894 shutdown_connections 2024-03-30T01:42:03.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.635+0000 7f6f6811f700 1 -- 172.21.15.31:0/1374089894 wait complete. 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.636+0000 7f6f6811f700 1 Processor -- start 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.636+0000 7f6f6811f700 1 -- start start 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f6811f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f600c1300 0x7f6f600c0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f6811f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6014fc30 0x7f6f600a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f6811f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f60152650 0x7f6f6013ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f6811f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6014d5f0 con 0x7f6f600c1300 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f6811f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600a8ce0 con 0x7f6f6014fc30 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f6811f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600a8fc0 con 0x7f6f60152650 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f65ebb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f600c1300 0x7f6f600c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f65ebb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f600c1300 0x7f6f600c0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59806/0 (socket says 172.21.15.31:59806) 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f65ebb700 1 -- 172.21.15.31:0/3895520500 learned_addr learned my addr 172.21.15.31:0/3895520500 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f656ba700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6014fc30 0x7f6f600a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f666bc700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f60152650 0x7f6f6013ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f65ebb700 1 -- 172.21.15.31:0/3895520500 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f60152650 msgr2=0x7f6f6013ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f65ebb700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f60152650 0x7f6f6013ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f65ebb700 1 -- 172.21.15.31:0/3895520500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6014fc30 msgr2=0x7f6f600a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f65ebb700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6014fc30 0x7f6f600a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.637+0000 7f6f65ebb700 1 -- 172.21.15.31:0/3895520500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f58009a70 con 0x7f6f600c1300 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.638+0000 7f6f65ebb700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f600c1300 0x7f6f600c0600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c4e0 tx=0x7f6f5000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.638+0000 7f6f567fc700 1 -- 172.21.15.31:0/3895520500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f500052d0 con 0x7f6f600c1300 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.638+0000 7f6f567fc700 1 -- 172.21.15.31:0/3895520500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f50005430 con 0x7f6f600c1300 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.638+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f600a92a0 con 0x7f6f600c1300 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.638+0000 7f6f567fc700 1 -- 172.21.15.31:0/3895520500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f500038c0 con 0x7f6f600c1300 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.638+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f600c0f50 con 0x7f6f600c1300 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.639+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f600a97c0 con 0x7f6f600c1300 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.640+0000 7f6f567fc700 1 -- 172.21.15.31:0/3895520500 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6f50003a20 con 0x7f6f600c1300 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.640+0000 7f6f567fc700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4406ac90 0x7f6f4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.640+0000 7f6f567fc700 1 -- 172.21.15.31:0/3895520500 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6f50088360 con 0x7f6f600c1300 2024-03-30T01:42:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.640+0000 7f6f656ba700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4406ac90 0x7f6f4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:03.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.641+0000 7f6f656ba700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4406ac90 0x7f6f4406d150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c007b10 tx=0x7f6f5c007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:03.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.645+0000 7f6f567fc700 1 -- 172.21.15.31:0/3895520500 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6f600a97c0 con 0x7f6f600c1300 2024-03-30T01:42:03.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.948+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f6f600c2a40 con 0x7f6f600c1300 2024-03-30T01:42:03.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.952+0000 7f6f567fc700 1 -- 172.21.15.31:0/3895520500 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6f600c2a40 con 0x7f6f600c1300 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4406ac90 msgr2=0x7f6f4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4406ac90 0x7f6f4406d150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c007b10 tx=0x7f6f5c007a60 comp rx=0 tx=0).stop 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f600c1300 msgr2=0x7f6f600c0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f600c1300 0x7f6f600c0600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c4e0 tx=0x7f6f5000c7f0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 shutdown_connections 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f600c1300 0x7f6f600c0600 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4406ac90 0x7f6f4406d150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f60152650 0x7f6f6013ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 --2- 172.21.15.31:0/3895520500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f6014fc30 0x7f6f600a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:03.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 >> 172.21.15.31:0/3895520500 conn(0x7f6f600c3a60 msgr2=0x7f6f600c3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 shutdown_connections 2024-03-30T01:42:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:03.955+0000 7f6f6811f700 1 -- 172.21.15.31:0/3895520500 wait complete. 2024-03-30T01:42:03.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-30T01:42:04.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.105+0000 7f862bac1700 1 Processor -- start 2024-03-30T01:42:04.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.105+0000 7f862bac1700 1 -- start start 2024-03-30T01:42:04.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.106+0000 7f862bac1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f86240722b0 0x7f862406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.106+0000 7f862bac1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862406db20 0x7f86241479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.106+0000 7f862bac1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86240a88c0 con 0x7f8624154d20 2024-03-30T01:42:04.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.106+0000 7f862bac1700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86240c3110 con 0x7f86240722b0 2024-03-30T01:42:04.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.106+0000 7f862bac1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8624155810 con 0x7f862406db20 2024-03-30T01:42:04.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.106+0000 7f862905c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f86240722b0 0x7f862406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862905c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f86240722b0 0x7f862406d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59818/0 (socket says 172.21.15.31:59818) 2024-03-30T01:42:04.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862905c700 1 -- 172.21.15.31:0/3845326738 learned_addr learned my addr 172.21.15.31:0/3845326738 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:04.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862a05e700 1 --2- 172.21.15.31:0/3845326738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862406db20 0x7f86241479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862905c700 1 -- 172.21.15.31:0/3845326738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862406db20 msgr2=0x7f86241479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862905c700 1 --2- 172.21.15.31:0/3845326738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862406db20 0x7f86241479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862905c700 1 -- 172.21.15.31:0/3845326738 >> v1:172.21.15.59:6789/0 conn(0x7f8624154d20 legacy=0x7f8624155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862905c700 1 -- 172.21.15.31:0/3845326738 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86240a85e0 con 0x7f86240722b0 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862905c700 1 --2- 172.21.15.31:0/3845326738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86240722b0 0x7f862406d5e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f86140098f0 tx=0x7f8614009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21df79f62602c927 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862885b700 1 -- 172.21.15.31:0/3845326738 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861400b8a0 con 0x7f86240722b0 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862885b700 1 -- 172.21.15.31:0/3845326738 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861400ba00 con 0x7f86240722b0 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.107+0000 7f862885b700 1 -- 172.21.15.31:0/3845326738 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861400bd90 con 0x7f86240722b0 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.108+0000 7f862bac1700 1 -- 172.21.15.31:0/3845326738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86240722b0 msgr2=0x7f862406d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.108+0000 7f862bac1700 1 --2- 172.21.15.31:0/3845326738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86240722b0 0x7f862406d5e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f86140098f0 tx=0x7f8614009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.108+0000 7f862bac1700 1 -- 172.21.15.31:0/3845326738 shutdown_connections 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.108+0000 7f862bac1700 1 --2- 172.21.15.31:0/3845326738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862406db20 0x7f86241479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.108+0000 7f862bac1700 1 --2- 172.21.15.31:0/3845326738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86240722b0 0x7f862406d5e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.108+0000 7f862bac1700 1 -- 172.21.15.31:0/3845326738 >> 172.21.15.31:0/3845326738 conn(0x7f86240c3a60 msgr2=0x7f86240c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.108+0000 7f862bac1700 1 -- 172.21.15.31:0/3845326738 shutdown_connections 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.108+0000 7f862bac1700 1 -- 172.21.15.31:0/3845326738 wait complete. 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.109+0000 7f862bac1700 1 Processor -- start 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.109+0000 7f862bac1700 1 -- start start 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862bac1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f862406db20 0x7f86241ea390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862bac1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86240722b0 0x7f862406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862bac1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8624154d20 0x7f86241e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862bac1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86241e7890 con 0x7f8624154d20 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862bac1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86240ac320 con 0x7f862406db20 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862bac1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8624143fd0 con 0x7f86240722b0 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862a05e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8624154d20 0x7f86241e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862a05e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8624154d20 0x7f86241e7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59824/0 (socket says 172.21.15.31:59824) 2024-03-30T01:42:04.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862985d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f862406db20 0x7f86241ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862a05e700 1 -- 172.21.15.31:0/1073758101 learned_addr learned my addr 172.21.15.31:0/1073758101 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.110+0000 7f862905c700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86240722b0 0x7f862406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f862a05e700 1 -- 172.21.15.31:0/1073758101 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86240722b0 msgr2=0x7f862406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f862a05e700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86240722b0 0x7f862406e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f862a05e700 1 -- 172.21.15.31:0/1073758101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f862406db20 msgr2=0x7f86241ea390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f862a05e700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f862406db20 0x7f86241ea390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f862a05e700 1 -- 172.21.15.31:0/1073758101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8614009550 con 0x7f8624154d20 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f862a05e700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8624154d20 0x7f86241e7fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f861c007e70 tx=0x7f861c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f861a7fc700 1 -- 172.21.15.31:0/1073758101 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c00ed30 con 0x7f8624154d20 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f861a7fc700 1 -- 172.21.15.31:0/1073758101 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c00eeb0 con 0x7f8624154d20 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86241e7b10 con 0x7f8624154d20 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f861a7fc700 1 -- 172.21.15.31:0/1073758101 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c014640 con 0x7f8624154d20 2024-03-30T01:42:04.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.111+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86241eb810 con 0x7f8624154d20 2024-03-30T01:42:04.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.112+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86241e7ca0 con 0x7f8624154d20 2024-03-30T01:42:04.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.112+0000 7f861a7fc700 1 -- 172.21.15.31:0/1073758101 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f861c0147a0 con 0x7f8624154d20 2024-03-30T01:42:04.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.113+0000 7f861a7fc700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f860806acc0 0x7f860806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.113+0000 7f862985d700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f860806acc0 0x7f860806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.113+0000 7f861a7fc700 1 -- 172.21.15.31:0/1073758101 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f861c0897d0 con 0x7f8624154d20 2024-03-30T01:42:04.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.113+0000 7f862985d700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f860806acc0 0x7f860806d180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8620007fd0 tx=0x7f8620009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:04.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.118+0000 7f861a7fc700 1 -- 172.21.15.31:0/1073758101 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f861c054bb0 con 0x7f8624154d20 2024-03-30T01:42:04.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.420+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f862404ea80 con 0x7f8624154d20 2024-03-30T01:42:04.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.424+0000 7f861a7fc700 1 -- 172.21.15.31:0/1073758101 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f861c0581d0 con 0x7f8624154d20 2024-03-30T01:42:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f860806acc0 msgr2=0x7f860806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f860806acc0 0x7f860806d180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8620007fd0 tx=0x7f8620009040 comp rx=0 tx=0).stop 2024-03-30T01:42:04.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8624154d20 msgr2=0x7f86241e7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8624154d20 0x7f86241e7fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f861c007e70 tx=0x7f861c00e3c0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 shutdown_connections 2024-03-30T01:42:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8624154d20 0x7f86241e7fa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f860806acc0 0x7f860806d180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f862406db20 0x7f86241ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 --2- 172.21.15.31:0/1073758101 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86240722b0 0x7f862406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 >> 172.21.15.31:0/1073758101 conn(0x7f86240c3a60 msgr2=0x7f8624155100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.426+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 shutdown_connections 2024-03-30T01:42:04.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.427+0000 7f862bac1700 1 -- 172.21.15.31:0/1073758101 wait complete. 2024-03-30T01:42:04.447 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-30T01:42:04.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.576+0000 7f8ac3ca6700 1 Processor -- start 2024-03-30T01:42:04.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.577+0000 7f8ac3ca6700 1 -- start start 2024-03-30T01:42:04.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.577+0000 7f8ac3ca6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8abc14fd30 0x7f8abc156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac3ca6700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8abc0c47c0 0x7f8abc0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac3ca6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0c4d60 con 0x7f8abc150110 2024-03-30T01:42:04.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac3ca6700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0c5040 con 0x7f8abc0c47c0 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac3ca6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0c5320 con 0x7f8abc14fd30 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac2243700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8abc0c47c0 0x7f8abc0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac2243700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8abc0c47c0 0x7f8abc0ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59828/0 (socket says 172.21.15.31:59828) 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac2243700 1 -- 172.21.15.31:0/4274111879 learned_addr learned my addr 172.21.15.31:0/4274111879 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac1a42700 1 --2- 172.21.15.31:0/4274111879 >> v2:172.21.15.59:3300/0 conn(0x7f8abc14fd30 0x7f8abc156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac2243700 1 -- 172.21.15.31:0/4274111879 >> v2:172.21.15.59:3300/0 conn(0x7f8abc14fd30 msgr2=0x7f8abc156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac2243700 1 --2- 172.21.15.31:0/4274111879 >> v2:172.21.15.59:3300/0 conn(0x7f8abc14fd30 0x7f8abc156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac2243700 1 -- 172.21.15.31:0/4274111879 >> v1:172.21.15.59:6789/0 conn(0x7f8abc150110 legacy=0x7f8abc0c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac2243700 1 -- 172.21.15.31:0/4274111879 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0c5600 con 0x7f8abc0c47c0 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac2243700 1 --2- 172.21.15.31:0/4274111879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc0c47c0 0x7f8abc0ac630 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0c8350 tx=0x7f8ab400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fed64c8f567125 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.578+0000 7f8ac0a40700 1 -- 172.21.15.31:0/4274111879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400e070 con 0x7f8abc0c47c0 2024-03-30T01:42:04.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac0a40700 1 -- 172.21.15.31:0/4274111879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab400b910 con 0x7f8abc0c47c0 2024-03-30T01:42:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac0a40700 1 -- 172.21.15.31:0/4274111879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400bcc0 con 0x7f8abc0c47c0 2024-03-30T01:42:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4274111879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc0c47c0 msgr2=0x7f8abc0ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac3ca6700 1 --2- 172.21.15.31:0/4274111879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc0c47c0 0x7f8abc0ac630 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0c8350 tx=0x7f8ab400b330 comp rx=0 tx=0).stop 2024-03-30T01:42:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4274111879 shutdown_connections 2024-03-30T01:42:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac3ca6700 1 --2- 172.21.15.31:0/4274111879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc0c47c0 0x7f8abc0ac630 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac3ca6700 1 --2- 172.21.15.31:0/4274111879 >> v2:172.21.15.59:3300/0 conn(0x7f8abc14fd30 0x7f8abc156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4274111879 >> 172.21.15.31:0/4274111879 conn(0x7f8abc06ab20 msgr2=0x7f8abc06af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4274111879 shutdown_connections 2024-03-30T01:42:04.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.579+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4274111879 wait complete. 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.580+0000 7f8ac3ca6700 1 Processor -- start 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.580+0000 7f8ac3ca6700 1 -- start start 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac3ca6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8abc0c47c0 0x7f8abc153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac3ca6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc14fd30 0x7f8abc0a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac3ca6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8abc150110 0x7f8abc0a8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac3ca6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc1dc240 con 0x7f8abc14fd30 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac3ca6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc150d30 con 0x7f8abc0c47c0 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac3ca6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc1522b0 con 0x7f8abc150110 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac1241700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc14fd30 0x7f8abc0a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac1241700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc14fd30 0x7f8abc0a82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59842/0 (socket says 172.21.15.31:59842) 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac1241700 1 -- 172.21.15.31:0/4087150552 learned_addr learned my addr 172.21.15.31:0/4087150552 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac1a42700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8abc0c47c0 0x7f8abc153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.581+0000 7f8ac2243700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8abc150110 0x7f8abc0a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ac1241700 1 -- 172.21.15.31:0/4087150552 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8abc150110 msgr2=0x7f8abc0a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ac1241700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8abc150110 0x7f8abc0a8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ac1241700 1 -- 172.21.15.31:0/4087150552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8abc0c47c0 msgr2=0x7f8abc153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ac1241700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8abc0c47c0 0x7f8abc153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ac1241700 1 -- 172.21.15.31:0/4087150552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009b30 con 0x7f8abc14fd30 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ac1241700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc14fd30 0x7f8abc0a82d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c450 tx=0x7f8aac00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ab27fc700 1 -- 172.21.15.31:0/4087150552 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0052d0 con 0x7f8abc14fd30 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ab27fc700 1 -- 172.21.15.31:0/4087150552 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac005430 con 0x7f8abc14fd30 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc152590 con 0x7f8abc14fd30 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ab27fc700 1 -- 172.21.15.31:0/4087150552 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0038c0 con 0x7f8abc14fd30 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.582+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc152d90 con 0x7f8abc14fd30 2024-03-30T01:42:04.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.583+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc073760 con 0x7f8abc14fd30 2024-03-30T01:42:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.584+0000 7f8ab27fc700 1 -- 172.21.15.31:0/4087150552 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8aac003a20 con 0x7f8abc14fd30 2024-03-30T01:42:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.584+0000 7f8ab27fc700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8aa006ac90 0x7f8aa006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.584+0000 7f8ab27fc700 1 -- 172.21.15.31:0/4087150552 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8aac0883a0 con 0x7f8abc14fd30 2024-03-30T01:42:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.584+0000 7f8ac1a42700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8aa006ac90 0x7f8aa006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:04.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.585+0000 7f8ac1a42700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8aa006ac90 0x7f8aa006d150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8abc14b830 tx=0x7f8ab8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:04.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.589+0000 7f8ab27fc700 1 -- 172.21.15.31:0/4087150552 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8aac053820 con 0x7f8abc14fd30 2024-03-30T01:42:04.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.897+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f8abc0a7960 con 0x7f8abc14fd30 2024-03-30T01:42:04.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.901+0000 7f8ab27fc700 1 -- 172.21.15.31:0/4087150552 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8aac00ce60 con 0x7f8abc14fd30 2024-03-30T01:42:04.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8aa006ac90 msgr2=0x7f8aa006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8aa006ac90 0x7f8aa006d150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8abc14b830 tx=0x7f8ab8009210 comp rx=0 tx=0).stop 2024-03-30T01:42:04.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc14fd30 msgr2=0x7f8abc0a82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:04.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc14fd30 0x7f8abc0a82d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c450 tx=0x7f8aac00c760 comp rx=0 tx=0).stop 2024-03-30T01:42:04.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 shutdown_connections 2024-03-30T01:42:04.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8aa006ac90 0x7f8aa006d150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8abc150110 0x7f8abc0a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8abc0c47c0 0x7f8abc153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 --2- 172.21.15.31:0/4087150552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8abc14fd30 0x7f8abc0a82d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:04.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.903+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 >> 172.21.15.31:0/4087150552 conn(0x7f8abc06ab20 msgr2=0x7f8abc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:04.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.904+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 shutdown_connections 2024-03-30T01:42:04.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:04.904+0000 7f8ac3ca6700 1 -- 172.21.15.31:0/4087150552 wait complete. 2024-03-30T01:42:04.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-30T01:42:05.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.054+0000 7f9d5f139700 1 Processor -- start 2024-03-30T01:42:05.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.054+0000 7f9d5f139700 1 -- start start 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.055+0000 7f9d5f139700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d580c8310 0x7f9d580c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.055+0000 7f9d5f139700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9d580c5df0 0x7f9d58151d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.055+0000 7f9d5f139700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d581556d0 con 0x7f9d580c2870 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.055+0000 7f9d5f139700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d581559b0 con 0x7f9d580c8310 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.055+0000 7f9d5f139700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d580c1f70 con 0x7f9d580c5df0 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.055+0000 7f9d5ced5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d580c8310 0x7f9d580c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.055+0000 7f9d5ced5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d580c8310 0x7f9d580c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59850/0 (socket says 172.21.15.31:59850) 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.055+0000 7f9d5ced5700 1 -- 172.21.15.31:0/2166649286 learned_addr learned my addr 172.21.15.31:0/2166649286 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.056+0000 7f9d57fff700 1 --2- 172.21.15.31:0/2166649286 >> v2:172.21.15.59:3300/0 conn(0x7f9d580c5df0 0x7f9d58151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.056+0000 7f9d5ced5700 1 -- 172.21.15.31:0/2166649286 >> v2:172.21.15.59:3300/0 conn(0x7f9d580c5df0 msgr2=0x7f9d58151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.056+0000 7f9d5ced5700 1 --2- 172.21.15.31:0/2166649286 >> v2:172.21.15.59:3300/0 conn(0x7f9d580c5df0 0x7f9d58151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.056+0000 7f9d5ced5700 1 -- 172.21.15.31:0/2166649286 >> v1:172.21.15.59:6789/0 conn(0x7f9d580c2870 legacy=0x7f9d580a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.056+0000 7f9d5ced5700 1 -- 172.21.15.31:0/2166649286 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d580c2250 con 0x7f9d580c8310 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.056+0000 7f9d5ced5700 1 --2- 172.21.15.31:0/2166649286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c8310 0x7f9d580c86f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009920 tx=0x7f9d48009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec69278a25ab39c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.056+0000 7f9d577fe700 1 -- 172.21.15.31:0/2166649286 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800b9e0 con 0x7f9d580c8310 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.056+0000 7f9d577fe700 1 -- 172.21.15.31:0/2166649286 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4800bb40 con 0x7f9d580c8310 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.056+0000 7f9d577fe700 1 -- 172.21.15.31:0/2166649286 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48009e70 con 0x7f9d580c8310 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.057+0000 7f9d5f139700 1 -- 172.21.15.31:0/2166649286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c8310 msgr2=0x7f9d580c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.057+0000 7f9d5f139700 1 --2- 172.21.15.31:0/2166649286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c8310 0x7f9d580c86f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009920 tx=0x7f9d48009c30 comp rx=0 tx=0).stop 2024-03-30T01:42:05.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.057+0000 7f9d5f139700 1 -- 172.21.15.31:0/2166649286 shutdown_connections 2024-03-30T01:42:05.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.057+0000 7f9d5f139700 1 --2- 172.21.15.31:0/2166649286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c8310 0x7f9d580c86f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.057+0000 7f9d5f139700 1 --2- 172.21.15.31:0/2166649286 >> v2:172.21.15.59:3300/0 conn(0x7f9d580c5df0 0x7f9d58151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.057+0000 7f9d5f139700 1 -- 172.21.15.31:0/2166649286 >> 172.21.15.31:0/2166649286 conn(0x7f9d5806ab20 msgr2=0x7f9d5806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:05.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.057+0000 7f9d5f139700 1 -- 172.21.15.31:0/2166649286 shutdown_connections 2024-03-30T01:42:05.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.057+0000 7f9d5f139700 1 -- 172.21.15.31:0/2166649286 wait complete. 2024-03-30T01:42:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.058+0000 7f9d5f139700 1 Processor -- start 2024-03-30T01:42:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.058+0000 7f9d5f139700 1 -- start start 2024-03-30T01:42:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5f139700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d580c2870 0x7f9d581ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5f139700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c5df0 0x7f9d581ece90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5f139700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d580c8310 0x7f9d58150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5f139700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5809a250 con 0x7f9d580c5df0 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5f139700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581ea910 con 0x7f9d580c8310 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5f139700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58151750 con 0x7f9d580c2870 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d57fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c5df0 0x7f9d581ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d57fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c5df0 0x7f9d581ece90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59858/0 (socket says 172.21.15.31:59858) 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d57fff700 1 -- 172.21.15.31:0/2835662080 learned_addr learned my addr 172.21.15.31:0/2835662080 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5d6d6700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d580c8310 0x7f9d58150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5ced5700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d580c2870 0x7f9d581ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d57fff700 1 -- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d580c2870 msgr2=0x7f9d581ea3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d57fff700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d580c2870 0x7f9d581ea3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d57fff700 1 -- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d580c8310 msgr2=0x7f9d58150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d57fff700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d580c8310 0x7f9d58150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d57fff700 1 -- 172.21.15.31:0/2835662080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48009550 con 0x7f9d580c5df0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5ced5700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d580c2870 0x7f9d581ea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.059+0000 7f9d5d6d6700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d580c8310 0x7f9d58150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.060+0000 7f9d57fff700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c5df0 0x7f9d581ece90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000e810 tx=0x7f9d4000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.060+0000 7f9d557fa700 1 -- 172.21.15.31:0/2835662080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40013610 con 0x7f9d580c5df0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.060+0000 7f9d557fa700 1 -- 172.21.15.31:0/2835662080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d40013770 con 0x7f9d580c5df0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.060+0000 7f9d557fa700 1 -- 172.21.15.31:0/2835662080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40013b20 con 0x7f9d580c5df0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.060+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d58151a30 con 0x7f9d580c5df0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.060+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d581e9eb0 con 0x7f9d580c5df0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.061+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d58073760 con 0x7f9d580c5df0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.061+0000 7f9d557fa700 1 -- 172.21.15.31:0/2835662080 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9d40013c80 con 0x7f9d580c5df0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.062+0000 7f9d557fa700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d3806ac90 0x7f9d3806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.062+0000 7f9d557fa700 1 -- 172.21.15.31:0/2835662080 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9d40088e50 con 0x7f9d580c5df0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.062+0000 7f9d5ced5700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d3806ac90 0x7f9d3806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.062+0000 7f9d5ced5700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d3806ac90 0x7f9d3806d150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9d48014040 tx=0x7f9d48008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:05.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.066+0000 7f9d557fa700 1 -- 172.21.15.31:0/2835662080 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9d40054250 con 0x7f9d580c5df0 2024-03-30T01:42:05.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.369+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9d580c2f30 con 0x7f9d580c5df0 2024-03-30T01:42:05.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.373+0000 7f9d557fa700 1 -- 172.21.15.31:0/2835662080 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9d40057870 con 0x7f9d580c5df0 2024-03-30T01:42:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d3806ac90 msgr2=0x7f9d3806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d3806ac90 0x7f9d3806d150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9d48014040 tx=0x7f9d48008030 comp rx=0 tx=0).stop 2024-03-30T01:42:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c5df0 msgr2=0x7f9d581ece90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c5df0 0x7f9d581ece90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000e810 tx=0x7f9d4000ebd0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 shutdown_connections 2024-03-30T01:42:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9d3806ac90 0x7f9d3806d150 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d580c8310 0x7f9d58150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d580c2870 0x7f9d581ea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 --2- 172.21.15.31:0/2835662080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d580c5df0 0x7f9d581ece90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 >> 172.21.15.31:0/2835662080 conn(0x7f9d5806ab20 msgr2=0x7f9d580ace90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.375+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 shutdown_connections 2024-03-30T01:42:05.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.376+0000 7f9d5f139700 1 -- 172.21.15.31:0/2835662080 wait complete. 2024-03-30T01:42:05.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-30T01:42:05.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.526+0000 7efea8ef9700 1 Processor -- start 2024-03-30T01:42:05.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.527+0000 7efea8ef9700 1 -- start start 2024-03-30T01:42:05.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea8ef9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea4154cf0 0x7efea40c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea8ef9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efea4072270 0x7efea4151490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea8ef9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea40a8460 con 0x7efea40c46a0 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea8ef9700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea406dcb0 con 0x7efea4072270 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea8ef9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea40c2e40 con 0x7efea4154cf0 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efea4072270 0x7efea4151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efea4072270 0x7efea4151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47118/0 (socket says 172.21.15.31:47118) 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea2d9d700 1 -- 172.21.15.31:0/1689629180 learned_addr learned my addr 172.21.15.31:0/1689629180 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea259c700 1 --2- 172.21.15.31:0/1689629180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea4154cf0 0x7efea40c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea2d9d700 1 -- 172.21.15.31:0/1689629180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea4154cf0 msgr2=0x7efea40c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea2d9d700 1 --2- 172.21.15.31:0/1689629180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea4154cf0 0x7efea40c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea2d9d700 1 -- 172.21.15.31:0/1689629180 >> v1:172.21.15.59:6789/0 conn(0x7efea40c46a0 legacy=0x7efea406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea2d9d700 1 -- 172.21.15.31:0/1689629180 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea4155630 con 0x7efea4072270 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea259c700 1 --2- 172.21.15.31:0/1689629180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea4154cf0 0x7efea40c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea2d9d700 1 --2- 172.21.15.31:0/1689629180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 0x7efea4151490 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d150 tx=0x7efe9c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ccf3c95000f6df server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.528+0000 7efea159a700 1 -- 172.21.15.31:0/1689629180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c010070 con 0x7efea4072270 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea159a700 1 -- 172.21.15.31:0/1689629180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c00ddc0 con 0x7efea4072270 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea159a700 1 -- 172.21.15.31:0/1689629180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00b310 con 0x7efea4072270 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea8ef9700 1 -- 172.21.15.31:0/1689629180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 msgr2=0x7efea4151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea8ef9700 1 --2- 172.21.15.31:0/1689629180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 0x7efea4151490 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d150 tx=0x7efe9c00d460 comp rx=0 tx=0).stop 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea8ef9700 1 -- 172.21.15.31:0/1689629180 shutdown_connections 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea8ef9700 1 --2- 172.21.15.31:0/1689629180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 0x7efea4151490 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea8ef9700 1 --2- 172.21.15.31:0/1689629180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea4154cf0 0x7efea40c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea8ef9700 1 -- 172.21.15.31:0/1689629180 >> 172.21.15.31:0/1689629180 conn(0x7efea40c3a60 msgr2=0x7efea40c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea8ef9700 1 -- 172.21.15.31:0/1689629180 shutdown_connections 2024-03-30T01:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.529+0000 7efea8ef9700 1 -- 172.21.15.31:0/1689629180 wait complete. 2024-03-30T01:42:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.531+0000 7efea8ef9700 1 Processor -- start 2024-03-30T01:42:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.531+0000 7efea8ef9700 1 -- start start 2024-03-30T01:42:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.531+0000 7efea8ef9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 0x7efea4144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.531+0000 7efea8ef9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea40c46a0 0x7efea414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.531+0000 7efea8ef9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efea4154cf0 0x7efea4152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.531+0000 7efea8ef9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea41525c0 con 0x7efea4072270 2024-03-30T01:42:05.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.531+0000 7efea8ef9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea4144770 con 0x7efea4154cf0 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.531+0000 7efea8ef9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea4143c50 con 0x7efea40c46a0 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 0x7efea4144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 0x7efea4144230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47126/0 (socket says 172.21.15.31:47126) 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea259c700 1 -- 172.21.15.31:0/3601789277 learned_addr learned my addr 172.21.15.31:0/3601789277 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea2d9d700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efea4154cf0 0x7efea4152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea1d9b700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea40c46a0 0x7efea414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea259c700 1 -- 172.21.15.31:0/3601789277 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea40c46a0 msgr2=0x7efea414d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea259c700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea40c46a0 0x7efea414d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea259c700 1 -- 172.21.15.31:0/3601789277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efea4154cf0 msgr2=0x7efea4152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea259c700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efea4154cf0 0x7efea4152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea259c700 1 -- 172.21.15.31:0/3601789277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c00bd40 con 0x7efea4072270 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea2d9d700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efea4154cf0 0x7efea4152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea259c700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 0x7efea4144230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe9400e840 tx=0x7efe9400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efe92ffd700 1 -- 172.21.15.31:0/3601789277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9400c020 con 0x7efea4072270 2024-03-30T01:42:05.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efe92ffd700 1 -- 172.21.15.31:0/3601789277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe94004250 con 0x7efea4072270 2024-03-30T01:42:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efe92ffd700 1 -- 172.21.15.31:0/3601789277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe940036a0 con 0x7efea4072270 2024-03-30T01:42:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea41e6a60 con 0x7efea4072270 2024-03-30T01:42:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.532+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea41527a0 con 0x7efea4072270 2024-03-30T01:42:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.533+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea40c6d80 con 0x7efea4072270 2024-03-30T01:42:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.534+0000 7efe92ffd700 1 -- 172.21.15.31:0/3601789277 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efe94003800 con 0x7efea4072270 2024-03-30T01:42:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.534+0000 7efe92ffd700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe8406ac90 0x7efe8406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.534+0000 7efe92ffd700 1 -- 172.21.15.31:0/3601789277 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7efe94088ab0 con 0x7efea4072270 2024-03-30T01:42:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.534+0000 7efea1d9b700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe8406ac90 0x7efe8406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:05.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.535+0000 7efea1d9b700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe8406ac90 0x7efe8406d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efea406b330 tx=0x7efe98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:05.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.539+0000 7efe92ffd700 1 -- 172.21.15.31:0/3601789277 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efe94053f30 con 0x7efea4072270 2024-03-30T01:42:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.844+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7efea404ea80 con 0x7efea4072270 2024-03-30T01:42:05.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.848+0000 7efe92ffd700 1 -- 172.21.15.31:0/3601789277 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7efe94057550 con 0x7efea4072270 2024-03-30T01:42:05.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe8406ac90 msgr2=0x7efe8406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe8406ac90 0x7efe8406d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efea406b330 tx=0x7efe98009040 comp rx=0 tx=0).stop 2024-03-30T01:42:05.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 msgr2=0x7efea4144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:05.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 0x7efea4144230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe9400e840 tx=0x7efe9400ec00 comp rx=0 tx=0).stop 2024-03-30T01:42:05.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 shutdown_connections 2024-03-30T01:42:05.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efea4072270 0x7efea4144230 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efe8406ac90 0x7efe8406d150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efea4154cf0 0x7efea4152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 --2- 172.21.15.31:0/3601789277 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efea40c46a0 0x7efea414d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:05.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 >> 172.21.15.31:0/3601789277 conn(0x7efea40c3a60 msgr2=0x7efea4146b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 shutdown_connections 2024-03-30T01:42:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:05.851+0000 7efea8ef9700 1 -- 172.21.15.31:0/3601789277 wait complete. 2024-03-30T01:42:05.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-30T01:42:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.002+0000 7fcfb3a50700 1 Processor -- start 2024-03-30T01:42:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.003+0000 7fcfb3a50700 1 -- start start 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.003+0000 7fcfb3a50700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfac0ac520 0x7fcfac0ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.003+0000 7fcfb3a50700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcfac0ace40 0x7fcfac0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.003+0000 7fcfb3a50700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac153590 con 0x7fcfac0a74e0 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.003+0000 7fcfb3a50700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfac0ab680 con 0x7fcfac0ace40 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.003+0000 7fcfb3a50700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1500c0 con 0x7fcfac0ac520 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfb0feb700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcfac0a74e0 0x7fcfac0c6000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41628/0 (socket says 172.21.15.31:41628) 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfb0feb700 1 -- 172.21.15.31:0/1568908211 learned_addr learned my addr 172.21.15.31:0/1568908211 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfb1fed700 1 --2- 172.21.15.31:0/1568908211 >> v2:172.21.15.59:3300/0 conn(0x7fcfac0ace40 0x7fcfac0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfb17ec700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfac0ac520 0x7fcfac0ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684591848 0 0) 0x7fcfac153590 con 0x7fcfac0a74e0 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf980035b0 con 0x7fcfac0a74e0 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3508474783 0 0) 0x7fcf980035b0 con 0x7fcfac0a74e0 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfac0ac520 msgr2=0x7fcfac0ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfac0ac520 0x7fcfac0ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 >> v2:172.21.15.59:3300/0 conn(0x7fcfac0ace40 msgr2=0x7fcfac0c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 --2- 172.21.15.31:0/1568908211 >> v2:172.21.15.59:3300/0 conn(0x7fcfac0ace40 0x7fcfac0c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0c7990 con 0x7fcfac0a74e0 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7fcf9c001670 con 0x7fcfac0a74e0 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 >> v1:172.21.15.31:6789/0 conn(0x7fcfac0a74e0 legacy=0x7fcfac0c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf98003a00 0x7fcf98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfac0ac520 0x7fcf98004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf98004d90 0x7fcf98005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0ab680 con 0x7fcf98003a00 2024-03-30T01:42:06.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1500c0 con 0x7fcf98004d90 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9800ba10 con 0x7fcfac0ac520 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfb1fed700 1 --2- 172.21.15.31:0/1568908211 >> v2:172.21.15.59:3300/0 conn(0x7fcfac0ace40 0x7fcfac0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfb17ec700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf98003a00 0x7fcf98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.004+0000 7fcfb0feb700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfac0ac520 0x7fcf98004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfb1fed700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf98004d90 0x7fcf98005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfb17ec700 1 -- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfac0ac520 msgr2=0x7fcf98004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfb17ec700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfac0ac520 0x7fcf98004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfb17ec700 1 -- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf98004d90 msgr2=0x7fcf98005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfb17ec700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf98004d90 0x7fcf98005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfb17ec700 1 -- 172.21.15.31:0/1568908211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf980035b0 con 0x7fcf98003a00 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfb0feb700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfac0ac520 0x7fcf98004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfb17ec700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf98003a00 0x7fcf98003e20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009b60 tx=0x7fcf98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fd7dcedcaab7da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800e6c0 con 0x7fcf98003a00 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa800e820 con 0x7fcf98003a00 2024-03-30T01:42:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.005+0000 7fcfa3fff700 1 -- 172.21.15.31:0/1568908211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800eb90 con 0x7fcf98003a00 2024-03-30T01:42:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.006+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1568908211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf98003a00 msgr2=0x7fcf98003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.006+0000 7fcfb3a50700 1 --2- 172.21.15.31:0/1568908211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf98003a00 0x7fcf98003e20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009b60 tx=0x7fcf98005880 comp rx=0 tx=0).stop 2024-03-30T01:42:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.006+0000 7fcfb1fed700 1 -- 172.21.15.31:0/1568908211 reap_dead start 2024-03-30T01:42:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.006+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1568908211 shutdown_connections 2024-03-30T01:42:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.006+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1568908211 >> 172.21.15.31:0/1568908211 conn(0x7fcfac06ad30 msgr2=0x7fcfac0a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.006+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1568908211 shutdown_connections 2024-03-30T01:42:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.006+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1568908211 wait complete. 2024-03-30T01:42:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.007+0000 7fcfb3a50700 1 Processor -- start 2024-03-30T01:42:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.007+0000 7fcfb3a50700 1 -- start start 2024-03-30T01:42:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb3a50700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf98003a00 0x7fcfac1549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb3a50700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfac14f060 0x7fcfac0c1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb3a50700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac06b8a0 0x7fcfac156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb3a50700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1eaa40 con 0x7fcfac06b8a0 2024-03-30T01:42:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb3a50700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1eaf70 con 0x7fcfac14f060 2024-03-30T01:42:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb3a50700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1eac90 con 0x7fcf98003a00 2024-03-30T01:42:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb1fed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac06b8a0 0x7fcfac156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb1fed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac06b8a0 0x7fcfac156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47146/0 (socket says 172.21.15.31:47146) 2024-03-30T01:42:06.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb1fed700 1 -- 172.21.15.31:0/1575085284 learned_addr learned my addr 172.21.15.31:0/1575085284 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb17ec700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf98003a00 0x7fcfac1549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb0feb700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfac14f060 0x7fcfac0c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb1fed700 1 -- 172.21.15.31:0/1575085284 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf98003a00 msgr2=0x7fcfac1549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb1fed700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf98003a00 0x7fcfac1549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb1fed700 1 -- 172.21.15.31:0/1575085284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfac14f060 msgr2=0x7fcfac0c1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb1fed700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfac14f060 0x7fcfac0c1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb1fed700 1 -- 172.21.15.31:0/1575085284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac1dbc90 con 0x7fcfac06b8a0 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.008+0000 7fcfb1fed700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac06b8a0 0x7fcfac156310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0abf50 tx=0x7fcfa4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.009+0000 7fcfa1ffb700 1 -- 172.21.15.31:0/1575085284 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa400e070 con 0x7fcfac06b8a0 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.009+0000 7fcfa1ffb700 1 -- 172.21.15.31:0/1575085284 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa4011d00 con 0x7fcfac06b8a0 2024-03-30T01:42:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.009+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac1dbf70 con 0x7fcfac06b8a0 2024-03-30T01:42:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.009+0000 7fcfa1ffb700 1 -- 172.21.15.31:0/1575085284 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4011e90 con 0x7fcfac06b8a0 2024-03-30T01:42:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.009+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac156850 con 0x7fcfac06b8a0 2024-03-30T01:42:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.010+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac04ea80 con 0x7fcfac06b8a0 2024-03-30T01:42:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.010+0000 7fcfa1ffb700 1 -- 172.21.15.31:0/1575085284 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcfa4003a20 con 0x7fcfac06b8a0 2024-03-30T01:42:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.011+0000 7fcfa1ffb700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9006ac90 0x7fcf9006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.011+0000 7fcfa1ffb700 1 -- 172.21.15.31:0/1575085284 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fcfa4089860 con 0x7fcfac06b8a0 2024-03-30T01:42:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.011+0000 7fcfb17ec700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9006ac90 0x7fcf9006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.012+0000 7fcfb17ec700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9006ac90 0x7fcf9006d150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80090b0 tx=0x7fcfa8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.016+0000 7fcfa1ffb700 1 -- 172.21.15.31:0/1575085284 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcfa4054c40 con 0x7fcfac06b8a0 2024-03-30T01:42:06.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.316+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fcfac0c29f0 con 0x7fcfac06b8a0 2024-03-30T01:42:06.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.320+0000 7fcfa1ffb700 1 -- 172.21.15.31:0/1575085284 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcfa4058260 con 0x7fcfac06b8a0 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9006ac90 msgr2=0x7fcf9006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9006ac90 0x7fcf9006d150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80090b0 tx=0x7fcfa8008030 comp rx=0 tx=0).stop 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac06b8a0 msgr2=0x7fcfac156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac06b8a0 0x7fcfac156310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0abf50 tx=0x7fcfa4009f90 comp rx=0 tx=0).stop 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 shutdown_connections 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac06b8a0 0x7fcfac156310 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9006ac90 0x7fcf9006d150 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf98003a00 0x7fcfac1549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 --2- 172.21.15.31:0/1575085284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfac14f060 0x7fcfac0c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 >> 172.21.15.31:0/1575085284 conn(0x7fcfac06ad30 msgr2=0x7fcfac152460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:06.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 shutdown_connections 2024-03-30T01:42:06.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.322+0000 7fcfb3a50700 1 -- 172.21.15.31:0/1575085284 wait complete. 2024-03-30T01:42:06.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:06.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.472+0000 7fd74974e700 1 Processor -- start 2024-03-30T01:42:06.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.472+0000 7fd74974e700 1 -- start start 2024-03-30T01:42:06.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.472+0000 7fd74974e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7440bd7f0 0x7fd744155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.472+0000 7fd74974e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7440c8d10 con 0x7fd744154760 2024-03-30T01:42:06.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.472+0000 7fd74974e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7440c98b0 con 0x7fd74406b170 2024-03-30T01:42:06.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.472+0000 7fd74974e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74406bef0 con 0x7fd7440bd7f0 2024-03-30T01:42:06.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd7437fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd744154760 0x7fd74415f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41634/0 (socket says 172.21.15.31:41634) 2024-03-30T01:42:06.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd7437fe700 1 -- 172.21.15.31:0/526699320 learned_addr learned my addr 172.21.15.31:0/526699320 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:06.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd7427fc700 1 --2- 172.21.15.31:0/526699320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7440bd7f0 0x7fd744155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd741ffb700 1 -- 172.21.15.31:0/526699320 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778450537 0 0) 0x7fd7440c8d10 con 0x7fd744154760 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd741ffb700 1 -- 172.21.15.31:0/526699320 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd72c0035b0 con 0x7fd744154760 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd741ffb700 1 -- 172.21.15.31:0/526699320 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540634562 0 0) 0x7fd7440c98b0 con 0x7fd74406b170 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd741ffb700 1 -- 172.21.15.31:0/526699320 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7440c8d10 con 0x7fd74406b170 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd7427fc700 1 -- 172.21.15.31:0/526699320 >> v1:172.21.15.59:6789/0 conn(0x7fd74406b170 legacy=0x7fd7440c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd7427fc700 1 -- 172.21.15.31:0/526699320 >> v1:172.21.15.31:6789/0 conn(0x7fd744154760 legacy=0x7fd74415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd7427fc700 1 -- 172.21.15.31:0/526699320 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7440c8100 con 0x7fd7440bd7f0 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.473+0000 7fd7427fc700 1 --2- 172.21.15.31:0/526699320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7440bd7f0 0x7fd744155c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd738009870 tx=0x7fd738009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7212c27192ee0690 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd741ffb700 1 -- 172.21.15.31:0/526699320 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738011830 con 0x7fd7440bd7f0 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd741ffb700 1 -- 172.21.15.31:0/526699320 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd738011990 con 0x7fd7440bd7f0 2024-03-30T01:42:06.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd741ffb700 1 -- 172.21.15.31:0/526699320 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738011d20 con 0x7fd7440bd7f0 2024-03-30T01:42:06.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd74974e700 1 -- 172.21.15.31:0/526699320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7440bd7f0 msgr2=0x7fd744155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd74974e700 1 --2- 172.21.15.31:0/526699320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7440bd7f0 0x7fd744155c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd738009870 tx=0x7fd738009b80 comp rx=0 tx=0).stop 2024-03-30T01:42:06.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd74974e700 1 -- 172.21.15.31:0/526699320 shutdown_connections 2024-03-30T01:42:06.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd74974e700 1 --2- 172.21.15.31:0/526699320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7440bd7f0 0x7fd744155c50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd74974e700 1 -- 172.21.15.31:0/526699320 >> 172.21.15.31:0/526699320 conn(0x7fd7440ca0d0 msgr2=0x7fd7440ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:06.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd74974e700 1 -- 172.21.15.31:0/526699320 shutdown_connections 2024-03-30T01:42:06.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.474+0000 7fd74974e700 1 -- 172.21.15.31:0/526699320 wait complete. 2024-03-30T01:42:06.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.476+0000 7fd74974e700 1 Processor -- start 2024-03-30T01:42:06.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.476+0000 7fd74974e700 1 -- start start 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.476+0000 7fd74974e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd74406b170 0x7fd7441eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.476+0000 7fd74974e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7440bd7f0 0x7fd7441ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd74974e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd744154760 0x7fd7440aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd74974e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7440c2f70 con 0x7fd7440bd7f0 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd74974e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7441ead10 con 0x7fd74406b170 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd74974e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7440ae100 con 0x7fd744154760 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7427fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7440bd7f0 0x7fd7441ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7427fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7440bd7f0 0x7fd7441ea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47150/0 (socket says 172.21.15.31:47150) 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7427fc700 1 -- 172.21.15.31:0/1739527035 learned_addr learned my addr 172.21.15.31:0/1739527035 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd742ffd700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd74406b170 0x7fd7441eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7437fe700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd744154760 0x7fd7440aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7427fc700 1 -- 172.21.15.31:0/1739527035 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd744154760 msgr2=0x7fd7440aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7427fc700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd744154760 0x7fd7440aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7427fc700 1 -- 172.21.15.31:0/1739527035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd74406b170 msgr2=0x7fd7441eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7427fc700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd74406b170 0x7fd7441eddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7427fc700 1 -- 172.21.15.31:0/1739527035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380094a0 con 0x7fd7440bd7f0 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd7427fc700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7440bd7f0 0x7fd7441ea7d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd7380090b0 tx=0x7fd738004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd72b7fe700 1 -- 172.21.15.31:0/1739527035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73801a860 con 0x7fd7440bd7f0 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd72b7fe700 1 -- 172.21.15.31:0/1739527035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73801a9c0 con 0x7fd7440bd7f0 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7441ed930 con 0x7fd7440bd7f0 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd72b7fe700 1 -- 172.21.15.31:0/1739527035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73801ad30 con 0x7fd7440bd7f0 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.477+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7440ae350 con 0x7fd7440bd7f0 2024-03-30T01:42:06.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.479+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74404ea80 con 0x7fd7440bd7f0 2024-03-30T01:42:06.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.479+0000 7fd72b7fe700 1 -- 172.21.15.31:0/1739527035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd738022070 con 0x7fd7440bd7f0 2024-03-30T01:42:06.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.479+0000 7fd72b7fe700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd73006ac90 0x7fd73006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.480+0000 7fd72b7fe700 1 -- 172.21.15.31:0/1739527035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd73808a160 con 0x7fd7440bd7f0 2024-03-30T01:42:06.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.480+0000 7fd742ffd700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd73006ac90 0x7fd73006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.480+0000 7fd742ffd700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd73006ac90 0x7fd73006d150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd734001220 tx=0x7fd734009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:06.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.484+0000 7fd72b7fe700 1 -- 172.21.15.31:0/1739527035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd7380555e0 con 0x7fd7440bd7f0 2024-03-30T01:42:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.698+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7440bce80 con 0x7fd7440bd7f0 2024-03-30T01:42:06.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.699+0000 7fd72b7fe700 1 -- 172.21.15.31:0/1739527035 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+12546 (secure 0 0 0) 0x7fd738058c00 con 0x7fd7440bd7f0 2024-03-30T01:42:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:06.706 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":35,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:02.999497+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd73006ac90 msgr2=0x7fd73006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd73006ac90 0x7fd73006d150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd734001220 tx=0x7fd734009040 comp rx=0 tx=0).stop 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7440bd7f0 msgr2=0x7fd7441ea7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7440bd7f0 0x7fd7441ea7d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd7380090b0 tx=0x7fd738004750 comp rx=0 tx=0).stop 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 shutdown_connections 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd73006ac90 0x7fd73006d150 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd744154760 0x7fd7440aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd74406b170 0x7fd7441eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 --2- 172.21.15.31:0/1739527035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7440bd7f0 0x7fd7441ea7d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 >> 172.21.15.31:0/1739527035 conn(0x7fd7440ca0d0 msgr2=0x7fd7441525d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:06.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 shutdown_connections 2024-03-30T01:42:06.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.701+0000 7fd74974e700 1 -- 172.21.15.31:0/1739527035 wait complete. 2024-03-30T01:42:06.722 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:42:06.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:42:06.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.855+0000 7fa1a8a9f700 1 Processor -- start 2024-03-30T01:42:06.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.855+0000 7fa1a8a9f700 1 -- start start 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.856+0000 7fa1a8a9f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.856+0000 7fa1a8a9f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1a40aa110 0x7fa1a41513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.856+0000 7fa1a8a9f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40a9e40 con 0x7fa1a40ab840 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.856+0000 7fa1a8a9f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a4152150 con 0x7fa1a40aa110 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.856+0000 7fa1a8a9f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40c2830 con 0x7fa1a414e980 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a259c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a1d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1a40aa110 0x7fa1a41513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a259c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a406d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:44498/0 (socket says 172.21.15.31:44498) 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a1d9b700 1 --2- 172.21.15.31:0/2080955127 >> v2:172.21.15.59:3300/0 conn(0x7fa1a40aa110 0x7fa1a41513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:55848/0 (socket says 172.21.15.31:55848) 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a259c700 1 -- 172.21.15.31:0/2080955127 learned_addr learned my addr 172.21.15.31:0/2080955127 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a2d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa1a40ab840 0x7fa1a4151930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:55246/0 (socket says 172.21.15.31:55246) 2024-03-30T01:42:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a259c700 1 -- 172.21.15.31:0/2080955127 >> v2:172.21.15.59:3300/0 conn(0x7fa1a40aa110 msgr2=0x7fa1a41513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a259c700 1 --2- 172.21.15.31:0/2080955127 >> v2:172.21.15.59:3300/0 conn(0x7fa1a40aa110 0x7fa1a41513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a259c700 1 -- 172.21.15.31:0/2080955127 >> v1:172.21.15.59:6789/0 conn(0x7fa1a40ab840 legacy=0x7fa1a4151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.857+0000 7fa1a259c700 1 -- 172.21.15.31:0/2080955127 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a40c2b10 con 0x7fa1a414e980 2024-03-30T01:42:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a259c700 1 --2- 172.21.15.31:0/2080955127 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a406d740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa1940098f0 tx=0x7fa194009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c483b70eb4c2afc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a159a700 1 -- 172.21.15.31:0/2080955127 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194011830 con 0x7fa1a414e980 2024-03-30T01:42:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a159a700 1 -- 172.21.15.31:0/2080955127 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa194011990 con 0x7fa1a414e980 2024-03-30T01:42:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a159a700 1 -- 172.21.15.31:0/2080955127 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194011d20 con 0x7fa1a414e980 2024-03-30T01:42:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2080955127 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 msgr2=0x7fa1a406d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a8a9f700 1 --2- 172.21.15.31:0/2080955127 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a406d740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa1940098f0 tx=0x7fa194009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2080955127 shutdown_connections 2024-03-30T01:42:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a8a9f700 1 --2- 172.21.15.31:0/2080955127 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a406d740 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a8a9f700 1 --2- 172.21.15.31:0/2080955127 >> v2:172.21.15.59:3300/0 conn(0x7fa1a40aa110 0x7fa1a41513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.858+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2080955127 >> 172.21.15.31:0/2080955127 conn(0x7fa1a40c6550 msgr2=0x7fa1a40c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.859+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2080955127 shutdown_connections 2024-03-30T01:42:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.859+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2080955127 wait complete. 2024-03-30T01:42:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.860+0000 7fa1a8a9f700 1 Processor -- start 2024-03-30T01:42:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.860+0000 7fa1a8a9f700 1 -- start start 2024-03-30T01:42:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.861+0000 7fa1a8a9f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a40aa110 0x7fa1a4149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.861+0000 7fa1a8a9f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1a40ab840 0x7fa1a4148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.861+0000 7fa1a8a9f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a4147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.861+0000 7fa1a8a9f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a414a790 con 0x7fa1a40ab840 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.861+0000 7fa1a8a9f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40729c0 con 0x7fa1a40aa110 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.861+0000 7fa1a8a9f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a414b740 con 0x7fa1a414e980 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.861+0000 7fa1a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1a40ab840 0x7fa1a4148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.861+0000 7fa1a1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1a40ab840 0x7fa1a4148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47164/0 (socket says 172.21.15.31:47164) 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a1d9b700 1 -- 172.21.15.31:0/2025990452 learned_addr learned my addr 172.21.15.31:0/2025990452 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a259c700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a40aa110 0x7fa1a4149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a2d9d700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a4147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a1d9b700 1 -- 172.21.15.31:0/2025990452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 msgr2=0x7fa1a4147f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a1d9b700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a4147f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a1d9b700 1 -- 172.21.15.31:0/2025990452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a40aa110 msgr2=0x7fa1a4149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a1d9b700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a40aa110 0x7fa1a4149ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:06.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a1d9b700 1 -- 172.21.15.31:0/2025990452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194009550 con 0x7fa1a40ab840 2024-03-30T01:42:06.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a1d9b700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1a40ab840 0x7fa1a4148d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa19800e9b0 tx=0x7fa19800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:06.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa192ffd700 1 -- 172.21.15.31:0/2025990452 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198009710 con 0x7fa1a40ab840 2024-03-30T01:42:06.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa192ffd700 1 -- 172.21.15.31:0/2025990452 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19800ee50 con 0x7fa1a40ab840 2024-03-30T01:42:06.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa192ffd700 1 -- 172.21.15.31:0/2025990452 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19800ca50 con 0x7fa1a40ab840 2024-03-30T01:42:06.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a414b9e0 con 0x7fa1a40ab840 2024-03-30T01:42:06.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.862+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a41eb1d0 con 0x7fa1a40ab840 2024-03-30T01:42:06.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.863+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a40c1f40 con 0x7fa1a40ab840 2024-03-30T01:42:06.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.864+0000 7fa192ffd700 1 -- 172.21.15.31:0/2025990452 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa19800cbb0 con 0x7fa1a40ab840 2024-03-30T01:42:06.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.864+0000 7fa192ffd700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa18406ac90 0x7fa18406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:06.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.864+0000 7fa192ffd700 1 -- 172.21.15.31:0/2025990452 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa198089170 con 0x7fa1a40ab840 2024-03-30T01:42:06.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.864+0000 7fa1a259c700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa18406ac90 0x7fa18406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:06.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.865+0000 7fa1a259c700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa18406ac90 0x7fa18406d150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa194009c40 tx=0x7fa194008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:06.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:06.869+0000 7fa192ffd700 1 -- 172.21.15.31:0/2025990452 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa1a40c1f40 con 0x7fa1a40ab840 2024-03-30T01:42:07.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.079+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1a40c1f40 con 0x7fa1a40ab840 2024-03-30T01:42:07.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.080+0000 7fa192ffd700 1 -- 172.21.15.31:0/2025990452 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1a40c1f40 con 0x7fa1a40ab840 2024-03-30T01:42:07.087 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:42:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa18406ac90 msgr2=0x7fa18406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa18406ac90 0x7fa18406d150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa194009c40 tx=0x7fa194008030 comp rx=0 tx=0).stop 2024-03-30T01:42:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1a40ab840 msgr2=0x7fa1a4148d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1a40ab840 0x7fa1a4148d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa19800e9b0 tx=0x7fa19800ecc0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 shutdown_connections 2024-03-30T01:42:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa18406ac90 0x7fa18406d150 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a414e980 0x7fa1a4147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a40aa110 0x7fa1a4149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 --2- 172.21.15.31:0/2025990452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1a40ab840 0x7fa1a4148d30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 >> 172.21.15.31:0/2025990452 conn(0x7fa1a40c6550 msgr2=0x7fa1a40c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 shutdown_connections 2024-03-30T01:42:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.082+0000 7fa1a8a9f700 1 -- 172.21.15.31:0/2025990452 wait complete. 2024-03-30T01:42:07.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T01:42:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.232+0000 7f18e9095700 1 Processor -- start 2024-03-30T01:42:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.232+0000 7f18e9095700 1 -- start start 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.232+0000 7f18e9095700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e40c5df0 0x7f18e40c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.232+0000 7f18e9095700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f18e40c2870 0x7f18e40a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.232+0000 7f18e9095700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e41556d0 con 0x7f18e40c8310 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.232+0000 7f18e9095700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e41559b0 con 0x7f18e40c2870 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.232+0000 7f18e9095700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40c1f70 con 0x7f18e40c5df0 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e2d9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f18e40c8310 0x7f18e4151d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41644/0 (socket says 172.21.15.31:41644) 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e2d9d700 1 -- 172.21.15.31:0/3126201227 learned_addr learned my addr 172.21.15.31:0/3126201227 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e359e700 1 --2- 172.21.15.31:0/3126201227 >> v2:172.21.15.59:3300/0 conn(0x7f18e40c2870 0x7f18e40a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e259c700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e40c5df0 0x7f18e40c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294785636 0 0) 0x7f18e41556d0 con 0x7f18e40c8310 2024-03-30T01:42:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18cc0035b0 con 0x7f18e40c8310 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409272424 0 0) 0x7f18cc0035b0 con 0x7f18e40c8310 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e40c5df0 msgr2=0x7f18e40c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e40c5df0 0x7f18e40c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 >> v2:172.21.15.59:3300/0 conn(0x7f18e40c2870 msgr2=0x7f18e40a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 --2- 172.21.15.31:0/3126201227 >> v2:172.21.15.59:3300/0 conn(0x7f18e40c2870 0x7f18e40a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e40c2250 con 0x7f18e40c8310 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f18d4001670 con 0x7f18e40c8310 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 >> v1:172.21.15.31:6789/0 conn(0x7f18e40c8310 legacy=0x7f18e4151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc003a00 0x7f18cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e40c5df0 0x7f18cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc004d90 0x7f18cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e41559b0 con 0x7f18cc004d90 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40c1f70 con 0x7f18cc003a00 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.233+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc00ba10 con 0x7f18e40c5df0 2024-03-30T01:42:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e359e700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc004d90 0x7f18cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e259c700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e40c5df0 0x7f18cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e2d9d700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc003a00 0x7f18cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e359e700 1 -- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e40c5df0 msgr2=0x7f18cc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e359e700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e40c5df0 0x7f18cc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e359e700 1 -- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc003a00 msgr2=0x7f18cc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e359e700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18cc003a00 0x7f18cc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e359e700 1 -- 172.21.15.31:0/3126201227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc0035b0 con 0x7f18cc004d90 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e359e700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc004d90 0x7f18cc005210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e170 tx=0x7f18cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f061665cd38cf697 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00ec90 con 0x7f18cc004d90 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc00f040 con 0x7f18cc004d90 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.234+0000 7f18e1d9b700 1 -- 172.21.15.31:0/3126201227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc014480 con 0x7f18cc004d90 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.235+0000 7f18e9095700 1 -- 172.21.15.31:0/3126201227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc004d90 msgr2=0x7f18cc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.235+0000 7f18e9095700 1 --2- 172.21.15.31:0/3126201227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc004d90 0x7f18cc005210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00e170 tx=0x7f18cc005880 comp rx=0 tx=0).stop 2024-03-30T01:42:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.235+0000 7f18e359e700 1 -- 172.21.15.31:0/3126201227 reap_dead start 2024-03-30T01:42:07.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.235+0000 7f18e9095700 1 -- 172.21.15.31:0/3126201227 shutdown_connections 2024-03-30T01:42:07.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.235+0000 7f18e9095700 1 -- 172.21.15.31:0/3126201227 >> 172.21.15.31:0/3126201227 conn(0x7f18e406ab20 msgr2=0x7f18e406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:07.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.235+0000 7f18e9095700 1 -- 172.21.15.31:0/3126201227 shutdown_connections 2024-03-30T01:42:07.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.235+0000 7f18e9095700 1 -- 172.21.15.31:0/3126201227 wait complete. 2024-03-30T01:42:07.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.237+0000 7f18e9095700 1 Processor -- start 2024-03-30T01:42:07.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.237+0000 7f18e9095700 1 -- start start 2024-03-30T01:42:07.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e9095700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e406e980 0x7f18e41ea860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e9095700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e41edd60 0x7f18e406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e9095700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18e41e7030 0x7f18e41e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e9095700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e41eada0 con 0x7f18e41edd60 2024-03-30T01:42:07.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e9095700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e41566f0 con 0x7f18e41e7030 2024-03-30T01:42:07.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e9095700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e41ed580 con 0x7f18e406e980 2024-03-30T01:42:07.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e41edd60 0x7f18e406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e41edd60 0x7f18e406f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47182/0 (socket says 172.21.15.31:47182) 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e259c700 1 -- 172.21.15.31:0/1804339987 learned_addr learned my addr 172.21.15.31:0/1804339987 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e359e700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18e41e7030 0x7f18e41e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e2d9d700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e406e980 0x7f18e41ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e259c700 1 -- 172.21.15.31:0/1804339987 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e406e980 msgr2=0x7f18e41ea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e259c700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e406e980 0x7f18e41ea860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e259c700 1 -- 172.21.15.31:0/1804339987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18e41e7030 msgr2=0x7f18e41e74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e259c700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18e41e7030 0x7f18e41e74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e259c700 1 -- 172.21.15.31:0/1804339987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4156130 con 0x7f18e41edd60 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.238+0000 7f18e359e700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18e41e7030 0x7f18e41e74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.239+0000 7f18e259c700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e41edd60 0x7f18e406f950 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f18d8011aa0 tx=0x7f18d8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.239+0000 7f18cb7fe700 1 -- 172.21.15.31:0/1804339987 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8011ee0 con 0x7f18e41edd60 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.239+0000 7f18cb7fe700 1 -- 172.21.15.31:0/1804339987 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d800f940 con 0x7f18e41edd60 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.239+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e4156410 con 0x7f18e41edd60 2024-03-30T01:42:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.239+0000 7f18cb7fe700 1 -- 172.21.15.31:0/1804339987 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d800fcf0 con 0x7f18e41edd60 2024-03-30T01:42:07.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.239+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e406fe90 con 0x7f18e41edd60 2024-03-30T01:42:07.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.240+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e4073760 con 0x7f18e41edd60 2024-03-30T01:42:07.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.241+0000 7f18cb7fe700 1 -- 172.21.15.31:0/1804339987 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f18d8014440 con 0x7f18e41edd60 2024-03-30T01:42:07.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.241+0000 7f18cb7fe700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d006ac90 0x7f18d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.242+0000 7f18cb7fe700 1 -- 172.21.15.31:0/1804339987 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f18d8089c30 con 0x7f18e41edd60 2024-03-30T01:42:07.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.242+0000 7f18e2d9d700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d006ac90 0x7f18d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.242+0000 7f18e2d9d700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d006ac90 0x7f18d006d150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f18d4001cf0 tx=0x7f18d4007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:07.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.245+0000 7f18cb7fe700 1 -- 172.21.15.31:0/1804339987 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f18d8055150 con 0x7f18e41edd60 2024-03-30T01:42:07.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.462+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f18e404ea80 con 0x7f18e41edd60 2024-03-30T01:42:07.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.462+0000 7f18cb7fe700 1 -- 172.21.15.31:0/1804339987 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7f18d8022090 con 0x7f18e41edd60 2024-03-30T01:42:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:07.469 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-03-30T01:42:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-03-30T01:42:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-03-30T01:42:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:07.470 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-03-30T01:42:07.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d006ac90 msgr2=0x7f18d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d006ac90 0x7f18d006d150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f18d4001cf0 tx=0x7f18d4007980 comp rx=0 tx=0).stop 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e41edd60 msgr2=0x7f18e406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e41edd60 0x7f18e406f950 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f18d8011aa0 tx=0x7f18d8011db0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 shutdown_connections 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f18d006ac90 0x7f18d006d150 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18e41e7030 0x7f18e41e74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18e406e980 0x7f18e41ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 --2- 172.21.15.31:0/1804339987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e41edd60 0x7f18e406f950 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 >> 172.21.15.31:0/1804339987 conn(0x7f18e406ab20 msgr2=0x7f18e41520f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 shutdown_connections 2024-03-30T01:42:07.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.465+0000 7f18e9095700 1 -- 172.21.15.31:0/1804339987 wait complete. 2024-03-30T01:42:07.485 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T01:42:07.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T01:42:07.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.618+0000 7f61b80f2700 1 Processor -- start 2024-03-30T01:42:07.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.618+0000 7f61b80f2700 1 -- start start 2024-03-30T01:42:07.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.619+0000 7f61b80f2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61b00a72e0 0x7f61b01554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.619+0000 7f61b80f2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c4130 0x7f61b0155d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b80f2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b00c5260 con 0x7f61b00c2a40 2024-03-30T01:42:07.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b80f2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b0152350 con 0x7f61b00a72e0 2024-03-30T01:42:07.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b80f2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00c4650 con 0x7f61b00c4130 2024-03-30T01:42:07.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b5e8e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61b00a72e0 0x7f61b01554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b668f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c4130 0x7f61b0155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b5e8e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61b00a72e0 0x7f61b01554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:55898/0 (socket says 172.21.15.31:55898) 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b668f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c4130 0x7f61b0155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:44520/0 (socket says 172.21.15.31:44520) 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b5e8e700 1 -- 172.21.15.31:0/751357461 learned_addr learned my addr 172.21.15.31:0/751357461 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b568d700 1 --1- 172.21.15.31:0/751357461 >> v1:172.21.15.59:6789/0 conn(0x7f61b00c2a40 0x7f61b00c1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:55262/0 (socket says 172.21.15.31:55262) 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b5e8e700 1 -- 172.21.15.31:0/751357461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c4130 msgr2=0x7f61b0155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b5e8e700 1 --2- 172.21.15.31:0/751357461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c4130 0x7f61b0155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b5e8e700 1 -- 172.21.15.31:0/751357461 >> v1:172.21.15.59:6789/0 conn(0x7f61b00c2a40 legacy=0x7f61b00c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b5e8e700 1 -- 172.21.15.31:0/751357461 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b00c4930 con 0x7f61b00a72e0 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b4e8c700 1 -- 172.21.15.31:0/751357461 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938109130 0 0) 0x7f61b00c5260 con 0x7f61b00c2a40 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.620+0000 7f61b5e8e700 1 --2- 172.21.15.31:0/751357461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00a72e0 0x7f61b01554a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0098f0 tx=0x7f61ac009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7afdc6c46a13c8b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b4e8c700 1 -- 172.21.15.31:0/751357461 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac011830 con 0x7f61b00a72e0 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b4e8c700 1 -- 172.21.15.31:0/751357461 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac011990 con 0x7f61b00a72e0 2024-03-30T01:42:07.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b4e8c700 1 -- 172.21.15.31:0/751357461 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac011d20 con 0x7f61b00a72e0 2024-03-30T01:42:07.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b80f2700 1 -- 172.21.15.31:0/751357461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00a72e0 msgr2=0x7f61b01554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b80f2700 1 --2- 172.21.15.31:0/751357461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00a72e0 0x7f61b01554a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0098f0 tx=0x7f61ac009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:07.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b80f2700 1 -- 172.21.15.31:0/751357461 shutdown_connections 2024-03-30T01:42:07.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b80f2700 1 --2- 172.21.15.31:0/751357461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00a72e0 0x7f61b01554a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b80f2700 1 --2- 172.21.15.31:0/751357461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c4130 0x7f61b0155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b80f2700 1 -- 172.21.15.31:0/751357461 >> 172.21.15.31:0/751357461 conn(0x7f61b006a700 msgr2=0x7f61b006aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b80f2700 1 -- 172.21.15.31:0/751357461 shutdown_connections 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.621+0000 7f61b80f2700 1 -- 172.21.15.31:0/751357461 wait complete. 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.623+0000 7f61b80f2700 1 Processor -- start 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.623+0000 7f61b80f2700 1 -- start start 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.623+0000 7f61b80f2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00a72e0 0x7f61b0156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.623+0000 7f61b80f2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c2a40 0x7f61b00c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.623+0000 7f61b80f2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b00c4130 0x7f61b00c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.623+0000 7f61b80f2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b01ea390 con 0x7f61b00c4130 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.623+0000 7f61b80f2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b01df4f0 con 0x7f61b00a72e0 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.623+0000 7f61b80f2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b01df7d0 con 0x7f61b00c2a40 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b668f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b00c4130 0x7f61b00c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b668f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b00c4130 0x7f61b00c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47190/0 (socket says 172.21.15.31:47190) 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b668f700 1 -- 172.21.15.31:0/1341147570 learned_addr learned my addr 172.21.15.31:0/1341147570 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:07.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b5e8e700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00a72e0 0x7f61b0156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b568d700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c2a40 0x7f61b00c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b668f700 1 -- 172.21.15.31:0/1341147570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c2a40 msgr2=0x7f61b00c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b668f700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c2a40 0x7f61b00c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b668f700 1 -- 172.21.15.31:0/1341147570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00a72e0 msgr2=0x7f61b0156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b668f700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00a72e0 0x7f61b0156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b668f700 1 -- 172.21.15.31:0/1341147570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac009550 con 0x7f61b00c4130 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b668f700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b00c4130 0x7f61b00c69e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f61a800efc0 tx=0x7f61a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61a27fc700 1 -- 172.21.15.31:0/1341147570 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8009b20 con 0x7f61b00c4130 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61a27fc700 1 -- 172.21.15.31:0/1341147570 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a800f040 con 0x7f61b00c4130 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61a27fc700 1 -- 172.21.15.31:0/1341147570 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8009e50 con 0x7f61b00c4130 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b01e9d40 con 0x7f61b00c4130 2024-03-30T01:42:07.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.624+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b01e9a10 con 0x7f61b00c4130 2024-03-30T01:42:07.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.625+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b00c56d0 con 0x7f61b00c4130 2024-03-30T01:42:07.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.626+0000 7f61a27fc700 1 -- 172.21.15.31:0/1341147570 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f61a8014480 con 0x7f61b00c4130 2024-03-30T01:42:07.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.626+0000 7f61a27fc700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f619406ac90 0x7f619406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:07.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.626+0000 7f61a27fc700 1 -- 172.21.15.31:0/1341147570 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f61a8089bd0 con 0x7f61b00c4130 2024-03-30T01:42:07.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.626+0000 7f61b5e8e700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f619406ac90 0x7f619406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:07.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.627+0000 7f61b5e8e700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f619406ac90 0x7f619406d150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009c40 tx=0x7f61ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.631+0000 7f61a27fc700 1 -- 172.21.15.31:0/1341147570 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f61b00c56d0 con 0x7f61b00c4130 2024-03-30T01:42:07.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:07.848+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f61b01ea540 con 0x7f61b00c4130 2024-03-30T01:42:08.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.806+0000 7f61a27fc700 1 -- 172.21.15.31:0/1341147570 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f61b01ea540 con 0x7f61b00c4130 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.810+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f619406ac90 msgr2=0x7f619406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.810+0000 7f61b80f2700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f619406ac90 0x7f619406d150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009c40 tx=0x7f61ac008030 comp rx=0 tx=0).stop 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.810+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b00c4130 msgr2=0x7f61b00c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.810+0000 7f61b80f2700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b00c4130 0x7f61b00c69e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f61a800efc0 tx=0x7f61a800c3a0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.810+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 shutdown_connections 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.811+0000 7f61b80f2700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61b00c4130 0x7f61b00c69e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.811+0000 7f61b80f2700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f619406ac90 0x7f619406d150 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.811+0000 7f61b80f2700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61b00a72e0 0x7f61b0156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.811+0000 7f61b80f2700 1 --2- 172.21.15.31:0/1341147570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61b00c2a40 0x7f61b00c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.811+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 >> 172.21.15.31:0/1341147570 conn(0x7f61b006a700 msgr2=0x7f61b0151370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:08.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.811+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 shutdown_connections 2024-03-30T01:42:08.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.811+0000 7f61b80f2700 1 -- 172.21.15.31:0/1341147570 wait complete. 2024-03-30T01:42:08.818 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-03-30T01:42:08.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T01:42:08.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.976+0000 7f82a1caf700 1 Processor -- start 2024-03-30T01:42:08.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.976+0000 7f82a1caf700 1 -- start start 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.977+0000 7f82a1caf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c06f310 0x7f829c06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.977+0000 7f82a1caf700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f829c0c2670 0x7f829c156620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.977+0000 7f82a1caf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829c0a8460 con 0x7f829c0c18f0 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.977+0000 7f82a1caf700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829c156b60 con 0x7f829c0c2670 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.977+0000 7f82a1caf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829c1526a0 con 0x7f829c06f310 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.977+0000 7f829bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f829c0c2670 0x7f829c156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.977+0000 7f829bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f829c0c2670 0x7f829c156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47194/0 (socket says 172.21.15.31:47194) 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.977+0000 7f829bfff700 1 -- 172.21.15.31:0/1173347675 learned_addr learned my addr 172.21.15.31:0/1173347675 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.977+0000 7f829b7fe700 1 --2- 172.21.15.31:0/1173347675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c06f310 0x7f829c06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.978+0000 7f829bfff700 1 -- 172.21.15.31:0/1173347675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c06f310 msgr2=0x7f829c06f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.978+0000 7f829bfff700 1 --2- 172.21.15.31:0/1173347675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c06f310 0x7f829c06f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.978+0000 7f829bfff700 1 -- 172.21.15.31:0/1173347675 >> v1:172.21.15.59:6789/0 conn(0x7f829c0c18f0 legacy=0x7f829c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.978+0000 7f829bfff700 1 -- 172.21.15.31:0/1173347675 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c0a86b0 con 0x7f829c0c2670 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.978+0000 7f829b7fe700 1 --2- 172.21.15.31:0/1173347675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c06f310 0x7f829c06f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.978+0000 7f829bfff700 1 --2- 172.21.15.31:0/1173347675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 0x7f829c156620 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8294009e50 tx=0x7f829400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11714932036820b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.978+0000 7f829a7fc700 1 -- 172.21.15.31:0/1173347675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829400e070 con 0x7f829c0c2670 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.978+0000 7f829a7fc700 1 -- 172.21.15.31:0/1173347675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829400b9c0 con 0x7f829c0c2670 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.978+0000 7f829a7fc700 1 -- 172.21.15.31:0/1173347675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829400bd70 con 0x7f829c0c2670 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.979+0000 7f82a1caf700 1 -- 172.21.15.31:0/1173347675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 msgr2=0x7f829c156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.979+0000 7f82a1caf700 1 --2- 172.21.15.31:0/1173347675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 0x7f829c156620 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8294009e50 tx=0x7f829400b330 comp rx=0 tx=0).stop 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.979+0000 7f82a1caf700 1 -- 172.21.15.31:0/1173347675 shutdown_connections 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.979+0000 7f82a1caf700 1 --2- 172.21.15.31:0/1173347675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 0x7f829c156620 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.979+0000 7f82a1caf700 1 --2- 172.21.15.31:0/1173347675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c06f310 0x7f829c06f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.979+0000 7f82a1caf700 1 -- 172.21.15.31:0/1173347675 >> 172.21.15.31:0/1173347675 conn(0x7f829c0c3980 msgr2=0x7f829c0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.979+0000 7f82a1caf700 1 -- 172.21.15.31:0/1173347675 shutdown_connections 2024-03-30T01:42:08.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.979+0000 7f82a1caf700 1 -- 172.21.15.31:0/1173347675 wait complete. 2024-03-30T01:42:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.980+0000 7f82a1caf700 1 Processor -- start 2024-03-30T01:42:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.980+0000 7f82a1caf700 1 -- start start 2024-03-30T01:42:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f82a1caf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c06f310 0x7f829c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f82a1caf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c0c18f0 0x7f829c1de6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f82a1caf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 0x7f829c0ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f82a1caf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c12d840 con 0x7f829c0c2670 2024-03-30T01:42:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f82a1caf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c06b0e0 con 0x7f829c06f310 2024-03-30T01:42:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f82a1caf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829c1debe0 con 0x7f829c0c18f0 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 0x7f829c0ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 0x7f829c0ab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47206/0 (socket says 172.21.15.31:47206) 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829bfff700 1 -- 172.21.15.31:0/1558924701 learned_addr learned my addr 172.21.15.31:0/1558924701 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829b7fe700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c06f310 0x7f829c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829affd700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c0c18f0 0x7f829c1de6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829bfff700 1 -- 172.21.15.31:0/1558924701 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c0c18f0 msgr2=0x7f829c1de6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829bfff700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c0c18f0 0x7f829c1de6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829bfff700 1 -- 172.21.15.31:0/1558924701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c06f310 msgr2=0x7f829c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829bfff700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c06f310 0x7f829c06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.981+0000 7f829bfff700 1 -- 172.21.15.31:0/1558924701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294009b30 con 0x7f829c0c2670 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.982+0000 7f829bfff700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 0x7f829c0ab880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f829400b730 tx=0x7f8294004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.982+0000 7f8293fff700 1 -- 172.21.15.31:0/1558924701 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82940099b0 con 0x7f829c0c2670 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.982+0000 7f8293fff700 1 -- 172.21.15.31:0/1558924701 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8294004250 con 0x7f829c0c2670 2024-03-30T01:42:08.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.982+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c133a50 con 0x7f829c0c2670 2024-03-30T01:42:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.982+0000 7f8293fff700 1 -- 172.21.15.31:0/1558924701 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82940045e0 con 0x7f829c0c2670 2024-03-30T01:42:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.982+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c134010 con 0x7f829c0c2670 2024-03-30T01:42:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.983+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829c0747d0 con 0x7f829c0c2670 2024-03-30T01:42:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.984+0000 7f8293fff700 1 -- 172.21.15.31:0/1558924701 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8294004740 con 0x7f829c0c2670 2024-03-30T01:42:08.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.984+0000 7f8293fff700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f827c06ac90 0x7f827c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:08.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.984+0000 7f8293fff700 1 -- 172.21.15.31:0/1558924701 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f829408aa00 con 0x7f829c0c2670 2024-03-30T01:42:08.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.984+0000 7f829b7fe700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f827c06ac90 0x7f827c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:08.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.985+0000 7f829b7fe700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f827c06ac90 0x7f827c06d150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f828c006f10 tx=0x7f828c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:08.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:08.989+0000 7f8293fff700 1 -- 172.21.15.31:0/1558924701 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8294055e30 con 0x7f829c0c2670 2024-03-30T01:42:09.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.204+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f829c0c2220 con 0x7f829c0c2670 2024-03-30T01:42:09.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.808+0000 7f8293fff700 1 -- 172.21.15.31:0/1558924701 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8294059450 con 0x7f829c0c2670 2024-03-30T01:42:09.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.817+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f827c06ac90 msgr2=0x7f827c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:09.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.817+0000 7f82a1caf700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f827c06ac90 0x7f827c06d150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f828c006f10 tx=0x7f828c008040 comp rx=0 tx=0).stop 2024-03-30T01:42:09.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.817+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 msgr2=0x7f829c0ab880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:09.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.817+0000 7f82a1caf700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 0x7f829c0ab880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f829400b730 tx=0x7f8294004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:09.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.818+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 shutdown_connections 2024-03-30T01:42:09.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.818+0000 7f82a1caf700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f829c0c2670 0x7f829c0ab880 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:09.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.818+0000 7f82a1caf700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f827c06ac90 0x7f827c06d150 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.818+0000 7f82a1caf700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c06f310 0x7f829c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.818+0000 7f82a1caf700 1 --2- 172.21.15.31:0/1558924701 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f829c0c18f0 0x7f829c1de6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.818+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 >> 172.21.15.31:0/1558924701 conn(0x7f829c0c3980 msgr2=0x7f829c136260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.818+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 shutdown_connections 2024-03-30T01:42:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.819+0000 7f82a1caf700 1 -- 172.21.15.31:0/1558924701 wait complete. 2024-03-30T01:42:09.828 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-03-30T01:42:09.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T01:42:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.986+0000 7fd9bf02f700 1 Processor -- start 2024-03-30T01:42:09.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.986+0000 7fd9bf02f700 1 -- start start 2024-03-30T01:42:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.987+0000 7fd9bf02f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9b80bc1e0 0x7fd9b806c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.987+0000 7fd9bf02f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 0x7fd9b80c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.987+0000 7fd9bf02f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b80c4080 con 0x7fd9b8072a30 2024-03-30T01:42:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.987+0000 7fd9bf02f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b80c3d20 con 0x7fd9b80bc1e0 2024-03-30T01:42:09.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.987+0000 7fd9bf02f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80bc5c0 con 0x7fd9b806c550 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.987+0000 7fd9bcdcb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9b80bc1e0 0x7fd9b806c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.987+0000 7fd9bcdcb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9b80bc1e0 0x7fd9b806c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47210/0 (socket says 172.21.15.31:47210) 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.987+0000 7fd9bcdcb700 1 -- 172.21.15.31:0/2092803225 learned_addr learned my addr 172.21.15.31:0/2092803225 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.987+0000 7fd9bd5cc700 1 --2- 172.21.15.31:0/2092803225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 0x7fd9b80c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.988+0000 7fd9bcdcb700 1 -- 172.21.15.31:0/2092803225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 msgr2=0x7fd9b80c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.988+0000 7fd9bcdcb700 1 --2- 172.21.15.31:0/2092803225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 0x7fd9b80c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.988+0000 7fd9bcdcb700 1 -- 172.21.15.31:0/2092803225 >> v1:172.21.15.59:6789/0 conn(0x7fd9b8072a30 legacy=0x7fd9b80c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.988+0000 7fd9bcdcb700 1 -- 172.21.15.31:0/2092803225 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8154460 con 0x7fd9b80bc1e0 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.988+0000 7fd9bcdcb700 1 --2- 172.21.15.31:0/2092803225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 0x7fd9b806c010 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800bb90 tx=0x7fd9a800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcdda8d6858eddc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.988+0000 7fd9b77fe700 1 -- 172.21.15.31:0/2092803225 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800dd30 con 0x7fd9b80bc1e0 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.988+0000 7fd9b77fe700 1 -- 172.21.15.31:0/2092803225 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a800de90 con 0x7fd9b80bc1e0 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.988+0000 7fd9b77fe700 1 -- 172.21.15.31:0/2092803225 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8010690 con 0x7fd9b80bc1e0 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.989+0000 7fd9bf02f700 1 -- 172.21.15.31:0/2092803225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 msgr2=0x7fd9b806c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.989+0000 7fd9bf02f700 1 --2- 172.21.15.31:0/2092803225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 0x7fd9b806c010 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800bb90 tx=0x7fd9a800bea0 comp rx=0 tx=0).stop 2024-03-30T01:42:09.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.989+0000 7fd9bf02f700 1 -- 172.21.15.31:0/2092803225 shutdown_connections 2024-03-30T01:42:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.989+0000 7fd9bf02f700 1 --2- 172.21.15.31:0/2092803225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 0x7fd9b806c010 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.989+0000 7fd9bf02f700 1 --2- 172.21.15.31:0/2092803225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 0x7fd9b80c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.989+0000 7fd9bf02f700 1 -- 172.21.15.31:0/2092803225 >> 172.21.15.31:0/2092803225 conn(0x7fd9b806b110 msgr2=0x7fd9b80a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.989+0000 7fd9bf02f700 1 -- 172.21.15.31:0/2092803225 shutdown_connections 2024-03-30T01:42:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.989+0000 7fd9bf02f700 1 -- 172.21.15.31:0/2092803225 wait complete. 2024-03-30T01:42:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.990+0000 7fd9bf02f700 1 Processor -- start 2024-03-30T01:42:09.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.991+0000 7fd9bf02f700 1 -- start start 2024-03-30T01:42:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bf02f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 0x7fd9b814d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bf02f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9b8072a30 0x7fd9b80a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bf02f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 0x7fd9b80aa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bf02f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80a7880 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bf02f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80a6490 con 0x7fd9b8072a30 2024-03-30T01:42:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bf02f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80a6770 con 0x7fd9b806c550 2024-03-30T01:42:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bd5cc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 0x7fd9b80aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bd5cc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 0x7fd9b80aa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47222/0 (socket says 172.21.15.31:47222) 2024-03-30T01:42:10.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bd5cc700 1 -- 172.21.15.31:0/3876790119 learned_addr learned my addr 172.21.15.31:0/3876790119 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bcdcb700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 0x7fd9b814d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9b7fff700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9b8072a30 0x7fd9b80a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bd5cc700 1 -- 172.21.15.31:0/3876790119 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 msgr2=0x7fd9b814d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bd5cc700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 0x7fd9b814d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bd5cc700 1 -- 172.21.15.31:0/3876790119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9b8072a30 msgr2=0x7fd9b80a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bd5cc700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9b8072a30 0x7fd9b80a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bd5cc700 1 -- 172.21.15.31:0/3876790119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a800b7f0 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9bd5cc700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 0x7fd9b80aa210 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000efc0 tx=0x7fd9b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9b57fa700 1 -- 172.21.15.31:0/3876790119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0009ba0 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.992+0000 7fd9b57fa700 1 -- 172.21.15.31:0/3876790119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b000f040 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.993+0000 7fd9b57fa700 1 -- 172.21.15.31:0/3876790119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0014480 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.993+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b80a6a50 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.993+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b80a5f20 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.994+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b814ceb0 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.994+0000 7fd9b57fa700 1 -- 172.21.15.31:0/3876790119 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd9b00145e0 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.995+0000 7fd9b57fa700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd99806ac90 0x7fd99806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.995+0000 7fd9b57fa700 1 -- 172.21.15.31:0/3876790119 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fd9b008a3b0 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.995+0000 7fd9bcdcb700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd99806ac90 0x7fd99806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.995+0000 7fd9bcdcb700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd99806ac90 0x7fd99806d150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800bee0 tx=0x7fd9a8013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:10.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:09.999+0000 7fd9b57fa700 1 -- 172.21.15.31:0/3876790119 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd9b0055590 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.262+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd9b814ceb0 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.271+0000 7fd9b57fa700 1 -- 172.21.15.31:0/3876790119 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd9b814ceb0 con 0x7fd9b80bc1e0 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd99806ac90 msgr2=0x7fd99806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd99806ac90 0x7fd99806d150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800bee0 tx=0x7fd9a8013000 comp rx=0 tx=0).stop 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 msgr2=0x7fd9b80aa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 0x7fd9b80aa210 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000efc0 tx=0x7fd9b000c3a0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 shutdown_connections 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b80bc1e0 0x7fd9b80aa210 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd99806ac90 0x7fd99806d150 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800bee0 tx=0x7fd9a8013000 comp rx=0 tx=0).stop 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd9b806c550 0x7fd9b814d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 --2- 172.21.15.31:0/3876790119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9b8072a30 0x7fd9b80a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.278+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 >> 172.21.15.31:0/3876790119 conn(0x7fd9b806b110 msgr2=0x7fd9b81526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.281+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 shutdown_connections 2024-03-30T01:42:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.281+0000 7fd9bf02f700 1 -- 172.21.15.31:0/3876790119 wait complete. 2024-03-30T01:42:10.300 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 5 and data pool 6 2024-03-30T01:42:10.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:10.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.445+0000 7f214e738700 1 Processor -- start 2024-03-30T01:42:10.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.445+0000 7f214e738700 1 -- start start 2024-03-30T01:42:10.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.446+0000 7f214e738700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2148071240 0x7f2148154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.446+0000 7f214e738700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2148072160 con 0x7f214814e5f0 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.446+0000 7f214e738700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21480c2bd0 con 0x7f2148071750 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.446+0000 7f214e738700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21480c2e90 con 0x7f2148071240 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.446+0000 7f2147fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f214814e5f0 0x7f21480c3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41646/0 (socket says 172.21.15.31:41646) 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.446+0000 7f2147fff700 1 -- 172.21.15.31:0/4066078245 learned_addr learned my addr 172.21.15.31:0/4066078245 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f21477fe700 1 --2- 172.21.15.31:0/4066078245 >> v2:172.21.15.59:3300/0 conn(0x7f2148071240 0x7f2148154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809028994 0 0) 0x7f2148072160 con 0x7f214814e5f0 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21300035b0 con 0x7f214814e5f0 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2310725273 0 0) 0x7f21300035b0 con 0x7f214814e5f0 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 >> v2:172.21.15.59:3300/0 conn(0x7f2148071240 msgr2=0x7f2148154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 --2- 172.21.15.31:0/4066078245 >> v2:172.21.15.59:3300/0 conn(0x7f2148071240 0x7f2148154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 >> v1:172.21.15.59:6789/0 conn(0x7f2148071750 legacy=0x7f2148154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21480723b0 con 0x7f214814e5f0 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f2138001670 con 0x7f214814e5f0 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 >> v1:172.21.15.31:6789/0 conn(0x7f214814e5f0 legacy=0x7f21480c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2130003a00 0x7f2130003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2130004850 0x7f2130004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2130005330 0x7f213000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21480c2e90 con 0x7f2130003a00 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213000c200 con 0x7f2130004850 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213000c3c0 con 0x7f2130005330 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f2147fff700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2130003a00 0x7f2130003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.447+0000 7f21477fe700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2130004850 0x7f2130004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f214ccd5700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2130005330 0x7f213000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f2147fff700 1 -- 172.21.15.31:0/4066078245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2130005330 msgr2=0x7f213000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f2147fff700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2130005330 0x7f213000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f2147fff700 1 -- 172.21.15.31:0/4066078245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2130004850 msgr2=0x7f2130004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f2147fff700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2130004850 0x7f2130004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f2147fff700 1 -- 172.21.15.31:0/4066078245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21300035b0 con 0x7f2130003a00 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f214ccd5700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2130005330 0x7f213000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f214ccd5700 1 -- 172.21.15.31:0/4066078245 reap_dead start 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f2147fff700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2130003a00 0x7f2130003e20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f213800c210 tx=0x7f2130005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05b8e52b44f139e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2138007c10 con 0x7f2130003a00 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213800cdb0 con 0x7f2130003a00 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.448+0000 7f2146ffd700 1 -- 172.21.15.31:0/4066078245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2138005ba0 con 0x7f2130003a00 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.449+0000 7f214e738700 1 -- 172.21.15.31:0/4066078245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2130003a00 msgr2=0x7f2130003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.449+0000 7f214e738700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2130003a00 0x7f2130003e20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f213800c210 tx=0x7f2130005880 comp rx=0 tx=0).stop 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.449+0000 7f214e738700 1 -- 172.21.15.31:0/4066078245 shutdown_connections 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.449+0000 7f214e738700 1 --2- 172.21.15.31:0/4066078245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2130003a00 0x7f2130003e20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.449+0000 7f214e738700 1 -- 172.21.15.31:0/4066078245 >> 172.21.15.31:0/4066078245 conn(0x7f214806ba20 msgr2=0x7f214806be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.449+0000 7f214e738700 1 -- 172.21.15.31:0/4066078245 shutdown_connections 2024-03-30T01:42:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.449+0000 7f214e738700 1 -- 172.21.15.31:0/4066078245 wait complete. 2024-03-30T01:42:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.450+0000 7f214e738700 1 Processor -- start 2024-03-30T01:42:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.450+0000 7f214e738700 1 -- start start 2024-03-30T01:42:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f214e738700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2130003a00 0x7f21481ed000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f214e738700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2148148850 0x7f21481ed540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f214e738700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2148149140 0x7f2148143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f214e738700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21481eac90 con 0x7f2148148850 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f214e738700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21481edb60 con 0x7f2130003a00 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f214e738700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21481eaf30 con 0x7f2148149140 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f21477fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2148148850 0x7f21481ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f21477fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2148148850 0x7f21481ed540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47240/0 (socket says 172.21.15.31:47240) 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f21477fe700 1 -- 172.21.15.31:0/630834280 learned_addr learned my addr 172.21.15.31:0/630834280 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f2147fff700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2130003a00 0x7f21481ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f214ccd5700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2148149140 0x7f2148143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f21477fe700 1 -- 172.21.15.31:0/630834280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2148149140 msgr2=0x7f2148143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f21477fe700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2148149140 0x7f2148143710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f21477fe700 1 -- 172.21.15.31:0/630834280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2130003a00 msgr2=0x7f21481ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f21477fe700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2130003a00 0x7f21481ed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.451+0000 7f21477fe700 1 -- 172.21.15.31:0/630834280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21481eb210 con 0x7f2148148850 2024-03-30T01:42:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.452+0000 7f21477fe700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2148148850 0x7f21481ed540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f213c011aa0 tx=0x7f213c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.452+0000 7f2144ff9700 1 -- 172.21.15.31:0/630834280 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00f940 con 0x7f2148148850 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.452+0000 7f2144ff9700 1 -- 172.21.15.31:0/630834280 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c00faa0 con 0x7f2148148850 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.452+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21481eb4f0 con 0x7f2148148850 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.452+0000 7f2144ff9700 1 -- 172.21.15.31:0/630834280 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00fe50 con 0x7f2148148850 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.452+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2148143e30 con 0x7f2148148850 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.453+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214804ea80 con 0x7f2148148850 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.454+0000 7f2144ff9700 1 -- 172.21.15.31:0/630834280 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f213c01d400 con 0x7f2148148850 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.454+0000 7f2144ff9700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f213406acc0 0x7f213406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.455+0000 7f2144ff9700 1 -- 172.21.15.31:0/630834280 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f213c089f70 con 0x7f2148148850 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.455+0000 7f2147fff700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f213406acc0 0x7f213406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.455+0000 7f2147fff700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f213406acc0 0x7f213406d180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2138000c00 tx=0x7f2138007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:10.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.458+0000 7f2144ff9700 1 -- 172.21.15.31:0/630834280 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f213c054f60 con 0x7f2148148850 2024-03-30T01:42:10.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.673+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21480c1140 con 0x7f2148148850 2024-03-30T01:42:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.674+0000 7f2144ff9700 1 -- 172.21.15.31:0/630834280 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f213c058580 con 0x7f2148148850 2024-03-30T01:42:10.680 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:10.681 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:10.264368+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f213406acc0 msgr2=0x7f213406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f213406acc0 0x7f213406d180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2138000c00 tx=0x7f2138007000 comp rx=0 tx=0).stop 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2148148850 msgr2=0x7f21481ed540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2148148850 0x7f21481ed540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f213c011aa0 tx=0x7f213c011db0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 shutdown_connections 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f213406acc0 0x7f213406d180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2148149140 0x7f2148143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2130003a00 0x7f21481ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 --2- 172.21.15.31:0/630834280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2148148850 0x7f21481ed540 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 >> 172.21.15.31:0/630834280 conn(0x7f214806ba20 msgr2=0x7f214814d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 shutdown_connections 2024-03-30T01:42:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.676+0000 7f214e738700 1 -- 172.21.15.31:0/630834280 wait complete. 2024-03-30T01:42:10.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:10.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.827+0000 7f2055adb700 1 Processor -- start 2024-03-30T01:42:10.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.827+0000 7f2055adb700 1 -- start start 2024-03-30T01:42:10.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.827+0000 7f2055adb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f20500c5df0 0x7f20500c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.827+0000 7f2055adb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c2870 0x7f20500a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.827+0000 7f2055adb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20501556d0 con 0x7f20500c8310 2024-03-30T01:42:10.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.827+0000 7f2055adb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20501559b0 con 0x7f20500c5df0 2024-03-30T01:42:10.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.827+0000 7f2055adb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20500c1f70 con 0x7f20500c2870 2024-03-30T01:42:10.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c2870 0x7f20500a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204effd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f20500c5df0 0x7f20500c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204f7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f20500c8310 0x7f2050151d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:55302/0 (socket says 172.21.15.31:55302) 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204effd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f20500c5df0 0x7f20500c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:55946/0 (socket says 172.21.15.31:55946) 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c2870 0x7f20500a6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:44614/0 (socket says 172.21.15.31:44614) 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204f7fe700 1 -- 172.21.15.31:0/2602187615 learned_addr learned my addr 172.21.15.31:0/2602187615 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204effd700 1 -- 172.21.15.31:0/2602187615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c2870 msgr2=0x7f20500a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204effd700 1 --2- 172.21.15.31:0/2602187615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c2870 0x7f20500a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204effd700 1 -- 172.21.15.31:0/2602187615 >> v1:172.21.15.59:6789/0 conn(0x7f20500c8310 legacy=0x7f2050151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204effd700 1 -- 172.21.15.31:0/2602187615 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500c2250 con 0x7f20500c5df0 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.828+0000 7f204effd700 1 --2- 172.21.15.31:0/2602187615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500c5df0 0x7f20500c86f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2044009870 tx=0x7f2044009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10df26a51185baf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f204e7fc700 1 -- 172.21.15.31:0/2602187615 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044011830 con 0x7f20500c5df0 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f204e7fc700 1 -- 172.21.15.31:0/2602187615 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2044011990 con 0x7f20500c5df0 2024-03-30T01:42:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f204e7fc700 1 -- 172.21.15.31:0/2602187615 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044011d20 con 0x7f20500c5df0 2024-03-30T01:42:10.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f2055adb700 1 -- 172.21.15.31:0/2602187615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500c5df0 msgr2=0x7f20500c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f2055adb700 1 --2- 172.21.15.31:0/2602187615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500c5df0 0x7f20500c86f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2044009870 tx=0x7f2044009b80 comp rx=0 tx=0).stop 2024-03-30T01:42:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f2055adb700 1 -- 172.21.15.31:0/2602187615 shutdown_connections 2024-03-30T01:42:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f2055adb700 1 --2- 172.21.15.31:0/2602187615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c2870 0x7f20500a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f2055adb700 1 --2- 172.21.15.31:0/2602187615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500c5df0 0x7f20500c86f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f2055adb700 1 -- 172.21.15.31:0/2602187615 >> 172.21.15.31:0/2602187615 conn(0x7f205006ab20 msgr2=0x7f205006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f2055adb700 1 -- 172.21.15.31:0/2602187615 shutdown_connections 2024-03-30T01:42:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.829+0000 7f2055adb700 1 -- 172.21.15.31:0/2602187615 wait complete. 2024-03-30T01:42:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.831+0000 7f2055adb700 1 Processor -- start 2024-03-30T01:42:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.831+0000 7f2055adb700 1 -- start start 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2055adb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500c2870 0x7f20500ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2055adb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500c5df0 0x7f205014e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2055adb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c8310 0x7f205014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2055adb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050156d50 con 0x7f20500c2870 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2055adb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205014ed10 con 0x7f20500c5df0 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2055adb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20501ed3f0 con 0x7f20500c8310 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500c2870 0x7f20500ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500c2870 0x7f20500ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47248/0 (socket says 172.21.15.31:47248) 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204f7fe700 1 -- 172.21.15.31:0/4101522497 learned_addr learned my addr 172.21.15.31:0/4101522497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204effd700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500c5df0 0x7f205014e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204ffff700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c8310 0x7f205014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204f7fe700 1 -- 172.21.15.31:0/4101522497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c8310 msgr2=0x7f205014e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204f7fe700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c8310 0x7f205014e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204f7fe700 1 -- 172.21.15.31:0/4101522497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500c5df0 msgr2=0x7f205014e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204f7fe700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500c5df0 0x7f205014e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204f7fe700 1 -- 172.21.15.31:0/4101522497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20440094a0 con 0x7f20500c2870 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f204f7fe700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500c2870 0x7f20500ac140 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f204000c480 tx=0x7f204000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2037fff700 1 -- 172.21.15.31:0/4101522497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040005d70 con 0x7f20500c2870 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2037fff700 1 -- 172.21.15.31:0/4101522497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20400070d0 con 0x7f20500c2870 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20501ed6d0 con 0x7f20500c2870 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.832+0000 7f2037fff700 1 -- 172.21.15.31:0/4101522497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040007440 con 0x7f20500c2870 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.833+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501edc90 con 0x7f20500c2870 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.833+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2050073760 con 0x7f20500c2870 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.834+0000 7f2037fff700 1 -- 172.21.15.31:0/4101522497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f20400075a0 con 0x7f20500c2870 2024-03-30T01:42:10.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.834+0000 7f2037fff700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f203c06ac90 0x7f203c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.835+0000 7f2037fff700 1 -- 172.21.15.31:0/4101522497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2040088660 con 0x7f20500c2870 2024-03-30T01:42:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.835+0000 7f204effd700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f203c06ac90 0x7f203c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:10.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.835+0000 7f204effd700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f203c06ac90 0x7f203c06d150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20440090b0 tx=0x7f2044008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:10.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:10.839+0000 7f2037fff700 1 -- 172.21.15.31:0/4101522497 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2040053820 con 0x7f20500c2870 2024-03-30T01:42:11.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.055+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f205004ea80 con 0x7f20500c2870 2024-03-30T01:42:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.056+0000 7f2037fff700 1 -- 172.21.15.31:0/4101522497 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f2040056e40 con 0x7f20500c2870 2024-03-30T01:42:11.063 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:11.063 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:10.264368+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f203c06ac90 msgr2=0x7f203c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f203c06ac90 0x7f203c06d150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20440090b0 tx=0x7f2044008030 comp rx=0 tx=0).stop 2024-03-30T01:42:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500c2870 msgr2=0x7f20500ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500c2870 0x7f20500ac140 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f204000c480 tx=0x7f204000c790 comp rx=0 tx=0).stop 2024-03-30T01:42:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 shutdown_connections 2024-03-30T01:42:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20500c2870 0x7f20500ac140 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f203c06ac90 0x7f203c06d150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20500c8310 0x7f205014e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 --2- 172.21.15.31:0/4101522497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20500c5df0 0x7f205014e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 >> 172.21.15.31:0/4101522497 conn(0x7f205006ab20 msgr2=0x7f2050153310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 shutdown_connections 2024-03-30T01:42:11.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.059+0000 7f2055adb700 1 -- 172.21.15.31:0/4101522497 wait complete. 2024-03-30T01:42:11.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T01:42:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.212+0000 7f30bdc44700 1 Processor -- start 2024-03-30T01:42:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.212+0000 7f30bdc44700 1 -- start start 2024-03-30T01:42:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30bdc44700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b8151d80 0x7f30b80ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30bdc44700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30b80c7f10 0x7f30b80c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30bdc44700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30b80ca890 0x7f30b80c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30bdc44700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b806cb20 con 0x7f30b80ca890 2024-03-30T01:42:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30bdc44700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b8155240 con 0x7f30b80c7f10 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30bdc44700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b81556d0 con 0x7f30b8151d80 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30b7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30b80ca890 0x7f30b80c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30b7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30b80ca890 0x7f30b80c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47264/0 (socket says 172.21.15.31:47264) 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30b7fff700 1 -- 172.21.15.31:0/2919997870 learned_addr learned my addr 172.21.15.31:0/2919997870 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30b6ffd700 1 --2- 172.21.15.31:0/2919997870 >> v2:172.21.15.59:3300/0 conn(0x7f30b80c7f10 0x7f30b80c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.213+0000 7f30b77fe700 1 --2- 172.21.15.31:0/2919997870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b8151d80 0x7f30b80ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.214+0000 7f30b7fff700 1 -- 172.21.15.31:0/2919997870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b8151d80 msgr2=0x7f30b80ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.214+0000 7f30b7fff700 1 --2- 172.21.15.31:0/2919997870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b8151d80 0x7f30b80ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.214+0000 7f30b7fff700 1 -- 172.21.15.31:0/2919997870 >> v2:172.21.15.59:3300/0 conn(0x7f30b80c7f10 msgr2=0x7f30b80c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.214+0000 7f30b7fff700 1 --2- 172.21.15.31:0/2919997870 >> v2:172.21.15.59:3300/0 conn(0x7f30b80c7f10 0x7f30b80c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.214+0000 7f30b7fff700 1 -- 172.21.15.31:0/2919997870 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80c74a0 con 0x7f30b80ca890 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.214+0000 7f30b7fff700 1 --2- 172.21.15.31:0/2919997870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b80ca890 0x7f30b80c6f60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f30b80c7af0 tx=0x7f30b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8149be561bede90 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.214+0000 7f30b67fc700 1 -- 172.21.15.31:0/2919997870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b000e070 con 0x7f30b80ca890 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.214+0000 7f30b67fc700 1 -- 172.21.15.31:0/2919997870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b000b910 con 0x7f30b80ca890 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.214+0000 7f30b67fc700 1 -- 172.21.15.31:0/2919997870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b000bcc0 con 0x7f30b80ca890 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.215+0000 7f30bdc44700 1 -- 172.21.15.31:0/2919997870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b80ca890 msgr2=0x7f30b80c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.215+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2919997870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b80ca890 0x7f30b80c6f60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f30b80c7af0 tx=0x7f30b000b330 comp rx=0 tx=0).stop 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.215+0000 7f30bdc44700 1 -- 172.21.15.31:0/2919997870 shutdown_connections 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.215+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2919997870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b80ca890 0x7f30b80c6f60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.215+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2919997870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b8151d80 0x7f30b80ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.215+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2919997870 >> v2:172.21.15.59:3300/0 conn(0x7f30b80c7f10 0x7f30b80c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.215+0000 7f30bdc44700 1 -- 172.21.15.31:0/2919997870 >> 172.21.15.31:0/2919997870 conn(0x7f30b8072010 msgr2=0x7f30b8072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.215+0000 7f30bdc44700 1 -- 172.21.15.31:0/2919997870 shutdown_connections 2024-03-30T01:42:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.215+0000 7f30bdc44700 1 -- 172.21.15.31:0/2919997870 wait complete. 2024-03-30T01:42:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.216+0000 7f30bdc44700 1 Processor -- start 2024-03-30T01:42:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.216+0000 7f30bdc44700 1 -- start start 2024-03-30T01:42:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30bdc44700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30b80c7f10 0x7f30b80ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30bdc44700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b80ca890 0x7f30b8153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30bdc44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b8151d80 0x7f30b8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30bdc44700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80ab7d0 con 0x7f30b8151d80 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30bdc44700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80a99d0 con 0x7f30b80c7f10 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30bdc44700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8155ba0 con 0x7f30b80ca890 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b8151d80 0x7f30b8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b8151d80 0x7f30b8156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47274/0 (socket says 172.21.15.31:47274) 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b7fff700 1 -- 172.21.15.31:0/2690185008 learned_addr learned my addr 172.21.15.31:0/2690185008 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b77fe700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30b80c7f10 0x7f30b80ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b6ffd700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b80ca890 0x7f30b8153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b7fff700 1 -- 172.21.15.31:0/2690185008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b80ca890 msgr2=0x7f30b8153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b7fff700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b80ca890 0x7f30b8153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b7fff700 1 -- 172.21.15.31:0/2690185008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30b80c7f10 msgr2=0x7f30b80ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:11.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b7fff700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30b80c7f10 0x7f30b80ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b7fff700 1 -- 172.21.15.31:0/2690185008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b0009b30 con 0x7f30b8151d80 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b6ffd700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b80ca890 0x7f30b8153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.217+0000 7f30b7fff700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b8151d80 0x7f30b8156010 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f30b80cafe0 tx=0x7f30b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.218+0000 7f309ffff700 1 -- 172.21.15.31:0/2690185008 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b00092e0 con 0x7f30b8151d80 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.218+0000 7f309ffff700 1 -- 172.21.15.31:0/2690185008 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b000ce80 con 0x7f30b8151d80 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.218+0000 7f309ffff700 1 -- 172.21.15.31:0/2690185008 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b0004490 con 0x7f30b8151d80 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.218+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b81481e0 con 0x7f30b8151d80 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.218+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b8157590 con 0x7f30b8151d80 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.219+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b8155ce0 con 0x7f30b8151d80 2024-03-30T01:42:11.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.219+0000 7f309ffff700 1 -- 172.21.15.31:0/2690185008 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f30b00045f0 con 0x7f30b8151d80 2024-03-30T01:42:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.220+0000 7f309ffff700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30a406ac90 0x7f30a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.220+0000 7f309ffff700 1 -- 172.21.15.31:0/2690185008 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f30b008aad0 con 0x7f30b8151d80 2024-03-30T01:42:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.220+0000 7f30b77fe700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30a406ac90 0x7f30a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.220+0000 7f30b77fe700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30a406ac90 0x7f30a406d150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009be0 tx=0x7f30a8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:11.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.224+0000 7f309ffff700 1 -- 172.21.15.31:0/2690185008 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f30b0055c00 con 0x7f30b8151d80 2024-03-30T01:42:11.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:11.491+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f30b80747d0 con 0x7f30b8151d80 2024-03-30T01:42:12.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.278+0000 7f309ffff700 1 -- 172.21.15.31:0/2690185008 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f30b0020020 con 0x7f30b8151d80 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30a406ac90 msgr2=0x7f30a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30a406ac90 0x7f30a406d150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009be0 tx=0x7f30a8009210 comp rx=0 tx=0).stop 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b8151d80 msgr2=0x7f30b8156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b8151d80 0x7f30b8156010 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f30b80cafe0 tx=0x7f30b0004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 shutdown_connections 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30b8151d80 0x7f30b8156010 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30a406ac90 0x7f30a406d150 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30b80c7f10 0x7f30b80ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 --2- 172.21.15.31:0/2690185008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30b80ca890 0x7f30b8153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 >> 172.21.15.31:0/2690185008 conn(0x7f30b8072010 msgr2=0x7f30b80c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:12.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 shutdown_connections 2024-03-30T01:42:12.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.281+0000 7f30bdc44700 1 -- 172.21.15.31:0/2690185008 wait complete. 2024-03-30T01:42:12.303 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T01:42:12.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T01:42:12.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.441+0000 7f5b2ccc2700 1 Processor -- start 2024-03-30T01:42:12.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.441+0000 7f5b2ccc2700 1 -- start start 2024-03-30T01:42:12.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.442+0000 7f5b2ccc2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5b28151b60 0x7f5b28070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.442+0000 7f5b2ccc2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b28151f40 0x7f5b280c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.442+0000 7f5b2ccc2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b28154990 con 0x7f5b280c5d40 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.442+0000 7f5b2ccc2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b28155610 con 0x7f5b28151b60 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.442+0000 7f5b2ccc2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b281547a0 con 0x7f5b28151f40 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.442+0000 7f5b25d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5b280c5d40 0x7f5b280c2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41650/0 (socket says 172.21.15.31:41650) 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b25d9b700 1 -- 172.21.15.31:0/3843019716 learned_addr learned my addr 172.21.15.31:0/3843019716 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2659c700 1 --2- 172.21.15.31:0/3843019716 >> v2:172.21.15.59:3300/0 conn(0x7f5b28151b60 0x7f5b28070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b26d9d700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b28151f40 0x7f5b280c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793963979 0 0) 0x7f5b28154990 con 0x7f5b280c5d40 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b100035b0 con 0x7f5b280c5d40 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1007511050 0 0) 0x7f5b100035b0 con 0x7f5b280c5d40 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b28151f40 msgr2=0x7f5b280c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b28151f40 0x7f5b280c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 >> v2:172.21.15.59:3300/0 conn(0x7f5b28151b60 msgr2=0x7f5b28070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 --2- 172.21.15.31:0/3843019716 >> v2:172.21.15.59:3300/0 conn(0x7f5b28151b60 0x7f5b28070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28154c70 con 0x7f5b280c5d40 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f5b1c001670 con 0x7f5b280c5d40 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 >> v1:172.21.15.31:6789/0 conn(0x7f5b280c5d40 legacy=0x7f5b280c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b28151f40 0x7f5b10003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b10004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b10004d90 0x7f5b10005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b28155610 con 0x7f5b10004060 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1000bac0 con 0x7f5b10004d90 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1000bc80 con 0x7f5b28151f40 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b25d9b700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b10004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b2659c700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b28151f40 0x7f5b10003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.443+0000 7f5b26d9d700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b10004d90 0x7f5b10005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.444+0000 7f5b25d9b700 1 -- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b28151f40 msgr2=0x7f5b10003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:12.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.444+0000 7f5b25d9b700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b28151f40 0x7f5b10003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.444+0000 7f5b25d9b700 1 -- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b10004d90 msgr2=0x7f5b10005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.444+0000 7f5b25d9b700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b10004d90 0x7f5b10005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.444+0000 7f5b25d9b700 1 -- 172.21.15.31:0/3843019716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b100035b0 con 0x7f5b10004060 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.444+0000 7f5b25d9b700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b10004850 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5b280c8db0 tx=0x7f5b10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927584cb78eee719 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.444+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c007c40 con 0x7f5b10004060 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.444+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c00ce00 con 0x7f5b10004060 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.444+0000 7f5b2559a700 1 -- 172.21.15.31:0/3843019716 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c005ba0 con 0x7f5b10004060 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.445+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/3843019716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 msgr2=0x7f5b10004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.445+0000 7f5b2ccc2700 1 --2- 172.21.15.31:0/3843019716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b10004850 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5b280c8db0 tx=0x7f5b10005880 comp rx=0 tx=0).stop 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.445+0000 7f5b26d9d700 1 -- 172.21.15.31:0/3843019716 reap_dead start 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.445+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/3843019716 shutdown_connections 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.445+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/3843019716 >> 172.21.15.31:0/3843019716 conn(0x7f5b280c7d60 msgr2=0x7f5b280c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.445+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/3843019716 shutdown_connections 2024-03-30T01:42:12.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.445+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/3843019716 wait complete. 2024-03-30T01:42:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.446+0000 7f5b2ccc2700 1 Processor -- start 2024-03-30T01:42:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.447+0000 7f5b2ccc2700 1 -- start start 2024-03-30T01:42:12.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.447+0000 7f5b2ccc2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b281ea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2ccc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b280aa410 0x7f5b280abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2ccc2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b281ed130 0x7f5b281ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2ccc2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b281581e0 con 0x7f5b10004060 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2ccc2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b281ecce0 con 0x7f5b280aa410 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2ccc2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b281ea730 con 0x7f5b281ed130 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b281ea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b281ea1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47294/0 (socket says 172.21.15.31:47294) 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2659c700 1 -- 172.21.15.31:0/2526328026 learned_addr learned my addr 172.21.15.31:0/2526328026 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b25d9b700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b280aa410 0x7f5b280abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b26d9d700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b281ed130 0x7f5b281ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2659c700 1 -- 172.21.15.31:0/2526328026 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b281ed130 msgr2=0x7f5b281ed5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2659c700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b281ed130 0x7f5b281ed5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2659c700 1 -- 172.21.15.31:0/2526328026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b280aa410 msgr2=0x7f5b280abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2659c700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b280aa410 0x7f5b280abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2659c700 1 -- 172.21.15.31:0/2526328026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28158a10 con 0x7f5b10004060 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b25d9b700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b280aa410 0x7f5b280abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2659c700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b281ea1f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b18011aa0 tx=0x7f5b18011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b16ffd700 1 -- 172.21.15.31:0/2526328026 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800f980 con 0x7f5b10004060 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b16ffd700 1 -- 172.21.15.31:0/2526328026 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1800fae0 con 0x7f5b10004060 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b16ffd700 1 -- 172.21.15.31:0/2526328026 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800fe90 con 0x7f5b10004060 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.448+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b280a8220 con 0x7f5b10004060 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.449+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2806f8e0 con 0x7f5b10004060 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.449+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b280747d0 con 0x7f5b10004060 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.450+0000 7f5b16ffd700 1 -- 172.21.15.31:0/2526328026 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5b18014440 con 0x7f5b10004060 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.450+0000 7f5b16ffd700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5b0806ac90 0x7f5b0806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.451+0000 7f5b16ffd700 1 -- 172.21.15.31:0/2526328026 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5b18089a00 con 0x7f5b10004060 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.451+0000 7f5b25d9b700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5b0806ac90 0x7f5b0806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:12.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.451+0000 7f5b25d9b700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5b0806ac90 0x7f5b0806d150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b280acd80 tx=0x7f5b1c00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.455+0000 7f5b16ffd700 1 -- 172.21.15.31:0/2526328026 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5b18054bf0 con 0x7f5b10004060 2024-03-30T01:42:12.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:12.717+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5b28158a10 con 0x7f5b10004060 2024-03-30T01:42:12.792 INFO:tasks.ceph.mds.c.smithi031.stderr:2024-03-30T01:42:12.786+0000 7fbce7086700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T01:42:13.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.280+0000 7f5b16ffd700 1 -- 172.21.15.31:0/2526328026 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5b28158a10 con 0x7f5b10004060 2024-03-30T01:42:13.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.283+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5b0806ac90 msgr2=0x7f5b0806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.283+0000 7f5b2ccc2700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5b0806ac90 0x7f5b0806d150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b280acd80 tx=0x7f5b1c00a000 comp rx=0 tx=0).stop 2024-03-30T01:42:13.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.283+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 msgr2=0x7f5b281ea1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.283+0000 7f5b2ccc2700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b281ea1f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b18011aa0 tx=0x7f5b18011e60 comp rx=0 tx=0).stop 2024-03-30T01:42:13.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.283+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 shutdown_connections 2024-03-30T01:42:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.284+0000 7f5b2ccc2700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b10004060 0x7f5b281ea1f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.284+0000 7f5b2ccc2700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5b0806ac90 0x7f5b0806d150 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.284+0000 7f5b2ccc2700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b281ed130 0x7f5b281ed5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.284+0000 7f5b2ccc2700 1 --2- 172.21.15.31:0/2526328026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b280aa410 0x7f5b280abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.284+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 >> 172.21.15.31:0/2526328026 conn(0x7f5b280c7d60 msgr2=0x7f5b2806ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.284+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 shutdown_connections 2024-03-30T01:42:13.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.284+0000 7f5b2ccc2700 1 -- 172.21.15.31:0/2526328026 wait complete. 2024-03-30T01:42:13.305 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T01:42:13.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:13.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.439+0000 7fba57cca700 1 Processor -- start 2024-03-30T01:42:13.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.439+0000 7fba57cca700 1 -- start start 2024-03-30T01:42:13.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.440+0000 7fba57cca700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba5006cb50 0x7fba500c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.440+0000 7fba57cca700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fba500c62f0 0x7fba500a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.440+0000 7fba57cca700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba500c2bd0 con 0x7fba5006cf30 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.440+0000 7fba57cca700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba500c3060 con 0x7fba500c62f0 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.440+0000 7fba57cca700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba500a7900 con 0x7fba5006cb50 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.440+0000 7fba55265700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fba500c62f0 0x7fba500a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.440+0000 7fba55265700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fba500c62f0 0x7fba500a6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47296/0 (socket says 172.21.15.31:47296) 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.440+0000 7fba55265700 1 -- 172.21.15.31:0/4037904992 learned_addr learned my addr 172.21.15.31:0/4037904992 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.440+0000 7fba55a66700 1 --2- 172.21.15.31:0/4037904992 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba5006cb50 0x7fba500c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.441+0000 7fba55265700 1 -- 172.21.15.31:0/4037904992 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba5006cb50 msgr2=0x7fba500c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.441+0000 7fba55265700 1 --2- 172.21.15.31:0/4037904992 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba5006cb50 0x7fba500c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.441+0000 7fba55265700 1 -- 172.21.15.31:0/4037904992 >> v1:172.21.15.59:6789/0 conn(0x7fba5006cf30 legacy=0x7fba500a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.441+0000 7fba55265700 1 -- 172.21.15.31:0/4037904992 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba501589c0 con 0x7fba500c62f0 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.441+0000 7fba55265700 1 --2- 172.21.15.31:0/4037904992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 0x7fba500a6ad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fba400098f0 tx=0x7fba40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2b94207b684220 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.441+0000 7fba54a64700 1 -- 172.21.15.31:0/4037904992 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000b8a0 con 0x7fba500c62f0 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.441+0000 7fba54a64700 1 -- 172.21.15.31:0/4037904992 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4000ba00 con 0x7fba500c62f0 2024-03-30T01:42:13.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.441+0000 7fba54a64700 1 -- 172.21.15.31:0/4037904992 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000bd90 con 0x7fba500c62f0 2024-03-30T01:42:13.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.442+0000 7fba57cca700 1 -- 172.21.15.31:0/4037904992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 msgr2=0x7fba500a6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.442+0000 7fba57cca700 1 --2- 172.21.15.31:0/4037904992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 0x7fba500a6ad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fba400098f0 tx=0x7fba40009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:13.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.442+0000 7fba57cca700 1 -- 172.21.15.31:0/4037904992 shutdown_connections 2024-03-30T01:42:13.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.442+0000 7fba57cca700 1 --2- 172.21.15.31:0/4037904992 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba5006cb50 0x7fba500c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.442+0000 7fba57cca700 1 --2- 172.21.15.31:0/4037904992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 0x7fba500a6ad0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.442+0000 7fba57cca700 1 -- 172.21.15.31:0/4037904992 >> 172.21.15.31:0/4037904992 conn(0x7fba5006a8e0 msgr2=0x7fba5006acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:13.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.442+0000 7fba57cca700 1 -- 172.21.15.31:0/4037904992 shutdown_connections 2024-03-30T01:42:13.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.442+0000 7fba57cca700 1 -- 172.21.15.31:0/4037904992 wait complete. 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.443+0000 7fba57cca700 1 Processor -- start 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.443+0000 7fba57cca700 1 -- start start 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba57cca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba5006cb50 0x7fba500a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba57cca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 0x7fba501ed580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba57cca700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba50153700 0x7fba500aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba57cca700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba501ed0f0 con 0x7fba500c62f0 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba57cca700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba501525a0 con 0x7fba5006cb50 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba57cca700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba50152840 con 0x7fba50153700 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55265700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 0x7fba501ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55265700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 0x7fba501ed580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47300/0 (socket says 172.21.15.31:47300) 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55265700 1 -- 172.21.15.31:0/394827360 learned_addr learned my addr 172.21.15.31:0/394827360 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55a66700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba5006cb50 0x7fba500a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba56267700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba50153700 0x7fba500aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55265700 1 -- 172.21.15.31:0/394827360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba50153700 msgr2=0x7fba500aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55265700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba50153700 0x7fba500aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55265700 1 -- 172.21.15.31:0/394827360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba5006cb50 msgr2=0x7fba500a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55265700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba5006cb50 0x7fba500a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55265700 1 -- 172.21.15.31:0/394827360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba40009550 con 0x7fba500c62f0 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.444+0000 7fba55265700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 0x7fba501ed580 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fba40009c40 tx=0x7fba40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.445+0000 7fba467fc700 1 -- 172.21.15.31:0/394827360 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40004510 con 0x7fba500c62f0 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.445+0000 7fba467fc700 1 -- 172.21.15.31:0/394827360 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4001e070 con 0x7fba500c62f0 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.445+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba501ecd80 con 0x7fba500c62f0 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.445+0000 7fba467fc700 1 -- 172.21.15.31:0/394827360 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400038e0 con 0x7fba500c62f0 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.445+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba501edac0 con 0x7fba500c62f0 2024-03-30T01:42:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.446+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba50073760 con 0x7fba500c62f0 2024-03-30T01:42:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.446+0000 7fba467fc700 1 -- 172.21.15.31:0/394827360 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fba40003a40 con 0x7fba500c62f0 2024-03-30T01:42:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.447+0000 7fba467fc700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fba3406ac90 0x7fba3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.447+0000 7fba55a66700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fba3406ac90 0x7fba3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.447+0000 7fba467fc700 1 -- 172.21.15.31:0/394827360 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fba40020070 con 0x7fba500c62f0 2024-03-30T01:42:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.447+0000 7fba55a66700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fba3406ac90 0x7fba3406d150 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba4c006b20 tx=0x7fba4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:13.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.452+0000 7fba467fc700 1 -- 172.21.15.31:0/394827360 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fba40055860 con 0x7fba500c62f0 2024-03-30T01:42:13.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.713+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba5004ea80 con 0x7fba500c62f0 2024-03-30T01:42:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.714+0000 7fba467fc700 1 -- 172.21.15.31:0/394827360 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fba40058e80 con 0x7fba500c62f0 2024-03-30T01:42:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:13.720 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:10.264695+0000","modified":"2024-03-30T01:42:13.278071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/1616247357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1616247357},{"type":"v1","addr":"172.21.15.31:6837","nonce":1616247357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T01:42:13.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fba3406ac90 msgr2=0x7fba3406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fba3406ac90 0x7fba3406d150 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba4c006b20 tx=0x7fba4c008040 comp rx=0 tx=0).stop 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 msgr2=0x7fba501ed580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 0x7fba501ed580 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fba40009c40 tx=0x7fba40004750 comp rx=0 tx=0).stop 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 shutdown_connections 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fba3406ac90 0x7fba3406d150 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba50153700 0x7fba500aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba5006cb50 0x7fba500a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 --2- 172.21.15.31:0/394827360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500c62f0 0x7fba501ed580 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 >> 172.21.15.31:0/394827360 conn(0x7fba5006a8e0 msgr2=0x7fba5006f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 shutdown_connections 2024-03-30T01:42:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.716+0000 7fba57cca700 1 -- 172.21.15.31:0/394827360 wait complete. 2024-03-30T01:42:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-03-30T01:42:13.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:13.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.865+0000 7fbcd4af3700 1 Processor -- start 2024-03-30T01:42:13.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.865+0000 7fbcd4af3700 1 -- start start 2024-03-30T01:42:13.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcd4af3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbcd0151f00 0x7fbcd00705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcd4af3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd00c5f50 0x7fbcd00c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcd4af3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbcd00c4d90 0x7fbcd01547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcd4af3700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd00c4010 con 0x7fbcd00c4d90 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcd4af3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd01522e0 con 0x7fbcd0151f00 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcd4af3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd0157d50 con 0x7fbcd00c5f50 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcced9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbcd00c4d90 0x7fbcd01547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcced9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbcd00c4d90 0x7fbcd01547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47310/0 (socket says 172.21.15.31:47310) 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcced9d700 1 -- 172.21.15.31:0/1542737905 learned_addr learned my addr 172.21.15.31:0/1542737905 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbcce59c700 1 --2- 172.21.15.31:0/1542737905 >> v2:172.21.15.59:3300/0 conn(0x7fbcd0151f00 0x7fbcd00705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.866+0000 7fbccdd9b700 1 --2- 172.21.15.31:0/1542737905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd00c5f50 0x7fbcd00c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbcced9d700 1 -- 172.21.15.31:0/1542737905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd00c5f50 msgr2=0x7fbcd00c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbcced9d700 1 --2- 172.21.15.31:0/1542737905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd00c5f50 0x7fbcd00c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbcced9d700 1 -- 172.21.15.31:0/1542737905 >> v2:172.21.15.59:3300/0 conn(0x7fbcd0151f00 msgr2=0x7fbcd00705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbcced9d700 1 --2- 172.21.15.31:0/1542737905 >> v2:172.21.15.59:3300/0 conn(0x7fbcd0151f00 0x7fbcd00705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbcced9d700 1 -- 172.21.15.31:0/1542737905 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0070c30 con 0x7fbcd00c4d90 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbcce59c700 1 --2- 172.21.15.31:0/1542737905 >> v2:172.21.15.59:3300/0 conn(0x7fbcd0151f00 0x7fbcd00705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbcced9d700 1 --2- 172.21.15.31:0/1542737905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 0x7fbcd01547e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00c5500 tx=0x7fbcc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b368fbd315388936 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbccd59a700 1 -- 172.21.15.31:0/1542737905 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc800e070 con 0x7fbcd00c4d90 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbccd59a700 1 -- 172.21.15.31:0/1542737905 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc800b9c0 con 0x7fbcd00c4d90 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.867+0000 7fbccd59a700 1 -- 172.21.15.31:0/1542737905 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc800bd70 con 0x7fbcd00c4d90 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.868+0000 7fbcd4af3700 1 -- 172.21.15.31:0/1542737905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 msgr2=0x7fbcd01547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.868+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/1542737905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 0x7fbcd01547e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00c5500 tx=0x7fbcc800b330 comp rx=0 tx=0).stop 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.868+0000 7fbcd4af3700 1 -- 172.21.15.31:0/1542737905 shutdown_connections 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.868+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/1542737905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 0x7fbcd01547e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.868+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/1542737905 >> v2:172.21.15.59:3300/0 conn(0x7fbcd0151f00 0x7fbcd00705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.868+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/1542737905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd00c5f50 0x7fbcd00c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.868+0000 7fbcd4af3700 1 -- 172.21.15.31:0/1542737905 >> 172.21.15.31:0/1542737905 conn(0x7fbcd006ad30 msgr2=0x7fbcd00a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.868+0000 7fbcd4af3700 1 -- 172.21.15.31:0/1542737905 shutdown_connections 2024-03-30T01:42:13.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.868+0000 7fbcd4af3700 1 -- 172.21.15.31:0/1542737905 wait complete. 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.870+0000 7fbcd4af3700 1 Processor -- start 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.870+0000 7fbcd4af3700 1 -- start start 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcd4af3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 0x7fbcd00acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcd4af3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcd00c5f50 0x7fbcd00aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcd4af3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd0151f00 0x7fbcd01ed380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcd4af3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd0152f00 con 0x7fbcd00c4d90 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcd4af3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd006e5f0 con 0x7fbcd00c5f50 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcd4af3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd01edcc0 con 0x7fbcd0151f00 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcce59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 0x7fbcd00acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcce59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 0x7fbcd00acc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47318/0 (socket says 172.21.15.31:47318) 2024-03-30T01:42:13.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcce59c700 1 -- 172.21.15.31:0/2558814930 learned_addr learned my addr 172.21.15.31:0/2558814930 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbccdd9b700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcd00c5f50 0x7fbcd00aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcced9d700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd0151f00 0x7fbcd01ed380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcce59c700 1 -- 172.21.15.31:0/2558814930 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd0151f00 msgr2=0x7fbcd01ed380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcce59c700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd0151f00 0x7fbcd01ed380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcce59c700 1 -- 172.21.15.31:0/2558814930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcd00c5f50 msgr2=0x7fbcd00aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcce59c700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcd00c5f50 0x7fbcd00aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcce59c700 1 -- 172.21.15.31:0/2558814930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc8009b30 con 0x7fbcd00c4d90 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcce59c700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 0x7fbcd00acc50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e810 tx=0x7fbcc000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/2558814930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000c020 con 0x7fbcd00c4d90 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/2558814930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc0004250 con 0x7fbcd00c4d90 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd0152ad0 con 0x7fbcd00c4d90 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.871+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/2558814930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00036a0 con 0x7fbcd00c4d90 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.872+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd01527f0 con 0x7fbcd00c4d90 2024-03-30T01:42:13.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.872+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd0152650 con 0x7fbcd00c4d90 2024-03-30T01:42:13.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.873+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/2558814930 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbcc0003800 con 0x7fbcd00c4d90 2024-03-30T01:42:13.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.873+0000 7fbcc6ffd700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbcb006ac90 0x7fbcb006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:13.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.874+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/2558814930 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbcc0088f40 con 0x7fbcd00c4d90 2024-03-30T01:42:13.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.874+0000 7fbccdd9b700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbcb006ac90 0x7fbcb006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:13.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.874+0000 7fbccdd9b700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbcb006ac90 0x7fbcb006d150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0157ce0 tx=0x7fbcb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:13.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:13.878+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/2558814930 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbcd0152650 con 0x7fbcd00c4d90 2024-03-30T01:42:14.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.097+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcd0152650 con 0x7fbcd00c4d90 2024-03-30T01:42:14.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.098+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/2558814930 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16258 (secure 0 0 0) 0x7fbcd0152650 con 0x7fbcd00c4d90 2024-03-30T01:42:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:14.105 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":39,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:11.266485+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:14.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbcb006ac90 msgr2=0x7fbcb006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbcb006ac90 0x7fbcb006d150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0157ce0 tx=0x7fbcb8008040 comp rx=0 tx=0).stop 2024-03-30T01:42:14.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 msgr2=0x7fbcd00acc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 0x7fbcd00acc50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e810 tx=0x7fbcc000ebd0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 shutdown_connections 2024-03-30T01:42:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcd00c4d90 0x7fbcd00acc50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fbcb006ac90 0x7fbcb006d150 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbcd0151f00 0x7fbcd01ed380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 --2- 172.21.15.31:0/2558814930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcd00c5f50 0x7fbcd00aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 >> 172.21.15.31:0/2558814930 conn(0x7fbcd006ad30 msgr2=0x7fbcd00c1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 shutdown_connections 2024-03-30T01:42:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.101+0000 7fbcd4af3700 1 -- 172.21.15.31:0/2558814930 wait complete. 2024-03-30T01:42:14.121 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T01:42:14.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:42:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.250+0000 7f26e2b6e700 1 Processor -- start 2024-03-30T01:42:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.251+0000 7f26e2b6e700 1 -- start start 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.251+0000 7f26e2b6e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26dc0c8310 0x7f26dc0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.251+0000 7f26e2b6e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26dc1556d0 con 0x7f26dc0c2870 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.251+0000 7f26e2b6e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26dc0c1f70 con 0x7f26dc0c5df0 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.251+0000 7f26e2b6e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26dc1559b0 con 0x7f26dc0c8310 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26e110b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f26dc0c2870 0x7f26dc0a6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41666/0 (socket says 172.21.15.31:41666) 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26e110b700 1 -- 172.21.15.31:0/2022055957 learned_addr learned my addr 172.21.15.31:0/2022055957 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26e090a700 1 --2- 172.21.15.31:0/2022055957 >> v2:172.21.15.59:3300/0 conn(0x7f26dc0c8310 0x7f26dc0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26db7fe700 1 -- 172.21.15.31:0/2022055957 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688330010 0 0) 0x7f26dc1556d0 con 0x7f26dc0c2870 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26db7fe700 1 -- 172.21.15.31:0/2022055957 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c80035b0 con 0x7f26dc0c2870 2024-03-30T01:42:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26e090a700 1 -- 172.21.15.31:0/2022055957 >> v1:172.21.15.59:6789/0 conn(0x7f26dc0c5df0 legacy=0x7f26dc151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26e090a700 1 -- 172.21.15.31:0/2022055957 >> v1:172.21.15.31:6789/0 conn(0x7f26dc0c2870 legacy=0x7f26dc0a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26e090a700 1 -- 172.21.15.31:0/2022055957 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc0c2250 con 0x7f26dc0c8310 2024-03-30T01:42:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26e090a700 1 --2- 172.21.15.31:0/2022055957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26dc0c8310 0x7f26dc0c86f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0098a0 tx=0x7f26cc009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=517f3809a8746970 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26db7fe700 1 -- 172.21.15.31:0/2022055957 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc011830 con 0x7f26dc0c8310 2024-03-30T01:42:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26db7fe700 1 -- 172.21.15.31:0/2022055957 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc011990 con 0x7f26dc0c8310 2024-03-30T01:42:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.252+0000 7f26db7fe700 1 -- 172.21.15.31:0/2022055957 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc011d20 con 0x7f26dc0c8310 2024-03-30T01:42:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.253+0000 7f26e2b6e700 1 -- 172.21.15.31:0/2022055957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26dc0c8310 msgr2=0x7f26dc0c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.253+0000 7f26e2b6e700 1 --2- 172.21.15.31:0/2022055957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26dc0c8310 0x7f26dc0c86f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0098a0 tx=0x7f26cc009bb0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.253+0000 7f26e2b6e700 1 -- 172.21.15.31:0/2022055957 shutdown_connections 2024-03-30T01:42:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.253+0000 7f26e2b6e700 1 --2- 172.21.15.31:0/2022055957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26dc0c8310 0x7f26dc0c86f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.253+0000 7f26e2b6e700 1 -- 172.21.15.31:0/2022055957 >> 172.21.15.31:0/2022055957 conn(0x7f26dc06ab20 msgr2=0x7f26dc06af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.253+0000 7f26e2b6e700 1 -- 172.21.15.31:0/2022055957 shutdown_connections 2024-03-30T01:42:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.253+0000 7f26e2b6e700 1 -- 172.21.15.31:0/2022055957 wait complete. 2024-03-30T01:42:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.254+0000 7f26e2b6e700 1 Processor -- start 2024-03-30T01:42:14.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.254+0000 7f26e2b6e700 1 -- start start 2024-03-30T01:42:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.255+0000 7f26e2b6e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26dc0c2870 0x7f26dc06f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.255+0000 7f26e2b6e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26dc0c5df0 0x7f26dc1e9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.255+0000 7f26e2b6e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26dc0c8310 0x7f26dc14e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.255+0000 7f26e2b6e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc1520f0 con 0x7f26dc0c8310 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.255+0000 7f26e2b6e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc152d30 con 0x7f26dc0c2870 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.255+0000 7f26e2b6e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc1ed800 con 0x7f26dc0c5df0 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.255+0000 7f26e110b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26dc0c8310 0x7f26dc14e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.255+0000 7f26e110b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26dc0c8310 0x7f26dc14e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47328/0 (socket says 172.21.15.31:47328) 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.255+0000 7f26e110b700 1 -- 172.21.15.31:0/1280188018 learned_addr learned my addr 172.21.15.31:0/1280188018 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26e090a700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26dc0c2870 0x7f26dc06f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26dbfff700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26dc0c5df0 0x7f26dc1e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26e110b700 1 -- 172.21.15.31:0/1280188018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26dc0c5df0 msgr2=0x7f26dc1e9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26e110b700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26dc0c5df0 0x7f26dc1e9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26e110b700 1 -- 172.21.15.31:0/1280188018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26dc0c2870 msgr2=0x7f26dc06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26e110b700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26dc0c2870 0x7f26dc06f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26e110b700 1 -- 172.21.15.31:0/1280188018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc0094a0 con 0x7f26dc0c8310 2024-03-30T01:42:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26e110b700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26dc0c8310 0x7f26dc14e1d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f26d400cb40 tx=0x7f26d400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26d97fa700 1 -- 172.21.15.31:0/1280188018 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d400a9e0 con 0x7f26dc0c8310 2024-03-30T01:42:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26d97fa700 1 -- 172.21.15.31:0/1280188018 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d400ab40 con 0x7f26dc0c8310 2024-03-30T01:42:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc152fd0 con 0x7f26dc0c8310 2024-03-30T01:42:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26d97fa700 1 -- 172.21.15.31:0/1280188018 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4012400 con 0x7f26dc0c8310 2024-03-30T01:42:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.256+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc158c30 con 0x7f26dc0c8310 2024-03-30T01:42:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.257+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26dc073760 con 0x7f26dc0c8310 2024-03-30T01:42:14.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.258+0000 7f26d97fa700 1 -- 172.21.15.31:0/1280188018 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f26d401b3b0 con 0x7f26dc0c8310 2024-03-30T01:42:14.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.258+0000 7f26d97fa700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f26bc06ac90 0x7f26bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.258+0000 7f26d97fa700 1 -- 172.21.15.31:0/1280188018 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f26d4088f50 con 0x7f26dc0c8310 2024-03-30T01:42:14.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.258+0000 7f26e090a700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f26bc06ac90 0x7f26bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.259+0000 7f26e090a700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f26bc06ac90 0x7f26bc06d150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0090b0 tx=0x7f26cc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:14.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.263+0000 7f26d97fa700 1 -- 172.21.15.31:0/1280188018 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f26d40540a0 con 0x7f26dc0c8310 2024-03-30T01:42:14.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.475+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26dc0c2f60 con 0x7f26bc06ac90 2024-03-30T01:42:14.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.484+0000 7f26d97fa700 1 -- 172.21.15.31:0/1280188018 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288419 (secure 0 0 0) 0x7f26cc011b80 con 0x7f26bc06ac90 2024-03-30T01:42:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f26bc06ac90 msgr2=0x7f26bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f26bc06ac90 0x7f26bc06d150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0090b0 tx=0x7f26cc01c040 comp rx=0 tx=0).stop 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26dc0c8310 msgr2=0x7f26dc14e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26dc0c8310 0x7f26dc14e1d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f26d400cb40 tx=0x7f26d400ce50 comp rx=0 tx=0).stop 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 shutdown_connections 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26dc0c8310 0x7f26dc14e1d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f26bc06ac90 0x7f26bc06d150 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26dc0c2870 0x7f26dc06f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 --2- 172.21.15.31:0/1280188018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26dc0c5df0 0x7f26dc1e9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 >> 172.21.15.31:0/1280188018 conn(0x7f26dc06ab20 msgr2=0x7f26dc06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 shutdown_connections 2024-03-30T01:42:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.488+0000 7f26e2b6e700 1 -- 172.21.15.31:0/1280188018 wait complete. 2024-03-30T01:42:14.498 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:42:14.508 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-03-30T01:42:13.825078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220948,"kb_used_data":4740,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753772,"statfs":{"total":773094113280,"available":772867862528,"internally_reserved":0,"allocated":4853760,"data_stored":2737411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.019887"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878628+0000","last_change":"2024-03-30T01:42:10.288258+0000","last_active":"2024-03-30T01:42:11.878628+0000","last_peered":"2024-03-30T01:42:11.878628+0000","last_clean":"2024-03-30T01:42:11.878628+0000","last_became_active":"2024-03-30T01:42:10.287993+0000","last_became_peered":"2024-03-30T01:42:10.287993+0000","last_unstale":"2024-03-30T01:42:11.878628+0000","last_undegraded":"2024-03-30T01:42:11.878628+0000","last_fullsized":"2024-03-30T01:42:11.878628+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878866+0000","last_change":"2024-03-30T01:42:10.294965+0000","last_active":"2024-03-30T01:42:11.878866+0000","last_peered":"2024-03-30T01:42:11.878866+0000","last_clean":"2024-03-30T01:42:11.878866+0000","last_became_active":"2024-03-30T01:42:10.294685+0000","last_became_peered":"2024-03-30T01:42:10.294685+0000","last_unstale":"2024-03-30T01:42:11.878866+0000","last_undegraded":"2024-03-30T01:42:11.878866+0000","last_fullsized":"2024-03-30T01:42:11.878866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878403+0000","last_change":"2024-03-30T01:42:10.285050+0000","last_active":"2024-03-30T01:42:11.878403+0000","last_peered":"2024-03-30T01:42:11.878403+0000","last_clean":"2024-03-30T01:42:11.878403+0000","last_became_active":"2024-03-30T01:42:10.284771+0000","last_became_peered":"2024-03-30T01:42:10.284771+0000","last_unstale":"2024-03-30T01:42:11.878403+0000","last_undegraded":"2024-03-30T01:42:11.878403+0000","last_fullsized":"2024-03-30T01:42:11.878403+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278035+0000","last_change":"2024-03-30T01:42:10.294277+0000","last_active":"2024-03-30T01:42:11.278035+0000","last_peered":"2024-03-30T01:42:11.278035+0000","last_clean":"2024-03-30T01:42:11.278035+0000","last_became_active":"2024-03-30T01:42:10.293995+0000","last_became_peered":"2024-03-30T01:42:10.293995+0000","last_unstale":"2024-03-30T01:42:11.278035+0000","last_undegraded":"2024-03-30T01:42:11.278035+0000","last_fullsized":"2024-03-30T01:42:11.278035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:03.888060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277927+0000","last_change":"2024-03-30T01:42:10.299868+0000","last_active":"2024-03-30T01:42:11.277927+0000","last_peered":"2024-03-30T01:42:11.277927+0000","last_clean":"2024-03-30T01:42:11.277927+0000","last_became_active":"2024-03-30T01:42:10.299612+0000","last_became_peered":"2024-03-30T01:42:10.299612+0000","last_unstale":"2024-03-30T01:42:11.277927+0000","last_undegraded":"2024-03-30T01:42:11.277927+0000","last_fullsized":"2024-03-30T01:42:11.277927+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:49.471045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878254+0000","last_change":"2024-03-30T01:42:10.287001+0000","last_active":"2024-03-30T01:42:11.878254+0000","last_peered":"2024-03-30T01:42:11.878254+0000","last_clean":"2024-03-30T01:42:11.878254+0000","last_became_active":"2024-03-30T01:42:10.286700+0000","last_became_peered":"2024-03-30T01:42:10.286700+0000","last_unstale":"2024-03-30T01:42:11.878254+0000","last_undegraded":"2024-03-30T01:42:11.878254+0000","last_fullsized":"2024-03-30T01:42:11.878254+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279027+0000","last_change":"2024-03-30T01:42:10.289068+0000","last_active":"2024-03-30T01:42:11.279027+0000","last_peered":"2024-03-30T01:42:11.279027+0000","last_clean":"2024-03-30T01:42:11.279027+0000","last_became_active":"2024-03-30T01:42:10.288282+0000","last_became_peered":"2024-03-30T01:42:10.288282+0000","last_unstale":"2024-03-30T01:42:11.279027+0000","last_undegraded":"2024-03-30T01:42:11.279027+0000","last_fullsized":"2024-03-30T01:42:11.279027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878325+0000","last_change":"2024-03-30T01:42:10.298731+0000","last_active":"2024-03-30T01:42:11.878325+0000","last_peered":"2024-03-30T01:42:11.878325+0000","last_clean":"2024-03-30T01:42:11.878325+0000","last_became_active":"2024-03-30T01:42:10.298525+0000","last_became_peered":"2024-03-30T01:42:10.298525+0000","last_unstale":"2024-03-30T01:42:11.878325+0000","last_undegraded":"2024-03-30T01:42:11.878325+0000","last_fullsized":"2024-03-30T01:42:11.878325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878035+0000","last_change":"2024-03-30T01:42:10.285717+0000","last_active":"2024-03-30T01:42:11.878035+0000","last_peered":"2024-03-30T01:42:11.878035+0000","last_clean":"2024-03-30T01:42:11.878035+0000","last_became_active":"2024-03-30T01:42:10.285437+0000","last_became_peered":"2024-03-30T01:42:10.285437+0000","last_unstale":"2024-03-30T01:42:11.878035+0000","last_undegraded":"2024-03-30T01:42:11.878035+0000","last_fullsized":"2024-03-30T01:42:11.878035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279381+0000","last_change":"2024-03-30T01:42:10.295963+0000","last_active":"2024-03-30T01:42:11.279381+0000","last_peered":"2024-03-30T01:42:11.279381+0000","last_clean":"2024-03-30T01:42:11.279381+0000","last_became_active":"2024-03-30T01:42:10.295545+0000","last_became_peered":"2024-03-30T01:42:10.295545+0000","last_unstale":"2024-03-30T01:42:11.279381+0000","last_undegraded":"2024-03-30T01:42:11.279381+0000","last_fullsized":"2024-03-30T01:42:11.279381+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279241+0000","last_change":"2024-03-30T01:42:10.296160+0000","last_active":"2024-03-30T01:42:11.279241+0000","last_peered":"2024-03-30T01:42:11.279241+0000","last_clean":"2024-03-30T01:42:11.279241+0000","last_became_active":"2024-03-30T01:42:10.295867+0000","last_became_peered":"2024-03-30T01:42:10.295867+0000","last_unstale":"2024-03-30T01:42:11.279241+0000","last_undegraded":"2024-03-30T01:42:11.279241+0000","last_fullsized":"2024-03-30T01:42:11.279241+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878076+0000","last_change":"2024-03-30T01:42:10.294103+0000","last_active":"2024-03-30T01:42:11.878076+0000","last_peered":"2024-03-30T01:42:11.878076+0000","last_clean":"2024-03-30T01:42:11.878076+0000","last_became_active":"2024-03-30T01:42:10.293833+0000","last_became_peered":"2024-03-30T01:42:10.293833+0000","last_unstale":"2024-03-30T01:42:11.878076+0000","last_undegraded":"2024-03-30T01:42:11.878076+0000","last_fullsized":"2024-03-30T01:42:11.878076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278814+0000","last_change":"2024-03-30T01:42:10.295808+0000","last_active":"2024-03-30T01:42:11.278814+0000","last_peered":"2024-03-30T01:42:11.278814+0000","last_clean":"2024-03-30T01:42:11.278814+0000","last_became_active":"2024-03-30T01:42:10.295460+0000","last_became_peered":"2024-03-30T01:42:10.295460+0000","last_unstale":"2024-03-30T01:42:11.278814+0000","last_undegraded":"2024-03-30T01:42:11.278814+0000","last_fullsized":"2024-03-30T01:42:11.278814+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878487+0000","last_change":"2024-03-30T01:42:10.294667+0000","last_active":"2024-03-30T01:42:11.878487+0000","last_peered":"2024-03-30T01:42:11.878487+0000","last_clean":"2024-03-30T01:42:11.878487+0000","last_became_active":"2024-03-30T01:42:10.294330+0000","last_became_peered":"2024-03-30T01:42:10.294330+0000","last_unstale":"2024-03-30T01:42:11.878487+0000","last_undegraded":"2024-03-30T01:42:11.878487+0000","last_fullsized":"2024-03-30T01:42:11.878487+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278796+0000","last_change":"2024-03-30T01:42:10.293953+0000","last_active":"2024-03-30T01:42:11.278796+0000","last_peered":"2024-03-30T01:42:11.278796+0000","last_clean":"2024-03-30T01:42:11.278796+0000","last_became_active":"2024-03-30T01:42:10.293641+0000","last_became_peered":"2024-03-30T01:42:10.293641+0000","last_unstale":"2024-03-30T01:42:11.278796+0000","last_undegraded":"2024-03-30T01:42:11.278796+0000","last_fullsized":"2024-03-30T01:42:11.278796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877469+0000","last_change":"2024-03-30T01:42:10.284419+0000","last_active":"2024-03-30T01:42:11.877469+0000","last_peered":"2024-03-30T01:42:11.877469+0000","last_clean":"2024-03-30T01:42:11.877469+0000","last_became_active":"2024-03-30T01:42:10.284098+0000","last_became_peered":"2024-03-30T01:42:10.284098+0000","last_unstale":"2024-03-30T01:42:11.877469+0000","last_undegraded":"2024-03-30T01:42:11.877469+0000","last_fullsized":"2024-03-30T01:42:11.877469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878378+0000","last_change":"2024-03-30T01:42:10.285733+0000","last_active":"2024-03-30T01:42:11.878378+0000","last_peered":"2024-03-30T01:42:11.878378+0000","last_clean":"2024-03-30T01:42:11.878378+0000","last_became_active":"2024-03-30T01:42:10.285428+0000","last_became_peered":"2024-03-30T01:42:10.285428+0000","last_unstale":"2024-03-30T01:42:11.878378+0000","last_undegraded":"2024-03-30T01:42:11.878378+0000","last_fullsized":"2024-03-30T01:42:11.878378+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277996+0000","last_change":"2024-03-30T01:42:10.300393+0000","last_active":"2024-03-30T01:42:11.277996+0000","last_peered":"2024-03-30T01:42:11.277996+0000","last_clean":"2024-03-30T01:42:11.277996+0000","last_became_active":"2024-03-30T01:42:10.299795+0000","last_became_peered":"2024-03-30T01:42:10.299795+0000","last_unstale":"2024-03-30T01:42:11.277996+0000","last_undegraded":"2024-03-30T01:42:11.277996+0000","last_fullsized":"2024-03-30T01:42:11.277996+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:21.256043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278164+0000","last_change":"2024-03-30T01:42:10.292351+0000","last_active":"2024-03-30T01:42:11.278164+0000","last_peered":"2024-03-30T01:42:11.278164+0000","last_clean":"2024-03-30T01:42:11.278164+0000","last_became_active":"2024-03-30T01:42:10.292040+0000","last_became_peered":"2024-03-30T01:42:10.292040+0000","last_unstale":"2024-03-30T01:42:11.278164+0000","last_undegraded":"2024-03-30T01:42:11.278164+0000","last_fullsized":"2024-03-30T01:42:11.278164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:09.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277912+0000","last_change":"2024-03-30T01:42:10.300160+0000","last_active":"2024-03-30T01:42:11.277912+0000","last_peered":"2024-03-30T01:42:11.277912+0000","last_clean":"2024-03-30T01:42:11.277912+0000","last_became_active":"2024-03-30T01:42:10.299728+0000","last_became_peered":"2024-03-30T01:42:10.299728+0000","last_unstale":"2024-03-30T01:42:11.277912+0000","last_undegraded":"2024-03-30T01:42:11.277912+0000","last_fullsized":"2024-03-30T01:42:11.277912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278944+0000","last_change":"2024-03-30T01:42:10.304038+0000","last_active":"2024-03-30T01:42:11.278944+0000","last_peered":"2024-03-30T01:42:11.278944+0000","last_clean":"2024-03-30T01:42:11.278944+0000","last_became_active":"2024-03-30T01:42:10.303747+0000","last_became_peered":"2024-03-30T01:42:10.303747+0000","last_unstale":"2024-03-30T01:42:11.278944+0000","last_undegraded":"2024-03-30T01:42:11.278944+0000","last_fullsized":"2024-03-30T01:42:11.278944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877668+0000","last_change":"2024-03-30T01:42:10.299194+0000","last_active":"2024-03-30T01:42:11.877668+0000","last_peered":"2024-03-30T01:42:11.877668+0000","last_clean":"2024-03-30T01:42:11.877668+0000","last_became_active":"2024-03-30T01:42:10.298900+0000","last_became_peered":"2024-03-30T01:42:10.298900+0000","last_unstale":"2024-03-30T01:42:11.877668+0000","last_undegraded":"2024-03-30T01:42:11.877668+0000","last_fullsized":"2024-03-30T01:42:11.877668+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878157+0000","last_change":"2024-03-30T01:42:10.281331+0000","last_active":"2024-03-30T01:42:11.878157+0000","last_peered":"2024-03-30T01:42:11.878157+0000","last_clean":"2024-03-30T01:42:11.878157+0000","last_became_active":"2024-03-30T01:42:10.281011+0000","last_became_peered":"2024-03-30T01:42:10.281011+0000","last_unstale":"2024-03-30T01:42:11.878157+0000","last_undegraded":"2024-03-30T01:42:11.878157+0000","last_fullsized":"2024-03-30T01:42:11.878157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277895+0000","last_change":"2024-03-30T01:42:10.286887+0000","last_active":"2024-03-30T01:42:11.277895+0000","last_peered":"2024-03-30T01:42:11.277895+0000","last_clean":"2024-03-30T01:42:11.277895+0000","last_became_active":"2024-03-30T01:42:10.278107+0000","last_became_peered":"2024-03-30T01:42:10.278107+0000","last_unstale":"2024-03-30T01:42:11.277895+0000","last_undegraded":"2024-03-30T01:42:11.277895+0000","last_fullsized":"2024-03-30T01:42:11.277895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279098+0000","last_change":"2024-03-30T01:42:10.285378+0000","last_active":"2024-03-30T01:42:11.279098+0000","last_peered":"2024-03-30T01:42:11.279098+0000","last_clean":"2024-03-30T01:42:11.279098+0000","last_became_active":"2024-03-30T01:42:10.285105+0000","last_became_peered":"2024-03-30T01:42:10.285105+0000","last_unstale":"2024-03-30T01:42:11.279098+0000","last_undegraded":"2024-03-30T01:42:11.279098+0000","last_fullsized":"2024-03-30T01:42:11.279098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:52.824468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878480+0000","last_change":"2024-03-30T01:42:10.290378+0000","last_active":"2024-03-30T01:42:11.878480+0000","last_peered":"2024-03-30T01:42:11.878480+0000","last_clean":"2024-03-30T01:42:11.878480+0000","last_became_active":"2024-03-30T01:42:10.290022+0000","last_became_peered":"2024-03-30T01:42:10.290022+0000","last_unstale":"2024-03-30T01:42:11.878480+0000","last_undegraded":"2024-03-30T01:42:11.878480+0000","last_fullsized":"2024-03-30T01:42:11.878480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278078+0000","last_change":"2024-03-30T01:42:10.274845+0000","last_active":"2024-03-30T01:42:11.278078+0000","last_peered":"2024-03-30T01:42:11.278078+0000","last_clean":"2024-03-30T01:42:11.278078+0000","last_became_active":"2024-03-30T01:42:10.274566+0000","last_became_peered":"2024-03-30T01:42:10.274566+0000","last_unstale":"2024-03-30T01:42:11.278078+0000","last_undegraded":"2024-03-30T01:42:11.278078+0000","last_fullsized":"2024-03-30T01:42:11.278078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:32.801396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279479+0000","last_change":"2024-03-30T01:42:10.288633+0000","last_active":"2024-03-30T01:42:11.279479+0000","last_peered":"2024-03-30T01:42:11.279479+0000","last_clean":"2024-03-30T01:42:11.279479+0000","last_became_active":"2024-03-30T01:42:10.288326+0000","last_became_peered":"2024-03-30T01:42:10.288326+0000","last_unstale":"2024-03-30T01:42:11.279479+0000","last_undegraded":"2024-03-30T01:42:11.279479+0000","last_fullsized":"2024-03-30T01:42:11.279479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278129+0000","last_change":"2024-03-30T01:42:10.291099+0000","last_active":"2024-03-30T01:42:11.278129+0000","last_peered":"2024-03-30T01:42:11.278129+0000","last_clean":"2024-03-30T01:42:11.278129+0000","last_became_active":"2024-03-30T01:42:10.290802+0000","last_became_peered":"2024-03-30T01:42:10.290802+0000","last_unstale":"2024-03-30T01:42:11.278129+0000","last_undegraded":"2024-03-30T01:42:11.278129+0000","last_fullsized":"2024-03-30T01:42:11.278129+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:13.070677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277537+0000","last_change":"2024-03-30T01:42:10.290078+0000","last_active":"2024-03-30T01:42:11.277537+0000","last_peered":"2024-03-30T01:42:11.277537+0000","last_clean":"2024-03-30T01:42:11.277537+0000","last_became_active":"2024-03-30T01:42:10.289127+0000","last_became_peered":"2024-03-30T01:42:10.289127+0000","last_unstale":"2024-03-30T01:42:11.277537+0000","last_undegraded":"2024-03-30T01:42:11.277537+0000","last_fullsized":"2024-03-30T01:42:11.277537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:36.303285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877529+0000","last_change":"2024-03-30T01:42:10.274200+0000","last_active":"2024-03-30T01:42:11.877529+0000","last_peered":"2024-03-30T01:42:11.877529+0000","last_clean":"2024-03-30T01:42:11.877529+0000","last_became_active":"2024-03-30T01:42:10.273923+0000","last_became_peered":"2024-03-30T01:42:10.273923+0000","last_unstale":"2024-03-30T01:42:11.877529+0000","last_undegraded":"2024-03-30T01:42:11.877529+0000","last_fullsized":"2024-03-30T01:42:11.877529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:44.766475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877662+0000","last_change":"2024-03-30T01:42:10.297189+0000","last_active":"2024-03-30T01:42:11.877662+0000","last_peered":"2024-03-30T01:42:11.877662+0000","last_clean":"2024-03-30T01:42:11.877662+0000","last_became_active":"2024-03-30T01:42:10.296821+0000","last_became_peered":"2024-03-30T01:42:10.296821+0000","last_unstale":"2024-03-30T01:42:11.877662+0000","last_undegraded":"2024-03-30T01:42:11.877662+0000","last_fullsized":"2024-03-30T01:42:11.877662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877932+0000","last_change":"2024-03-30T01:42:10.285919+0000","last_active":"2024-03-30T01:42:11.877932+0000","last_peered":"2024-03-30T01:42:11.877932+0000","last_clean":"2024-03-30T01:42:11.877932+0000","last_became_active":"2024-03-30T01:42:10.285629+0000","last_became_peered":"2024-03-30T01:42:10.285629+0000","last_unstale":"2024-03-30T01:42:11.877932+0000","last_undegraded":"2024-03-30T01:42:11.877932+0000","last_fullsized":"2024-03-30T01:42:11.877932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279214+0000","last_change":"2024-03-30T01:42:09.824850+0000","last_active":"2024-03-30T01:42:11.279214+0000","last_peered":"2024-03-30T01:42:11.279214+0000","last_clean":"2024-03-30T01:42:11.279214+0000","last_became_active":"2024-03-30T01:42:09.824537+0000","last_became_peered":"2024-03-30T01:42:09.824537+0000","last_unstale":"2024-03-30T01:42:11.279214+0000","last_undegraded":"2024-03-30T01:42:11.279214+0000","last_fullsized":"2024-03-30T01:42:11.279214+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:17.759503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878227+0000","last_change":"2024-03-30T01:42:10.298381+0000","last_active":"2024-03-30T01:42:11.878227+0000","last_peered":"2024-03-30T01:42:11.878227+0000","last_clean":"2024-03-30T01:42:11.878227+0000","last_became_active":"2024-03-30T01:42:10.298109+0000","last_became_peered":"2024-03-30T01:42:10.298109+0000","last_unstale":"2024-03-30T01:42:11.878227+0000","last_undegraded":"2024-03-30T01:42:11.878227+0000","last_fullsized":"2024-03-30T01:42:11.878227+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878784+0000","last_change":"2024-03-30T01:42:09.829899+0000","last_active":"2024-03-30T01:42:11.878784+0000","last_peered":"2024-03-30T01:42:11.878784+0000","last_clean":"2024-03-30T01:42:11.878784+0000","last_became_active":"2024-03-30T01:42:09.829601+0000","last_became_peered":"2024-03-30T01:42:09.829601+0000","last_unstale":"2024-03-30T01:42:11.878784+0000","last_undegraded":"2024-03-30T01:42:11.878784+0000","last_fullsized":"2024-03-30T01:42:11.878784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:30.887459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878336+0000","last_change":"2024-03-30T01:42:10.300292+0000","last_active":"2024-03-30T01:42:11.878336+0000","last_peered":"2024-03-30T01:42:11.878336+0000","last_clean":"2024-03-30T01:42:11.878336+0000","last_became_active":"2024-03-30T01:42:10.299867+0000","last_became_peered":"2024-03-30T01:42:10.299867+0000","last_unstale":"2024-03-30T01:42:11.878336+0000","last_undegraded":"2024-03-30T01:42:11.878336+0000","last_fullsized":"2024-03-30T01:42:11.878336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878423+0000","last_change":"2024-03-30T01:42:09.829349+0000","last_active":"2024-03-30T01:42:11.878423+0000","last_peered":"2024-03-30T01:42:11.878423+0000","last_clean":"2024-03-30T01:42:11.878423+0000","last_became_active":"2024-03-30T01:42:09.828931+0000","last_became_peered":"2024-03-30T01:42:09.828931+0000","last_unstale":"2024-03-30T01:42:11.878423+0000","last_undegraded":"2024-03-30T01:42:11.878423+0000","last_fullsized":"2024-03-30T01:42:11.878423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878765+0000","last_change":"2024-03-30T01:42:10.291979+0000","last_active":"2024-03-30T01:42:11.878765+0000","last_peered":"2024-03-30T01:42:11.878765+0000","last_clean":"2024-03-30T01:42:11.878765+0000","last_became_active":"2024-03-30T01:42:10.291708+0000","last_became_peered":"2024-03-30T01:42:10.291708+0000","last_unstale":"2024-03-30T01:42:11.878765+0000","last_undegraded":"2024-03-30T01:42:11.878765+0000","last_fullsized":"2024-03-30T01:42:11.878765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877952+0000","last_change":"2024-03-30T01:42:09.824809+0000","last_active":"2024-03-30T01:42:11.877952+0000","last_peered":"2024-03-30T01:42:11.877952+0000","last_clean":"2024-03-30T01:42:11.877952+0000","last_became_active":"2024-03-30T01:42:09.824493+0000","last_became_peered":"2024-03-30T01:42:09.824493+0000","last_unstale":"2024-03-30T01:42:11.877952+0000","last_undegraded":"2024-03-30T01:42:11.877952+0000","last_fullsized":"2024-03-30T01:42:11.877952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877821+0000","last_change":"2024-03-30T01:42:10.287526+0000","last_active":"2024-03-30T01:42:11.877821+0000","last_peered":"2024-03-30T01:42:11.877821+0000","last_clean":"2024-03-30T01:42:11.877821+0000","last_became_active":"2024-03-30T01:42:10.287255+0000","last_became_peered":"2024-03-30T01:42:10.287255+0000","last_unstale":"2024-03-30T01:42:11.877821+0000","last_undegraded":"2024-03-30T01:42:11.877821+0000","last_fullsized":"2024-03-30T01:42:11.877821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279059+0000","last_change":"2024-03-30T01:42:09.831496+0000","last_active":"2024-03-30T01:42:11.279059+0000","last_peered":"2024-03-30T01:42:11.279059+0000","last_clean":"2024-03-30T01:42:11.279059+0000","last_became_active":"2024-03-30T01:42:09.831197+0000","last_became_peered":"2024-03-30T01:42:09.831197+0000","last_unstale":"2024-03-30T01:42:11.279059+0000","last_undegraded":"2024-03-30T01:42:11.279059+0000","last_fullsized":"2024-03-30T01:42:11.279059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278809+0000","last_change":"2024-03-30T01:42:10.289213+0000","last_active":"2024-03-30T01:42:11.278809+0000","last_peered":"2024-03-30T01:42:11.278809+0000","last_clean":"2024-03-30T01:42:11.278809+0000","last_became_active":"2024-03-30T01:42:10.288677+0000","last_became_peered":"2024-03-30T01:42:10.288677+0000","last_unstale":"2024-03-30T01:42:11.278809+0000","last_undegraded":"2024-03-30T01:42:11.278809+0000","last_fullsized":"2024-03-30T01:42:11.278809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877953+0000","last_change":"2024-03-30T01:42:09.831981+0000","last_active":"2024-03-30T01:42:11.877953+0000","last_peered":"2024-03-30T01:42:11.877953+0000","last_clean":"2024-03-30T01:42:11.877953+0000","last_became_active":"2024-03-30T01:42:09.831647+0000","last_became_peered":"2024-03-30T01:42:09.831647+0000","last_unstale":"2024-03-30T01:42:11.877953+0000","last_undegraded":"2024-03-30T01:42:11.877953+0000","last_fullsized":"2024-03-30T01:42:11.877953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:02.883477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878150+0000","last_change":"2024-03-30T01:42:10.303359+0000","last_active":"2024-03-30T01:42:11.878150+0000","last_peered":"2024-03-30T01:42:11.878150+0000","last_clean":"2024-03-30T01:42:11.878150+0000","last_became_active":"2024-03-30T01:42:10.303039+0000","last_became_peered":"2024-03-30T01:42:10.303039+0000","last_unstale":"2024-03-30T01:42:11.878150+0000","last_undegraded":"2024-03-30T01:42:11.878150+0000","last_fullsized":"2024-03-30T01:42:11.878150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878771+0000","last_change":"2024-03-30T01:42:09.826326+0000","last_active":"2024-03-30T01:42:11.878771+0000","last_peered":"2024-03-30T01:42:11.878771+0000","last_clean":"2024-03-30T01:42:11.878771+0000","last_became_active":"2024-03-30T01:42:09.826050+0000","last_became_peered":"2024-03-30T01:42:09.826050+0000","last_unstale":"2024-03-30T01:42:11.878771+0000","last_undegraded":"2024-03-30T01:42:11.878771+0000","last_fullsized":"2024-03-30T01:42:11.878771+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.879341+0000","last_change":"2024-03-30T01:42:10.291872+0000","last_active":"2024-03-30T01:42:11.879341+0000","last_peered":"2024-03-30T01:42:11.879341+0000","last_clean":"2024-03-30T01:42:11.879341+0000","last_became_active":"2024-03-30T01:42:10.291199+0000","last_became_peered":"2024-03-30T01:42:10.291199+0000","last_unstale":"2024-03-30T01:42:11.879341+0000","last_undegraded":"2024-03-30T01:42:11.879341+0000","last_fullsized":"2024-03-30T01:42:11.879341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877605+0000","last_change":"2024-03-30T01:42:09.820512+0000","last_active":"2024-03-30T01:42:11.877605+0000","last_peered":"2024-03-30T01:42:11.877605+0000","last_clean":"2024-03-30T01:42:11.877605+0000","last_became_active":"2024-03-30T01:42:09.820253+0000","last_became_peered":"2024-03-30T01:42:09.820253+0000","last_unstale":"2024-03-30T01:42:11.877605+0000","last_undegraded":"2024-03-30T01:42:11.877605+0000","last_fullsized":"2024-03-30T01:42:11.877605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.455801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278160+0000","last_change":"2024-03-30T01:42:10.295259+0000","last_active":"2024-03-30T01:42:11.278160+0000","last_peered":"2024-03-30T01:42:11.278160+0000","last_clean":"2024-03-30T01:42:11.278160+0000","last_became_active":"2024-03-30T01:42:10.295054+0000","last_became_peered":"2024-03-30T01:42:10.295054+0000","last_unstale":"2024-03-30T01:42:11.278160+0000","last_undegraded":"2024-03-30T01:42:11.278160+0000","last_fullsized":"2024-03-30T01:42:11.278160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:36.942583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878902+0000","last_change":"2024-03-30T01:42:09.828624+0000","last_active":"2024-03-30T01:42:11.878902+0000","last_peered":"2024-03-30T01:42:11.878902+0000","last_clean":"2024-03-30T01:42:11.878902+0000","last_became_active":"2024-03-30T01:42:09.828341+0000","last_became_peered":"2024-03-30T01:42:09.828341+0000","last_unstale":"2024-03-30T01:42:11.878902+0000","last_undegraded":"2024-03-30T01:42:11.878902+0000","last_fullsized":"2024-03-30T01:42:11.878902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:32.138427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878174+0000","last_change":"2024-03-30T01:42:10.284755+0000","last_active":"2024-03-30T01:42:11.878174+0000","last_peered":"2024-03-30T01:42:11.878174+0000","last_clean":"2024-03-30T01:42:11.878174+0000","last_became_active":"2024-03-30T01:42:10.284421+0000","last_became_peered":"2024-03-30T01:42:10.284421+0000","last_unstale":"2024-03-30T01:42:11.878174+0000","last_undegraded":"2024-03-30T01:42:11.878174+0000","last_fullsized":"2024-03-30T01:42:11.878174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279440+0000","last_change":"2024-03-30T01:42:09.826561+0000","last_active":"2024-03-30T01:42:11.279440+0000","last_peered":"2024-03-30T01:42:11.279440+0000","last_clean":"2024-03-30T01:42:11.279440+0000","last_became_active":"2024-03-30T01:42:09.826059+0000","last_became_peered":"2024-03-30T01:42:09.826059+0000","last_unstale":"2024-03-30T01:42:11.279440+0000","last_undegraded":"2024-03-30T01:42:11.279440+0000","last_fullsized":"2024-03-30T01:42:11.279440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877892+0000","last_change":"2024-03-30T01:42:10.292182+0000","last_active":"2024-03-30T01:42:11.877892+0000","last_peered":"2024-03-30T01:42:11.877892+0000","last_clean":"2024-03-30T01:42:11.877892+0000","last_became_active":"2024-03-30T01:42:10.291868+0000","last_became_peered":"2024-03-30T01:42:10.291868+0000","last_unstale":"2024-03-30T01:42:11.877892+0000","last_undegraded":"2024-03-30T01:42:11.877892+0000","last_fullsized":"2024-03-30T01:42:11.877892+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279386+0000","last_change":"2024-03-30T01:42:09.833439+0000","last_active":"2024-03-30T01:42:11.279386+0000","last_peered":"2024-03-30T01:42:11.279386+0000","last_clean":"2024-03-30T01:42:11.279386+0000","last_became_active":"2024-03-30T01:42:09.833137+0000","last_became_peered":"2024-03-30T01:42:09.833137+0000","last_unstale":"2024-03-30T01:42:11.279386+0000","last_undegraded":"2024-03-30T01:42:11.279386+0000","last_fullsized":"2024-03-30T01:42:11.279386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877995+0000","last_change":"2024-03-30T01:42:10.296944+0000","last_active":"2024-03-30T01:42:11.877995+0000","last_peered":"2024-03-30T01:42:11.877995+0000","last_clean":"2024-03-30T01:42:11.877995+0000","last_became_active":"2024-03-30T01:42:10.296714+0000","last_became_peered":"2024-03-30T01:42:10.296714+0000","last_unstale":"2024-03-30T01:42:11.877995+0000","last_undegraded":"2024-03-30T01:42:11.877995+0000","last_fullsized":"2024-03-30T01:42:11.877995+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878974+0000","last_change":"2024-03-30T01:42:09.823152+0000","last_active":"2024-03-30T01:42:11.878974+0000","last_peered":"2024-03-30T01:42:11.878974+0000","last_clean":"2024-03-30T01:42:11.878974+0000","last_became_active":"2024-03-30T01:42:09.822826+0000","last_became_peered":"2024-03-30T01:42:09.822826+0000","last_unstale":"2024-03-30T01:42:11.878974+0000","last_undegraded":"2024-03-30T01:42:11.878974+0000","last_fullsized":"2024-03-30T01:42:11.878974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:17.799215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877695+0000","last_change":"2024-03-30T01:42:10.285053+0000","last_active":"2024-03-30T01:42:11.877695+0000","last_peered":"2024-03-30T01:42:11.877695+0000","last_clean":"2024-03-30T01:42:11.877695+0000","last_became_active":"2024-03-30T01:42:10.284814+0000","last_became_peered":"2024-03-30T01:42:10.284814+0000","last_unstale":"2024-03-30T01:42:11.877695+0000","last_undegraded":"2024-03-30T01:42:11.877695+0000","last_fullsized":"2024-03-30T01:42:11.877695+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:28.875197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878965+0000","last_change":"2024-03-30T01:42:09.832318+0000","last_active":"2024-03-30T01:42:11.878965+0000","last_peered":"2024-03-30T01:42:11.878965+0000","last_clean":"2024-03-30T01:42:11.878965+0000","last_became_active":"2024-03-30T01:42:09.831964+0000","last_became_peered":"2024-03-30T01:42:09.831964+0000","last_unstale":"2024-03-30T01:42:11.878965+0000","last_undegraded":"2024-03-30T01:42:11.878965+0000","last_fullsized":"2024-03-30T01:42:11.878965+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:05.700970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878486+0000","last_change":"2024-03-30T01:42:10.288326+0000","last_active":"2024-03-30T01:42:11.878486+0000","last_peered":"2024-03-30T01:42:11.878486+0000","last_clean":"2024-03-30T01:42:11.878486+0000","last_became_active":"2024-03-30T01:42:10.288050+0000","last_became_peered":"2024-03-30T01:42:10.288050+0000","last_unstale":"2024-03-30T01:42:11.878486+0000","last_undegraded":"2024-03-30T01:42:11.878486+0000","last_fullsized":"2024-03-30T01:42:11.878486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878573+0000","last_change":"2024-03-30T01:42:09.829194+0000","last_active":"2024-03-30T01:42:11.878573+0000","last_peered":"2024-03-30T01:42:11.878573+0000","last_clean":"2024-03-30T01:42:11.878573+0000","last_became_active":"2024-03-30T01:42:09.828257+0000","last_became_peered":"2024-03-30T01:42:09.828257+0000","last_unstale":"2024-03-30T01:42:11.878573+0000","last_undegraded":"2024-03-30T01:42:11.878573+0000","last_fullsized":"2024-03-30T01:42:11.878573+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877821+0000","last_change":"2024-03-30T01:42:10.279681+0000","last_active":"2024-03-30T01:42:11.877821+0000","last_peered":"2024-03-30T01:42:11.877821+0000","last_clean":"2024-03-30T01:42:11.877821+0000","last_became_active":"2024-03-30T01:42:10.279333+0000","last_became_peered":"2024-03-30T01:42:10.279333+0000","last_unstale":"2024-03-30T01:42:11.877821+0000","last_undegraded":"2024-03-30T01:42:11.877821+0000","last_fullsized":"2024-03-30T01:42:11.877821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277537+0000","last_change":"2024-03-30T01:42:09.820451+0000","last_active":"2024-03-30T01:42:11.277537+0000","last_peered":"2024-03-30T01:42:11.277537+0000","last_clean":"2024-03-30T01:42:11.277537+0000","last_became_active":"2024-03-30T01:42:09.820152+0000","last_became_peered":"2024-03-30T01:42:09.820152+0000","last_unstale":"2024-03-30T01:42:11.277537+0000","last_undegraded":"2024-03-30T01:42:11.277537+0000","last_fullsized":"2024-03-30T01:42:11.277537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:04.153303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878324+0000","last_change":"2024-03-30T01:42:10.288399+0000","last_active":"2024-03-30T01:42:11.878324+0000","last_peered":"2024-03-30T01:42:11.878324+0000","last_clean":"2024-03-30T01:42:11.878324+0000","last_became_active":"2024-03-30T01:42:10.287890+0000","last_became_peered":"2024-03-30T01:42:10.287890+0000","last_unstale":"2024-03-30T01:42:11.878324+0000","last_undegraded":"2024-03-30T01:42:11.878324+0000","last_fullsized":"2024-03-30T01:42:11.878324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877618+0000","last_change":"2024-03-30T01:42:09.831869+0000","last_active":"2024-03-30T01:42:11.877618+0000","last_peered":"2024-03-30T01:42:11.877618+0000","last_clean":"2024-03-30T01:42:11.877618+0000","last_became_active":"2024-03-30T01:42:09.831562+0000","last_became_peered":"2024-03-30T01:42:09.831562+0000","last_unstale":"2024-03-30T01:42:11.877618+0000","last_undegraded":"2024-03-30T01:42:11.877618+0000","last_fullsized":"2024-03-30T01:42:11.877618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:35.938001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878831+0000","last_change":"2024-03-30T01:42:10.291564+0000","last_active":"2024-03-30T01:42:11.878831+0000","last_peered":"2024-03-30T01:42:11.878831+0000","last_clean":"2024-03-30T01:42:11.878831+0000","last_became_active":"2024-03-30T01:42:10.291285+0000","last_became_peered":"2024-03-30T01:42:10.291285+0000","last_unstale":"2024-03-30T01:42:11.878831+0000","last_undegraded":"2024-03-30T01:42:11.878831+0000","last_fullsized":"2024-03-30T01:42:11.878831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279538+0000","last_change":"2024-03-30T01:42:09.826895+0000","last_active":"2024-03-30T01:42:11.279538+0000","last_peered":"2024-03-30T01:42:11.279538+0000","last_clean":"2024-03-30T01:42:11.279538+0000","last_became_active":"2024-03-30T01:42:09.826609+0000","last_became_peered":"2024-03-30T01:42:09.826609+0000","last_unstale":"2024-03-30T01:42:11.279538+0000","last_undegraded":"2024-03-30T01:42:11.279538+0000","last_fullsized":"2024-03-30T01:42:11.279538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:42:59.036073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279123+0000","last_change":"2024-03-30T01:42:10.295516+0000","last_active":"2024-03-30T01:42:11.279123+0000","last_peered":"2024-03-30T01:42:11.279123+0000","last_clean":"2024-03-30T01:42:11.279123+0000","last_became_active":"2024-03-30T01:42:10.295133+0000","last_became_peered":"2024-03-30T01:42:10.295133+0000","last_unstale":"2024-03-30T01:42:11.279123+0000","last_undegraded":"2024-03-30T01:42:11.279123+0000","last_fullsized":"2024-03-30T01:42:11.279123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878440+0000","last_change":"2024-03-30T01:42:09.829713+0000","last_active":"2024-03-30T01:42:11.878440+0000","last_peered":"2024-03-30T01:42:11.878440+0000","last_clean":"2024-03-30T01:42:11.878440+0000","last_became_active":"2024-03-30T01:42:09.829375+0000","last_became_peered":"2024-03-30T01:42:09.829375+0000","last_unstale":"2024-03-30T01:42:11.878440+0000","last_undegraded":"2024-03-30T01:42:11.878440+0000","last_fullsized":"2024-03-30T01:42:11.878440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:49:51.601663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878538+0000","last_change":"2024-03-30T01:42:10.304085+0000","last_active":"2024-03-30T01:42:11.878538+0000","last_peered":"2024-03-30T01:42:11.878538+0000","last_clean":"2024-03-30T01:42:11.878538+0000","last_became_active":"2024-03-30T01:42:10.303817+0000","last_became_peered":"2024-03-30T01:42:10.303817+0000","last_unstale":"2024-03-30T01:42:11.878538+0000","last_undegraded":"2024-03-30T01:42:11.878538+0000","last_fullsized":"2024-03-30T01:42:11.878538+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278117+0000","last_change":"2024-03-30T01:42:09.823221+0000","last_active":"2024-03-30T01:42:11.278117+0000","last_peered":"2024-03-30T01:42:11.278117+0000","last_clean":"2024-03-30T01:42:11.278117+0000","last_became_active":"2024-03-30T01:42:09.822882+0000","last_became_peered":"2024-03-30T01:42:09.822882+0000","last_unstale":"2024-03-30T01:42:11.278117+0000","last_undegraded":"2024-03-30T01:42:11.278117+0000","last_fullsized":"2024-03-30T01:42:11.278117+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:00:31.185685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278271+0000","last_change":"2024-03-30T01:42:10.286900+0000","last_active":"2024-03-30T01:42:11.278271+0000","last_peered":"2024-03-30T01:42:11.278271+0000","last_clean":"2024-03-30T01:42:11.278271+0000","last_became_active":"2024-03-30T01:42:10.286598+0000","last_became_peered":"2024-03-30T01:42:10.286598+0000","last_unstale":"2024-03-30T01:42:11.278271+0000","last_undegraded":"2024-03-30T01:42:11.278271+0000","last_fullsized":"2024-03-30T01:42:11.278271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:46.592007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878240+0000","last_change":"2024-03-30T01:42:09.828572+0000","last_active":"2024-03-30T01:42:11.878240+0000","last_peered":"2024-03-30T01:42:11.878240+0000","last_clean":"2024-03-30T01:42:11.878240+0000","last_became_active":"2024-03-30T01:42:09.828225+0000","last_became_peered":"2024-03-30T01:42:09.828225+0000","last_unstale":"2024-03-30T01:42:11.878240+0000","last_undegraded":"2024-03-30T01:42:11.878240+0000","last_fullsized":"2024-03-30T01:42:11.878240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877568+0000","last_change":"2024-03-30T01:42:10.279913+0000","last_active":"2024-03-30T01:42:11.877568+0000","last_peered":"2024-03-30T01:42:11.877568+0000","last_clean":"2024-03-30T01:42:11.877568+0000","last_became_active":"2024-03-30T01:42:10.279450+0000","last_became_peered":"2024-03-30T01:42:10.279450+0000","last_unstale":"2024-03-30T01:42:11.877568+0000","last_undegraded":"2024-03-30T01:42:11.877568+0000","last_fullsized":"2024-03-30T01:42:11.877568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278159+0000","last_change":"2024-03-30T01:42:09.830452+0000","last_active":"2024-03-30T01:42:11.278159+0000","last_peered":"2024-03-30T01:42:11.278159+0000","last_clean":"2024-03-30T01:42:11.278159+0000","last_became_active":"2024-03-30T01:42:09.830173+0000","last_became_peered":"2024-03-30T01:42:09.830173+0000","last_unstale":"2024-03-30T01:42:11.278159+0000","last_undegraded":"2024-03-30T01:42:11.278159+0000","last_fullsized":"2024-03-30T01:42:11.278159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:12:42.807604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878090+0000","last_change":"2024-03-30T01:42:10.290469+0000","last_active":"2024-03-30T01:42:11.878090+0000","last_peered":"2024-03-30T01:42:11.878090+0000","last_clean":"2024-03-30T01:42:11.878090+0000","last_became_active":"2024-03-30T01:42:10.290148+0000","last_became_peered":"2024-03-30T01:42:10.290148+0000","last_unstale":"2024-03-30T01:42:11.878090+0000","last_undegraded":"2024-03-30T01:42:11.878090+0000","last_fullsized":"2024-03-30T01:42:11.878090+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278053+0000","last_change":"2024-03-30T01:42:09.826256+0000","last_active":"2024-03-30T01:42:11.278053+0000","last_peered":"2024-03-30T01:42:11.278053+0000","last_clean":"2024-03-30T01:42:11.278053+0000","last_became_active":"2024-03-30T01:42:09.825956+0000","last_became_peered":"2024-03-30T01:42:09.825956+0000","last_unstale":"2024-03-30T01:42:11.278053+0000","last_undegraded":"2024-03-30T01:42:11.278053+0000","last_fullsized":"2024-03-30T01:42:11.278053+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:59:41.446927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279087+0000","last_change":"2024-03-30T01:42:10.285649+0000","last_active":"2024-03-30T01:42:11.279087+0000","last_peered":"2024-03-30T01:42:11.279087+0000","last_clean":"2024-03-30T01:42:11.279087+0000","last_became_active":"2024-03-30T01:42:10.285297+0000","last_became_peered":"2024-03-30T01:42:10.285297+0000","last_unstale":"2024-03-30T01:42:11.279087+0000","last_undegraded":"2024-03-30T01:42:11.279087+0000","last_fullsized":"2024-03-30T01:42:11.279087+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878281+0000","last_change":"2024-03-30T01:42:09.829777+0000","last_active":"2024-03-30T01:42:11.878281+0000","last_peered":"2024-03-30T01:42:11.878281+0000","last_clean":"2024-03-30T01:42:11.878281+0000","last_became_active":"2024-03-30T01:42:09.829483+0000","last_became_peered":"2024-03-30T01:42:09.829483+0000","last_unstale":"2024-03-30T01:42:11.878281+0000","last_undegraded":"2024-03-30T01:42:11.878281+0000","last_fullsized":"2024-03-30T01:42:11.878281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:45.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878808+0000","last_change":"2024-03-30T01:42:10.292043+0000","last_active":"2024-03-30T01:42:11.878808+0000","last_peered":"2024-03-30T01:42:11.878808+0000","last_clean":"2024-03-30T01:42:11.878808+0000","last_became_active":"2024-03-30T01:42:10.291598+0000","last_became_peered":"2024-03-30T01:42:10.291598+0000","last_unstale":"2024-03-30T01:42:11.878808+0000","last_undegraded":"2024-03-30T01:42:11.878808+0000","last_fullsized":"2024-03-30T01:42:11.878808+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277994+0000","last_change":"2024-03-30T01:42:09.829638+0000","last_active":"2024-03-30T01:42:11.277994+0000","last_peered":"2024-03-30T01:42:11.277994+0000","last_clean":"2024-03-30T01:42:11.277994+0000","last_became_active":"2024-03-30T01:42:09.829317+0000","last_became_peered":"2024-03-30T01:42:09.829317+0000","last_unstale":"2024-03-30T01:42:11.277994+0000","last_undegraded":"2024-03-30T01:42:11.277994+0000","last_fullsized":"2024-03-30T01:42:11.277994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:33.639097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878108+0000","last_change":"2024-03-30T01:42:10.298216+0000","last_active":"2024-03-30T01:42:11.878108+0000","last_peered":"2024-03-30T01:42:11.878108+0000","last_clean":"2024-03-30T01:42:11.878108+0000","last_became_active":"2024-03-30T01:42:10.297976+0000","last_became_peered":"2024-03-30T01:42:10.297976+0000","last_unstale":"2024-03-30T01:42:11.878108+0000","last_undegraded":"2024-03-30T01:42:11.878108+0000","last_fullsized":"2024-03-30T01:42:11.878108+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279604+0000","last_change":"2024-03-30T01:42:09.833189+0000","last_active":"2024-03-30T01:42:11.279604+0000","last_peered":"2024-03-30T01:42:11.279604+0000","last_clean":"2024-03-30T01:42:11.279604+0000","last_became_active":"2024-03-30T01:42:09.832891+0000","last_became_peered":"2024-03-30T01:42:09.832891+0000","last_unstale":"2024-03-30T01:42:11.279604+0000","last_undegraded":"2024-03-30T01:42:11.279604+0000","last_fullsized":"2024-03-30T01:42:11.279604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878290+0000","last_change":"2024-03-30T01:42:10.284232+0000","last_active":"2024-03-30T01:42:11.878290+0000","last_peered":"2024-03-30T01:42:11.878290+0000","last_clean":"2024-03-30T01:42:11.878290+0000","last_became_active":"2024-03-30T01:42:10.283909+0000","last_became_peered":"2024-03-30T01:42:10.283909+0000","last_unstale":"2024-03-30T01:42:11.878290+0000","last_undegraded":"2024-03-30T01:42:11.878290+0000","last_fullsized":"2024-03-30T01:42:11.878290+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:33.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.959456+0000","last_change":"2024-03-30T01:42:09.824373+0000","last_active":"2024-03-30T01:42:11.959456+0000","last_peered":"2024-03-30T01:42:11.959456+0000","last_clean":"2024-03-30T01:42:11.959456+0000","last_became_active":"2024-03-30T01:42:09.824089+0000","last_became_peered":"2024-03-30T01:42:09.824089+0000","last_unstale":"2024-03-30T01:42:11.959456+0000","last_undegraded":"2024-03-30T01:42:11.959456+0000","last_fullsized":"2024-03-30T01:42:11.959456+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:37.434399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878956+0000","last_change":"2024-03-30T01:42:10.299986+0000","last_active":"2024-03-30T01:42:11.878956+0000","last_peered":"2024-03-30T01:42:11.878956+0000","last_clean":"2024-03-30T01:42:11.878956+0000","last_became_active":"2024-03-30T01:42:10.299776+0000","last_became_peered":"2024-03-30T01:42:10.299776+0000","last_unstale":"2024-03-30T01:42:11.878956+0000","last_undegraded":"2024-03-30T01:42:11.878956+0000","last_fullsized":"2024-03-30T01:42:11.878956+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878497+0000","last_change":"2024-03-30T01:42:09.830490+0000","last_active":"2024-03-30T01:42:11.878497+0000","last_peered":"2024-03-30T01:42:11.878497+0000","last_clean":"2024-03-30T01:42:11.878497+0000","last_became_active":"2024-03-30T01:42:09.830197+0000","last_became_peered":"2024-03-30T01:42:09.830197+0000","last_unstale":"2024-03-30T01:42:11.878497+0000","last_undegraded":"2024-03-30T01:42:11.878497+0000","last_fullsized":"2024-03-30T01:42:11.878497+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:46.433516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877767+0000","last_change":"2024-03-30T01:42:10.288334+0000","last_active":"2024-03-30T01:42:11.877767+0000","last_peered":"2024-03-30T01:42:11.877767+0000","last_clean":"2024-03-30T01:42:11.877767+0000","last_became_active":"2024-03-30T01:42:10.288093+0000","last_became_peered":"2024-03-30T01:42:10.288093+0000","last_unstale":"2024-03-30T01:42:11.877767+0000","last_undegraded":"2024-03-30T01:42:11.877767+0000","last_fullsized":"2024-03-30T01:42:11.877767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878858+0000","last_change":"2024-03-30T01:42:09.827116+0000","last_active":"2024-03-30T01:42:11.878858+0000","last_peered":"2024-03-30T01:42:11.878858+0000","last_clean":"2024-03-30T01:42:11.878858+0000","last_became_active":"2024-03-30T01:42:09.826876+0000","last_became_peered":"2024-03-30T01:42:09.826876+0000","last_unstale":"2024-03-30T01:42:11.878858+0000","last_undegraded":"2024-03-30T01:42:11.878858+0000","last_fullsized":"2024-03-30T01:42:11.878858+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:38.792762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878701+0000","last_change":"2024-03-30T01:42:10.291029+0000","last_active":"2024-03-30T01:42:11.878701+0000","last_peered":"2024-03-30T01:42:11.878701+0000","last_clean":"2024-03-30T01:42:11.878701+0000","last_became_active":"2024-03-30T01:42:10.290728+0000","last_became_peered":"2024-03-30T01:42:10.290728+0000","last_unstale":"2024-03-30T01:42:11.878701+0000","last_undegraded":"2024-03-30T01:42:11.878701+0000","last_fullsized":"2024-03-30T01:42:11.878701+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277776+0000","last_change":"2024-03-30T01:42:09.822074+0000","last_active":"2024-03-30T01:42:11.277776+0000","last_peered":"2024-03-30T01:42:11.277776+0000","last_clean":"2024-03-30T01:42:11.277776+0000","last_became_active":"2024-03-30T01:42:09.821786+0000","last_became_peered":"2024-03-30T01:42:09.821786+0000","last_unstale":"2024-03-30T01:42:11.277776+0000","last_undegraded":"2024-03-30T01:42:11.277776+0000","last_fullsized":"2024-03-30T01:42:11.277776+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:18.840885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878277+0000","last_change":"2024-03-30T01:42:10.299170+0000","last_active":"2024-03-30T01:42:11.878277+0000","last_peered":"2024-03-30T01:42:11.878277+0000","last_clean":"2024-03-30T01:42:11.878277+0000","last_became_active":"2024-03-30T01:42:10.298846+0000","last_became_peered":"2024-03-30T01:42:10.298846+0000","last_unstale":"2024-03-30T01:42:11.878277+0000","last_undegraded":"2024-03-30T01:42:11.878277+0000","last_fullsized":"2024-03-30T01:42:11.878277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878295+0000","last_change":"2024-03-30T01:42:09.832991+0000","last_active":"2024-03-30T01:42:11.878295+0000","last_peered":"2024-03-30T01:42:11.878295+0000","last_clean":"2024-03-30T01:42:11.878295+0000","last_became_active":"2024-03-30T01:42:09.832682+0000","last_became_peered":"2024-03-30T01:42:09.832682+0000","last_unstale":"2024-03-30T01:42:11.878295+0000","last_undegraded":"2024-03-30T01:42:11.878295+0000","last_fullsized":"2024-03-30T01:42:11.878295+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:37.427434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877952+0000","last_change":"2024-03-30T01:42:10.284566+0000","last_active":"2024-03-30T01:42:11.877952+0000","last_peered":"2024-03-30T01:42:11.877952+0000","last_clean":"2024-03-30T01:42:11.877952+0000","last_became_active":"2024-03-30T01:42:10.284192+0000","last_became_peered":"2024-03-30T01:42:10.284192+0000","last_unstale":"2024-03-30T01:42:11.877952+0000","last_undegraded":"2024-03-30T01:42:11.877952+0000","last_fullsized":"2024-03-30T01:42:11.877952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:05.460383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878466+0000","last_change":"2024-03-30T01:42:09.829174+0000","last_active":"2024-03-30T01:42:11.878466+0000","last_peered":"2024-03-30T01:42:11.878466+0000","last_clean":"2024-03-30T01:42:11.878466+0000","last_became_active":"2024-03-30T01:42:09.828730+0000","last_became_peered":"2024-03-30T01:42:09.828730+0000","last_unstale":"2024-03-30T01:42:11.878466+0000","last_undegraded":"2024-03-30T01:42:11.878466+0000","last_fullsized":"2024-03-30T01:42:11.878466+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279516+0000","last_change":"2024-03-30T01:42:10.289721+0000","last_active":"2024-03-30T01:42:11.279516+0000","last_peered":"2024-03-30T01:42:11.279516+0000","last_clean":"2024-03-30T01:42:11.279516+0000","last_became_active":"2024-03-30T01:42:10.289413+0000","last_became_peered":"2024-03-30T01:42:10.289413+0000","last_unstale":"2024-03-30T01:42:11.279516+0000","last_undegraded":"2024-03-30T01:42:11.279516+0000","last_fullsized":"2024-03-30T01:42:11.279516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878595+0000","last_change":"2024-03-30T01:42:09.836532+0000","last_active":"2024-03-30T01:42:11.878595+0000","last_peered":"2024-03-30T01:42:11.878595+0000","last_clean":"2024-03-30T01:42:11.878595+0000","last_became_active":"2024-03-30T01:42:09.836262+0000","last_became_peered":"2024-03-30T01:42:09.836262+0000","last_unstale":"2024-03-30T01:42:11.878595+0000","last_undegraded":"2024-03-30T01:42:11.878595+0000","last_fullsized":"2024-03-30T01:42:11.878595+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:08.249431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T01:42:10.271303+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:42:10.271303+0000","last_peered":"2024-03-30T01:42:10.271303+0000","last_clean":"2024-03-30T01:42:10.271303+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:42:10.271303+0000","last_undegraded":"2024-03-30T01:42:10.271303+0000","last_fullsized":"2024-03-30T01:42:10.271303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":786117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.624}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.72699999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.2009999999999996,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.441}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65600000000000003}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.90600000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.82499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.70799999999999996}]},{"osd":1,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.3250000000000002,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.68300000000000005}]},{"osd":2,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.66300000000000003}]},{"osd":3,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.503},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61199999999999999}]},{"osd":5,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.72299999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":15.038,"5min":15.038,"15min":15.038},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":11.858000000000001,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.33500000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.434},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1289999999999996,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.377}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":11.99,"5min":11.99,"15min":11.99},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":15.006,"5min":15.006,"15min":15.006},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":11.926,"5min":11.926,"15min":11.926},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":14.974,"5min":14.974,"15min":14.974},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.960000000000001,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":14.923999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.48899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":783548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.946999999999999,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.33400000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":12.983000000000001,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.2279999999999998,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.59599999999999997}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.3869999999999996,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.75800000000000001}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.627},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":5.468,"5min":5.468,"15min":5.468},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":12.478999999999999,"5min":12.478999999999999,"15min":12.478999999999999},"last":2.1890000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":2.1469999999999998}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":12.425000000000001,"5min":12.425000000000001,"15min":12.425000000000001},"last":2.1309999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":13.217000000000001,"5min":13.217000000000001,"15min":13.217000000000001},"last":1.8759999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":8.7400000000000002,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":1.956},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":8.5640000000000001,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":2.0630000000000002}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":8.7759999999999998,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":1.9159999999999999}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":8.5190000000000001,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":2.1139999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":3.1829999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.503},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":8.5939999999999994,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":3.105}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":195713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.375}]},{"osd":1,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.40200000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.442}]},{"osd":4,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.48099999999999998}]},{"osd":5,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.5},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.63400000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.504},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":9.3970000000000002,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.627},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.42899999999999999}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":5.5289999999999999,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":9.452,"5min":9.452,"15min":9.452},"last":0.57099999999999995}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47799999999999998}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.45400000000000001}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":6.6760000000000002,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.88200000000000001}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7480000000000002,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.7709999999999999,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.94099999999999995}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.8410000000000002,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.77000000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.72499999999999998}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.85299999999999998}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64900000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:42:14.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:42:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.680+0000 7fc87db3b700 1 Processor -- start 2024-03-30T01:42:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.680+0000 7fc87db3b700 1 -- start start 2024-03-30T01:42:14.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.681+0000 7fc87db3b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8780c8310 0x7fc8780c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.681+0000 7fc87db3b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8780c5df0 0x7fc878151d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.681+0000 7fc87db3b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8781556d0 con 0x7fc8780c2870 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.681+0000 7fc87db3b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8781559b0 con 0x7fc8780c8310 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.681+0000 7fc87db3b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8780c1f70 con 0x7fc8780c5df0 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.681+0000 7fc8777fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8780c8310 0x7fc8780c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.681+0000 7fc8777fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8780c8310 0x7fc8780c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47336/0 (socket says 172.21.15.31:47336) 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.681+0000 7fc8777fe700 1 -- 172.21.15.31:0/1561623377 learned_addr learned my addr 172.21.15.31:0/1561623377 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.681+0000 7fc877fff700 1 --1- 172.21.15.31:0/1561623377 >> v1:172.21.15.31:6789/0 conn(0x7fc8780c2870 0x7fc8780a6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41676/0 (socket says 172.21.15.31:41676) 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc876ffd700 1 --2- 172.21.15.31:0/1561623377 >> v2:172.21.15.59:3300/0 conn(0x7fc8780c5df0 0x7fc878151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc8777fe700 1 -- 172.21.15.31:0/1561623377 >> v2:172.21.15.59:3300/0 conn(0x7fc8780c5df0 msgr2=0x7fc878151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc8777fe700 1 --2- 172.21.15.31:0/1561623377 >> v2:172.21.15.59:3300/0 conn(0x7fc8780c5df0 0x7fc878151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc8777fe700 1 -- 172.21.15.31:0/1561623377 >> v1:172.21.15.31:6789/0 conn(0x7fc8780c2870 legacy=0x7fc8780a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc8777fe700 1 -- 172.21.15.31:0/1561623377 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780c2250 con 0x7fc8780c8310 2024-03-30T01:42:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc8767fc700 1 -- 172.21.15.31:0/1561623377 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 214251187 0 0) 0x7fc8781556d0 con 0x7fc8780c2870 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc8777fe700 1 --2- 172.21.15.31:0/1561623377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c8310 0x7fc8780c86f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc868009920 tx=0x7fc868009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc34f85ea26b88c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc8767fc700 1 -- 172.21.15.31:0/1561623377 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800b9e0 con 0x7fc8780c8310 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc8767fc700 1 -- 172.21.15.31:0/1561623377 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86800bb40 con 0x7fc8780c8310 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.682+0000 7fc8767fc700 1 -- 172.21.15.31:0/1561623377 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868009e70 con 0x7fc8780c8310 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.683+0000 7fc87db3b700 1 -- 172.21.15.31:0/1561623377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c8310 msgr2=0x7fc8780c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.683+0000 7fc87db3b700 1 --2- 172.21.15.31:0/1561623377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c8310 0x7fc8780c86f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc868009920 tx=0x7fc868009c30 comp rx=0 tx=0).stop 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.683+0000 7fc87db3b700 1 -- 172.21.15.31:0/1561623377 shutdown_connections 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.683+0000 7fc87db3b700 1 --2- 172.21.15.31:0/1561623377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c8310 0x7fc8780c86f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.683+0000 7fc87db3b700 1 --2- 172.21.15.31:0/1561623377 >> v2:172.21.15.59:3300/0 conn(0x7fc8780c5df0 0x7fc878151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.683+0000 7fc87db3b700 1 -- 172.21.15.31:0/1561623377 >> 172.21.15.31:0/1561623377 conn(0x7fc87806ab20 msgr2=0x7fc87806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.683+0000 7fc87db3b700 1 -- 172.21.15.31:0/1561623377 shutdown_connections 2024-03-30T01:42:14.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.683+0000 7fc87db3b700 1 -- 172.21.15.31:0/1561623377 wait complete. 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.684+0000 7fc87db3b700 1 Processor -- start 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.684+0000 7fc87db3b700 1 -- start start 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc87db3b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8780c2870 0x7fc87814e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc87db3b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c5df0 0x7fc87814e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc87db3b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8780c8310 0x7fc878157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc87db3b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87806e8d0 con 0x7fc8780c5df0 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc87db3b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780aae80 con 0x7fc8780c8310 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc87db3b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87806e5f0 con 0x7fc8780c2870 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc876ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c5df0 0x7fc87814e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc876ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c5df0 0x7fc87814e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47348/0 (socket says 172.21.15.31:47348) 2024-03-30T01:42:14.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc876ffd700 1 -- 172.21.15.31:0/2892459333 learned_addr learned my addr 172.21.15.31:0/2892459333 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc8777fe700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8780c2870 0x7fc87814e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.685+0000 7fc877fff700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8780c8310 0x7fc878157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc876ffd700 1 -- 172.21.15.31:0/2892459333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8780c2870 msgr2=0x7fc87814e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc876ffd700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8780c2870 0x7fc87814e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc876ffd700 1 -- 172.21.15.31:0/2892459333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8780c8310 msgr2=0x7fc878157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc876ffd700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8780c8310 0x7fc878157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc876ffd700 1 -- 172.21.15.31:0/2892459333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868009550 con 0x7fc8780c5df0 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc876ffd700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c5df0 0x7fc87814e630 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00e810 tx=0x7fc86c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc85ffff700 1 -- 172.21.15.31:0/2892459333 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00c070 con 0x7fc8780c5df0 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc85ffff700 1 -- 172.21.15.31:0/2892459333 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c013610 con 0x7fc8780c5df0 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc85ffff700 1 -- 172.21.15.31:0/2892459333 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0139c0 con 0x7fc8780c5df0 2024-03-30T01:42:14.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc878155e30 con 0x7fc8780c5df0 2024-03-30T01:42:14.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.686+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc878156110 con 0x7fc8780c5df0 2024-03-30T01:42:14.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.687+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc878073760 con 0x7fc8780c5df0 2024-03-30T01:42:14.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.688+0000 7fc85ffff700 1 -- 172.21.15.31:0/2892459333 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc86c013b20 con 0x7fc8780c5df0 2024-03-30T01:42:14.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.688+0000 7fc85ffff700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc86406ac90 0x7fc86406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:14.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.688+0000 7fc85ffff700 1 -- 172.21.15.31:0/2892459333 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc86c0891c0 con 0x7fc8780c5df0 2024-03-30T01:42:14.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.688+0000 7fc8777fe700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc86406ac90 0x7fc86406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:14.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.689+0000 7fc8777fe700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc86406ac90 0x7fc86406d150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc868009450 tx=0x7fc868008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:14.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.693+0000 7fc85ffff700 1 -- 172.21.15.31:0/2892459333 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc86c054270 con 0x7fc8780c5df0 2024-03-30T01:42:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.905+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8780c2f60 con 0x7fc86406ac90 2024-03-30T01:42:14.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.914+0000 7fc85ffff700 1 -- 172.21.15.31:0/2892459333 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288419 (secure 0 0 0) 0x7fc86800bcb0 con 0x7fc86406ac90 2024-03-30T01:42:14.924 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc86406ac90 msgr2=0x7fc86406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc86406ac90 0x7fc86406d150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc868009450 tx=0x7fc868008030 comp rx=0 tx=0).stop 2024-03-30T01:42:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c5df0 msgr2=0x7fc87814e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c5df0 0x7fc87814e630 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00e810 tx=0x7fc86c00eb20 comp rx=0 tx=0).stop 2024-03-30T01:42:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 shutdown_connections 2024-03-30T01:42:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc86406ac90 0x7fc86406d150 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8780c8310 0x7fc878157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8780c2870 0x7fc87814e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 --2- 172.21.15.31:0/2892459333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8780c5df0 0x7fc87814e630 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 >> 172.21.15.31:0/2892459333 conn(0x7fc87806ab20 msgr2=0x7fc8780a91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 shutdown_connections 2024-03-30T01:42:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:14.917+0000 7fc87db3b700 1 -- 172.21.15.31:0/2892459333 wait complete. 2024-03-30T01:42:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:42:14.937 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-03-30T01:42:13.825078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220948,"kb_used_data":4740,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753772,"statfs":{"total":773094113280,"available":772867862528,"internally_reserved":0,"allocated":4853760,"data_stored":2737411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.019887"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878628+0000","last_change":"2024-03-30T01:42:10.288258+0000","last_active":"2024-03-30T01:42:11.878628+0000","last_peered":"2024-03-30T01:42:11.878628+0000","last_clean":"2024-03-30T01:42:11.878628+0000","last_became_active":"2024-03-30T01:42:10.287993+0000","last_became_peered":"2024-03-30T01:42:10.287993+0000","last_unstale":"2024-03-30T01:42:11.878628+0000","last_undegraded":"2024-03-30T01:42:11.878628+0000","last_fullsized":"2024-03-30T01:42:11.878628+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878866+0000","last_change":"2024-03-30T01:42:10.294965+0000","last_active":"2024-03-30T01:42:11.878866+0000","last_peered":"2024-03-30T01:42:11.878866+0000","last_clean":"2024-03-30T01:42:11.878866+0000","last_became_active":"2024-03-30T01:42:10.294685+0000","last_became_peered":"2024-03-30T01:42:10.294685+0000","last_unstale":"2024-03-30T01:42:11.878866+0000","last_undegraded":"2024-03-30T01:42:11.878866+0000","last_fullsized":"2024-03-30T01:42:11.878866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878403+0000","last_change":"2024-03-30T01:42:10.285050+0000","last_active":"2024-03-30T01:42:11.878403+0000","last_peered":"2024-03-30T01:42:11.878403+0000","last_clean":"2024-03-30T01:42:11.878403+0000","last_became_active":"2024-03-30T01:42:10.284771+0000","last_became_peered":"2024-03-30T01:42:10.284771+0000","last_unstale":"2024-03-30T01:42:11.878403+0000","last_undegraded":"2024-03-30T01:42:11.878403+0000","last_fullsized":"2024-03-30T01:42:11.878403+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278035+0000","last_change":"2024-03-30T01:42:10.294277+0000","last_active":"2024-03-30T01:42:11.278035+0000","last_peered":"2024-03-30T01:42:11.278035+0000","last_clean":"2024-03-30T01:42:11.278035+0000","last_became_active":"2024-03-30T01:42:10.293995+0000","last_became_peered":"2024-03-30T01:42:10.293995+0000","last_unstale":"2024-03-30T01:42:11.278035+0000","last_undegraded":"2024-03-30T01:42:11.278035+0000","last_fullsized":"2024-03-30T01:42:11.278035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:03.888060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277927+0000","last_change":"2024-03-30T01:42:10.299868+0000","last_active":"2024-03-30T01:42:11.277927+0000","last_peered":"2024-03-30T01:42:11.277927+0000","last_clean":"2024-03-30T01:42:11.277927+0000","last_became_active":"2024-03-30T01:42:10.299612+0000","last_became_peered":"2024-03-30T01:42:10.299612+0000","last_unstale":"2024-03-30T01:42:11.277927+0000","last_undegraded":"2024-03-30T01:42:11.277927+0000","last_fullsized":"2024-03-30T01:42:11.277927+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:49.471045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878254+0000","last_change":"2024-03-30T01:42:10.287001+0000","last_active":"2024-03-30T01:42:11.878254+0000","last_peered":"2024-03-30T01:42:11.878254+0000","last_clean":"2024-03-30T01:42:11.878254+0000","last_became_active":"2024-03-30T01:42:10.286700+0000","last_became_peered":"2024-03-30T01:42:10.286700+0000","last_unstale":"2024-03-30T01:42:11.878254+0000","last_undegraded":"2024-03-30T01:42:11.878254+0000","last_fullsized":"2024-03-30T01:42:11.878254+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279027+0000","last_change":"2024-03-30T01:42:10.289068+0000","last_active":"2024-03-30T01:42:11.279027+0000","last_peered":"2024-03-30T01:42:11.279027+0000","last_clean":"2024-03-30T01:42:11.279027+0000","last_became_active":"2024-03-30T01:42:10.288282+0000","last_became_peered":"2024-03-30T01:42:10.288282+0000","last_unstale":"2024-03-30T01:42:11.279027+0000","last_undegraded":"2024-03-30T01:42:11.279027+0000","last_fullsized":"2024-03-30T01:42:11.279027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878325+0000","last_change":"2024-03-30T01:42:10.298731+0000","last_active":"2024-03-30T01:42:11.878325+0000","last_peered":"2024-03-30T01:42:11.878325+0000","last_clean":"2024-03-30T01:42:11.878325+0000","last_became_active":"2024-03-30T01:42:10.298525+0000","last_became_peered":"2024-03-30T01:42:10.298525+0000","last_unstale":"2024-03-30T01:42:11.878325+0000","last_undegraded":"2024-03-30T01:42:11.878325+0000","last_fullsized":"2024-03-30T01:42:11.878325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878035+0000","last_change":"2024-03-30T01:42:10.285717+0000","last_active":"2024-03-30T01:42:11.878035+0000","last_peered":"2024-03-30T01:42:11.878035+0000","last_clean":"2024-03-30T01:42:11.878035+0000","last_became_active":"2024-03-30T01:42:10.285437+0000","last_became_peered":"2024-03-30T01:42:10.285437+0000","last_unstale":"2024-03-30T01:42:11.878035+0000","last_undegraded":"2024-03-30T01:42:11.878035+0000","last_fullsized":"2024-03-30T01:42:11.878035+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279381+0000","last_change":"2024-03-30T01:42:10.295963+0000","last_active":"2024-03-30T01:42:11.279381+0000","last_peered":"2024-03-30T01:42:11.279381+0000","last_clean":"2024-03-30T01:42:11.279381+0000","last_became_active":"2024-03-30T01:42:10.295545+0000","last_became_peered":"2024-03-30T01:42:10.295545+0000","last_unstale":"2024-03-30T01:42:11.279381+0000","last_undegraded":"2024-03-30T01:42:11.279381+0000","last_fullsized":"2024-03-30T01:42:11.279381+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279241+0000","last_change":"2024-03-30T01:42:10.296160+0000","last_active":"2024-03-30T01:42:11.279241+0000","last_peered":"2024-03-30T01:42:11.279241+0000","last_clean":"2024-03-30T01:42:11.279241+0000","last_became_active":"2024-03-30T01:42:10.295867+0000","last_became_peered":"2024-03-30T01:42:10.295867+0000","last_unstale":"2024-03-30T01:42:11.279241+0000","last_undegraded":"2024-03-30T01:42:11.279241+0000","last_fullsized":"2024-03-30T01:42:11.279241+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878076+0000","last_change":"2024-03-30T01:42:10.294103+0000","last_active":"2024-03-30T01:42:11.878076+0000","last_peered":"2024-03-30T01:42:11.878076+0000","last_clean":"2024-03-30T01:42:11.878076+0000","last_became_active":"2024-03-30T01:42:10.293833+0000","last_became_peered":"2024-03-30T01:42:10.293833+0000","last_unstale":"2024-03-30T01:42:11.878076+0000","last_undegraded":"2024-03-30T01:42:11.878076+0000","last_fullsized":"2024-03-30T01:42:11.878076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278814+0000","last_change":"2024-03-30T01:42:10.295808+0000","last_active":"2024-03-30T01:42:11.278814+0000","last_peered":"2024-03-30T01:42:11.278814+0000","last_clean":"2024-03-30T01:42:11.278814+0000","last_became_active":"2024-03-30T01:42:10.295460+0000","last_became_peered":"2024-03-30T01:42:10.295460+0000","last_unstale":"2024-03-30T01:42:11.278814+0000","last_undegraded":"2024-03-30T01:42:11.278814+0000","last_fullsized":"2024-03-30T01:42:11.278814+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878487+0000","last_change":"2024-03-30T01:42:10.294667+0000","last_active":"2024-03-30T01:42:11.878487+0000","last_peered":"2024-03-30T01:42:11.878487+0000","last_clean":"2024-03-30T01:42:11.878487+0000","last_became_active":"2024-03-30T01:42:10.294330+0000","last_became_peered":"2024-03-30T01:42:10.294330+0000","last_unstale":"2024-03-30T01:42:11.878487+0000","last_undegraded":"2024-03-30T01:42:11.878487+0000","last_fullsized":"2024-03-30T01:42:11.878487+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278796+0000","last_change":"2024-03-30T01:42:10.293953+0000","last_active":"2024-03-30T01:42:11.278796+0000","last_peered":"2024-03-30T01:42:11.278796+0000","last_clean":"2024-03-30T01:42:11.278796+0000","last_became_active":"2024-03-30T01:42:10.293641+0000","last_became_peered":"2024-03-30T01:42:10.293641+0000","last_unstale":"2024-03-30T01:42:11.278796+0000","last_undegraded":"2024-03-30T01:42:11.278796+0000","last_fullsized":"2024-03-30T01:42:11.278796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877469+0000","last_change":"2024-03-30T01:42:10.284419+0000","last_active":"2024-03-30T01:42:11.877469+0000","last_peered":"2024-03-30T01:42:11.877469+0000","last_clean":"2024-03-30T01:42:11.877469+0000","last_became_active":"2024-03-30T01:42:10.284098+0000","last_became_peered":"2024-03-30T01:42:10.284098+0000","last_unstale":"2024-03-30T01:42:11.877469+0000","last_undegraded":"2024-03-30T01:42:11.877469+0000","last_fullsized":"2024-03-30T01:42:11.877469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878378+0000","last_change":"2024-03-30T01:42:10.285733+0000","last_active":"2024-03-30T01:42:11.878378+0000","last_peered":"2024-03-30T01:42:11.878378+0000","last_clean":"2024-03-30T01:42:11.878378+0000","last_became_active":"2024-03-30T01:42:10.285428+0000","last_became_peered":"2024-03-30T01:42:10.285428+0000","last_unstale":"2024-03-30T01:42:11.878378+0000","last_undegraded":"2024-03-30T01:42:11.878378+0000","last_fullsized":"2024-03-30T01:42:11.878378+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277996+0000","last_change":"2024-03-30T01:42:10.300393+0000","last_active":"2024-03-30T01:42:11.277996+0000","last_peered":"2024-03-30T01:42:11.277996+0000","last_clean":"2024-03-30T01:42:11.277996+0000","last_became_active":"2024-03-30T01:42:10.299795+0000","last_became_peered":"2024-03-30T01:42:10.299795+0000","last_unstale":"2024-03-30T01:42:11.277996+0000","last_undegraded":"2024-03-30T01:42:11.277996+0000","last_fullsized":"2024-03-30T01:42:11.277996+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:21.256043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278164+0000","last_change":"2024-03-30T01:42:10.292351+0000","last_active":"2024-03-30T01:42:11.278164+0000","last_peered":"2024-03-30T01:42:11.278164+0000","last_clean":"2024-03-30T01:42:11.278164+0000","last_became_active":"2024-03-30T01:42:10.292040+0000","last_became_peered":"2024-03-30T01:42:10.292040+0000","last_unstale":"2024-03-30T01:42:11.278164+0000","last_undegraded":"2024-03-30T01:42:11.278164+0000","last_fullsized":"2024-03-30T01:42:11.278164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:09.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277912+0000","last_change":"2024-03-30T01:42:10.300160+0000","last_active":"2024-03-30T01:42:11.277912+0000","last_peered":"2024-03-30T01:42:11.277912+0000","last_clean":"2024-03-30T01:42:11.277912+0000","last_became_active":"2024-03-30T01:42:10.299728+0000","last_became_peered":"2024-03-30T01:42:10.299728+0000","last_unstale":"2024-03-30T01:42:11.277912+0000","last_undegraded":"2024-03-30T01:42:11.277912+0000","last_fullsized":"2024-03-30T01:42:11.277912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278944+0000","last_change":"2024-03-30T01:42:10.304038+0000","last_active":"2024-03-30T01:42:11.278944+0000","last_peered":"2024-03-30T01:42:11.278944+0000","last_clean":"2024-03-30T01:42:11.278944+0000","last_became_active":"2024-03-30T01:42:10.303747+0000","last_became_peered":"2024-03-30T01:42:10.303747+0000","last_unstale":"2024-03-30T01:42:11.278944+0000","last_undegraded":"2024-03-30T01:42:11.278944+0000","last_fullsized":"2024-03-30T01:42:11.278944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877668+0000","last_change":"2024-03-30T01:42:10.299194+0000","last_active":"2024-03-30T01:42:11.877668+0000","last_peered":"2024-03-30T01:42:11.877668+0000","last_clean":"2024-03-30T01:42:11.877668+0000","last_became_active":"2024-03-30T01:42:10.298900+0000","last_became_peered":"2024-03-30T01:42:10.298900+0000","last_unstale":"2024-03-30T01:42:11.877668+0000","last_undegraded":"2024-03-30T01:42:11.877668+0000","last_fullsized":"2024-03-30T01:42:11.877668+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878157+0000","last_change":"2024-03-30T01:42:10.281331+0000","last_active":"2024-03-30T01:42:11.878157+0000","last_peered":"2024-03-30T01:42:11.878157+0000","last_clean":"2024-03-30T01:42:11.878157+0000","last_became_active":"2024-03-30T01:42:10.281011+0000","last_became_peered":"2024-03-30T01:42:10.281011+0000","last_unstale":"2024-03-30T01:42:11.878157+0000","last_undegraded":"2024-03-30T01:42:11.878157+0000","last_fullsized":"2024-03-30T01:42:11.878157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277895+0000","last_change":"2024-03-30T01:42:10.286887+0000","last_active":"2024-03-30T01:42:11.277895+0000","last_peered":"2024-03-30T01:42:11.277895+0000","last_clean":"2024-03-30T01:42:11.277895+0000","last_became_active":"2024-03-30T01:42:10.278107+0000","last_became_peered":"2024-03-30T01:42:10.278107+0000","last_unstale":"2024-03-30T01:42:11.277895+0000","last_undegraded":"2024-03-30T01:42:11.277895+0000","last_fullsized":"2024-03-30T01:42:11.277895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279098+0000","last_change":"2024-03-30T01:42:10.285378+0000","last_active":"2024-03-30T01:42:11.279098+0000","last_peered":"2024-03-30T01:42:11.279098+0000","last_clean":"2024-03-30T01:42:11.279098+0000","last_became_active":"2024-03-30T01:42:10.285105+0000","last_became_peered":"2024-03-30T01:42:10.285105+0000","last_unstale":"2024-03-30T01:42:11.279098+0000","last_undegraded":"2024-03-30T01:42:11.279098+0000","last_fullsized":"2024-03-30T01:42:11.279098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:52.824468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878480+0000","last_change":"2024-03-30T01:42:10.290378+0000","last_active":"2024-03-30T01:42:11.878480+0000","last_peered":"2024-03-30T01:42:11.878480+0000","last_clean":"2024-03-30T01:42:11.878480+0000","last_became_active":"2024-03-30T01:42:10.290022+0000","last_became_peered":"2024-03-30T01:42:10.290022+0000","last_unstale":"2024-03-30T01:42:11.878480+0000","last_undegraded":"2024-03-30T01:42:11.878480+0000","last_fullsized":"2024-03-30T01:42:11.878480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278078+0000","last_change":"2024-03-30T01:42:10.274845+0000","last_active":"2024-03-30T01:42:11.278078+0000","last_peered":"2024-03-30T01:42:11.278078+0000","last_clean":"2024-03-30T01:42:11.278078+0000","last_became_active":"2024-03-30T01:42:10.274566+0000","last_became_peered":"2024-03-30T01:42:10.274566+0000","last_unstale":"2024-03-30T01:42:11.278078+0000","last_undegraded":"2024-03-30T01:42:11.278078+0000","last_fullsized":"2024-03-30T01:42:11.278078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:32.801396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279479+0000","last_change":"2024-03-30T01:42:10.288633+0000","last_active":"2024-03-30T01:42:11.279479+0000","last_peered":"2024-03-30T01:42:11.279479+0000","last_clean":"2024-03-30T01:42:11.279479+0000","last_became_active":"2024-03-30T01:42:10.288326+0000","last_became_peered":"2024-03-30T01:42:10.288326+0000","last_unstale":"2024-03-30T01:42:11.279479+0000","last_undegraded":"2024-03-30T01:42:11.279479+0000","last_fullsized":"2024-03-30T01:42:11.279479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278129+0000","last_change":"2024-03-30T01:42:10.291099+0000","last_active":"2024-03-30T01:42:11.278129+0000","last_peered":"2024-03-30T01:42:11.278129+0000","last_clean":"2024-03-30T01:42:11.278129+0000","last_became_active":"2024-03-30T01:42:10.290802+0000","last_became_peered":"2024-03-30T01:42:10.290802+0000","last_unstale":"2024-03-30T01:42:11.278129+0000","last_undegraded":"2024-03-30T01:42:11.278129+0000","last_fullsized":"2024-03-30T01:42:11.278129+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:13.070677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277537+0000","last_change":"2024-03-30T01:42:10.290078+0000","last_active":"2024-03-30T01:42:11.277537+0000","last_peered":"2024-03-30T01:42:11.277537+0000","last_clean":"2024-03-30T01:42:11.277537+0000","last_became_active":"2024-03-30T01:42:10.289127+0000","last_became_peered":"2024-03-30T01:42:10.289127+0000","last_unstale":"2024-03-30T01:42:11.277537+0000","last_undegraded":"2024-03-30T01:42:11.277537+0000","last_fullsized":"2024-03-30T01:42:11.277537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:36.303285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877529+0000","last_change":"2024-03-30T01:42:10.274200+0000","last_active":"2024-03-30T01:42:11.877529+0000","last_peered":"2024-03-30T01:42:11.877529+0000","last_clean":"2024-03-30T01:42:11.877529+0000","last_became_active":"2024-03-30T01:42:10.273923+0000","last_became_peered":"2024-03-30T01:42:10.273923+0000","last_unstale":"2024-03-30T01:42:11.877529+0000","last_undegraded":"2024-03-30T01:42:11.877529+0000","last_fullsized":"2024-03-30T01:42:11.877529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:44.766475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877662+0000","last_change":"2024-03-30T01:42:10.297189+0000","last_active":"2024-03-30T01:42:11.877662+0000","last_peered":"2024-03-30T01:42:11.877662+0000","last_clean":"2024-03-30T01:42:11.877662+0000","last_became_active":"2024-03-30T01:42:10.296821+0000","last_became_peered":"2024-03-30T01:42:10.296821+0000","last_unstale":"2024-03-30T01:42:11.877662+0000","last_undegraded":"2024-03-30T01:42:11.877662+0000","last_fullsized":"2024-03-30T01:42:11.877662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877932+0000","last_change":"2024-03-30T01:42:10.285919+0000","last_active":"2024-03-30T01:42:11.877932+0000","last_peered":"2024-03-30T01:42:11.877932+0000","last_clean":"2024-03-30T01:42:11.877932+0000","last_became_active":"2024-03-30T01:42:10.285629+0000","last_became_peered":"2024-03-30T01:42:10.285629+0000","last_unstale":"2024-03-30T01:42:11.877932+0000","last_undegraded":"2024-03-30T01:42:11.877932+0000","last_fullsized":"2024-03-30T01:42:11.877932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279214+0000","last_change":"2024-03-30T01:42:09.824850+0000","last_active":"2024-03-30T01:42:11.279214+0000","last_peered":"2024-03-30T01:42:11.279214+0000","last_clean":"2024-03-30T01:42:11.279214+0000","last_became_active":"2024-03-30T01:42:09.824537+0000","last_became_peered":"2024-03-30T01:42:09.824537+0000","last_unstale":"2024-03-30T01:42:11.279214+0000","last_undegraded":"2024-03-30T01:42:11.279214+0000","last_fullsized":"2024-03-30T01:42:11.279214+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:17.759503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878227+0000","last_change":"2024-03-30T01:42:10.298381+0000","last_active":"2024-03-30T01:42:11.878227+0000","last_peered":"2024-03-30T01:42:11.878227+0000","last_clean":"2024-03-30T01:42:11.878227+0000","last_became_active":"2024-03-30T01:42:10.298109+0000","last_became_peered":"2024-03-30T01:42:10.298109+0000","last_unstale":"2024-03-30T01:42:11.878227+0000","last_undegraded":"2024-03-30T01:42:11.878227+0000","last_fullsized":"2024-03-30T01:42:11.878227+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878784+0000","last_change":"2024-03-30T01:42:09.829899+0000","last_active":"2024-03-30T01:42:11.878784+0000","last_peered":"2024-03-30T01:42:11.878784+0000","last_clean":"2024-03-30T01:42:11.878784+0000","last_became_active":"2024-03-30T01:42:09.829601+0000","last_became_peered":"2024-03-30T01:42:09.829601+0000","last_unstale":"2024-03-30T01:42:11.878784+0000","last_undegraded":"2024-03-30T01:42:11.878784+0000","last_fullsized":"2024-03-30T01:42:11.878784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:30.887459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878336+0000","last_change":"2024-03-30T01:42:10.300292+0000","last_active":"2024-03-30T01:42:11.878336+0000","last_peered":"2024-03-30T01:42:11.878336+0000","last_clean":"2024-03-30T01:42:11.878336+0000","last_became_active":"2024-03-30T01:42:10.299867+0000","last_became_peered":"2024-03-30T01:42:10.299867+0000","last_unstale":"2024-03-30T01:42:11.878336+0000","last_undegraded":"2024-03-30T01:42:11.878336+0000","last_fullsized":"2024-03-30T01:42:11.878336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878423+0000","last_change":"2024-03-30T01:42:09.829349+0000","last_active":"2024-03-30T01:42:11.878423+0000","last_peered":"2024-03-30T01:42:11.878423+0000","last_clean":"2024-03-30T01:42:11.878423+0000","last_became_active":"2024-03-30T01:42:09.828931+0000","last_became_peered":"2024-03-30T01:42:09.828931+0000","last_unstale":"2024-03-30T01:42:11.878423+0000","last_undegraded":"2024-03-30T01:42:11.878423+0000","last_fullsized":"2024-03-30T01:42:11.878423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878765+0000","last_change":"2024-03-30T01:42:10.291979+0000","last_active":"2024-03-30T01:42:11.878765+0000","last_peered":"2024-03-30T01:42:11.878765+0000","last_clean":"2024-03-30T01:42:11.878765+0000","last_became_active":"2024-03-30T01:42:10.291708+0000","last_became_peered":"2024-03-30T01:42:10.291708+0000","last_unstale":"2024-03-30T01:42:11.878765+0000","last_undegraded":"2024-03-30T01:42:11.878765+0000","last_fullsized":"2024-03-30T01:42:11.878765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877952+0000","last_change":"2024-03-30T01:42:09.824809+0000","last_active":"2024-03-30T01:42:11.877952+0000","last_peered":"2024-03-30T01:42:11.877952+0000","last_clean":"2024-03-30T01:42:11.877952+0000","last_became_active":"2024-03-30T01:42:09.824493+0000","last_became_peered":"2024-03-30T01:42:09.824493+0000","last_unstale":"2024-03-30T01:42:11.877952+0000","last_undegraded":"2024-03-30T01:42:11.877952+0000","last_fullsized":"2024-03-30T01:42:11.877952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877821+0000","last_change":"2024-03-30T01:42:10.287526+0000","last_active":"2024-03-30T01:42:11.877821+0000","last_peered":"2024-03-30T01:42:11.877821+0000","last_clean":"2024-03-30T01:42:11.877821+0000","last_became_active":"2024-03-30T01:42:10.287255+0000","last_became_peered":"2024-03-30T01:42:10.287255+0000","last_unstale":"2024-03-30T01:42:11.877821+0000","last_undegraded":"2024-03-30T01:42:11.877821+0000","last_fullsized":"2024-03-30T01:42:11.877821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279059+0000","last_change":"2024-03-30T01:42:09.831496+0000","last_active":"2024-03-30T01:42:11.279059+0000","last_peered":"2024-03-30T01:42:11.279059+0000","last_clean":"2024-03-30T01:42:11.279059+0000","last_became_active":"2024-03-30T01:42:09.831197+0000","last_became_peered":"2024-03-30T01:42:09.831197+0000","last_unstale":"2024-03-30T01:42:11.279059+0000","last_undegraded":"2024-03-30T01:42:11.279059+0000","last_fullsized":"2024-03-30T01:42:11.279059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278809+0000","last_change":"2024-03-30T01:42:10.289213+0000","last_active":"2024-03-30T01:42:11.278809+0000","last_peered":"2024-03-30T01:42:11.278809+0000","last_clean":"2024-03-30T01:42:11.278809+0000","last_became_active":"2024-03-30T01:42:10.288677+0000","last_became_peered":"2024-03-30T01:42:10.288677+0000","last_unstale":"2024-03-30T01:42:11.278809+0000","last_undegraded":"2024-03-30T01:42:11.278809+0000","last_fullsized":"2024-03-30T01:42:11.278809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877953+0000","last_change":"2024-03-30T01:42:09.831981+0000","last_active":"2024-03-30T01:42:11.877953+0000","last_peered":"2024-03-30T01:42:11.877953+0000","last_clean":"2024-03-30T01:42:11.877953+0000","last_became_active":"2024-03-30T01:42:09.831647+0000","last_became_peered":"2024-03-30T01:42:09.831647+0000","last_unstale":"2024-03-30T01:42:11.877953+0000","last_undegraded":"2024-03-30T01:42:11.877953+0000","last_fullsized":"2024-03-30T01:42:11.877953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:02.883477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878150+0000","last_change":"2024-03-30T01:42:10.303359+0000","last_active":"2024-03-30T01:42:11.878150+0000","last_peered":"2024-03-30T01:42:11.878150+0000","last_clean":"2024-03-30T01:42:11.878150+0000","last_became_active":"2024-03-30T01:42:10.303039+0000","last_became_peered":"2024-03-30T01:42:10.303039+0000","last_unstale":"2024-03-30T01:42:11.878150+0000","last_undegraded":"2024-03-30T01:42:11.878150+0000","last_fullsized":"2024-03-30T01:42:11.878150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878771+0000","last_change":"2024-03-30T01:42:09.826326+0000","last_active":"2024-03-30T01:42:11.878771+0000","last_peered":"2024-03-30T01:42:11.878771+0000","last_clean":"2024-03-30T01:42:11.878771+0000","last_became_active":"2024-03-30T01:42:09.826050+0000","last_became_peered":"2024-03-30T01:42:09.826050+0000","last_unstale":"2024-03-30T01:42:11.878771+0000","last_undegraded":"2024-03-30T01:42:11.878771+0000","last_fullsized":"2024-03-30T01:42:11.878771+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.879341+0000","last_change":"2024-03-30T01:42:10.291872+0000","last_active":"2024-03-30T01:42:11.879341+0000","last_peered":"2024-03-30T01:42:11.879341+0000","last_clean":"2024-03-30T01:42:11.879341+0000","last_became_active":"2024-03-30T01:42:10.291199+0000","last_became_peered":"2024-03-30T01:42:10.291199+0000","last_unstale":"2024-03-30T01:42:11.879341+0000","last_undegraded":"2024-03-30T01:42:11.879341+0000","last_fullsized":"2024-03-30T01:42:11.879341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877605+0000","last_change":"2024-03-30T01:42:09.820512+0000","last_active":"2024-03-30T01:42:11.877605+0000","last_peered":"2024-03-30T01:42:11.877605+0000","last_clean":"2024-03-30T01:42:11.877605+0000","last_became_active":"2024-03-30T01:42:09.820253+0000","last_became_peered":"2024-03-30T01:42:09.820253+0000","last_unstale":"2024-03-30T01:42:11.877605+0000","last_undegraded":"2024-03-30T01:42:11.877605+0000","last_fullsized":"2024-03-30T01:42:11.877605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.455801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278160+0000","last_change":"2024-03-30T01:42:10.295259+0000","last_active":"2024-03-30T01:42:11.278160+0000","last_peered":"2024-03-30T01:42:11.278160+0000","last_clean":"2024-03-30T01:42:11.278160+0000","last_became_active":"2024-03-30T01:42:10.295054+0000","last_became_peered":"2024-03-30T01:42:10.295054+0000","last_unstale":"2024-03-30T01:42:11.278160+0000","last_undegraded":"2024-03-30T01:42:11.278160+0000","last_fullsized":"2024-03-30T01:42:11.278160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:36.942583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878902+0000","last_change":"2024-03-30T01:42:09.828624+0000","last_active":"2024-03-30T01:42:11.878902+0000","last_peered":"2024-03-30T01:42:11.878902+0000","last_clean":"2024-03-30T01:42:11.878902+0000","last_became_active":"2024-03-30T01:42:09.828341+0000","last_became_peered":"2024-03-30T01:42:09.828341+0000","last_unstale":"2024-03-30T01:42:11.878902+0000","last_undegraded":"2024-03-30T01:42:11.878902+0000","last_fullsized":"2024-03-30T01:42:11.878902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:32.138427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878174+0000","last_change":"2024-03-30T01:42:10.284755+0000","last_active":"2024-03-30T01:42:11.878174+0000","last_peered":"2024-03-30T01:42:11.878174+0000","last_clean":"2024-03-30T01:42:11.878174+0000","last_became_active":"2024-03-30T01:42:10.284421+0000","last_became_peered":"2024-03-30T01:42:10.284421+0000","last_unstale":"2024-03-30T01:42:11.878174+0000","last_undegraded":"2024-03-30T01:42:11.878174+0000","last_fullsized":"2024-03-30T01:42:11.878174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279440+0000","last_change":"2024-03-30T01:42:09.826561+0000","last_active":"2024-03-30T01:42:11.279440+0000","last_peered":"2024-03-30T01:42:11.279440+0000","last_clean":"2024-03-30T01:42:11.279440+0000","last_became_active":"2024-03-30T01:42:09.826059+0000","last_became_peered":"2024-03-30T01:42:09.826059+0000","last_unstale":"2024-03-30T01:42:11.279440+0000","last_undegraded":"2024-03-30T01:42:11.279440+0000","last_fullsized":"2024-03-30T01:42:11.279440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877892+0000","last_change":"2024-03-30T01:42:10.292182+0000","last_active":"2024-03-30T01:42:11.877892+0000","last_peered":"2024-03-30T01:42:11.877892+0000","last_clean":"2024-03-30T01:42:11.877892+0000","last_became_active":"2024-03-30T01:42:10.291868+0000","last_became_peered":"2024-03-30T01:42:10.291868+0000","last_unstale":"2024-03-30T01:42:11.877892+0000","last_undegraded":"2024-03-30T01:42:11.877892+0000","last_fullsized":"2024-03-30T01:42:11.877892+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279386+0000","last_change":"2024-03-30T01:42:09.833439+0000","last_active":"2024-03-30T01:42:11.279386+0000","last_peered":"2024-03-30T01:42:11.279386+0000","last_clean":"2024-03-30T01:42:11.279386+0000","last_became_active":"2024-03-30T01:42:09.833137+0000","last_became_peered":"2024-03-30T01:42:09.833137+0000","last_unstale":"2024-03-30T01:42:11.279386+0000","last_undegraded":"2024-03-30T01:42:11.279386+0000","last_fullsized":"2024-03-30T01:42:11.279386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877995+0000","last_change":"2024-03-30T01:42:10.296944+0000","last_active":"2024-03-30T01:42:11.877995+0000","last_peered":"2024-03-30T01:42:11.877995+0000","last_clean":"2024-03-30T01:42:11.877995+0000","last_became_active":"2024-03-30T01:42:10.296714+0000","last_became_peered":"2024-03-30T01:42:10.296714+0000","last_unstale":"2024-03-30T01:42:11.877995+0000","last_undegraded":"2024-03-30T01:42:11.877995+0000","last_fullsized":"2024-03-30T01:42:11.877995+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878974+0000","last_change":"2024-03-30T01:42:09.823152+0000","last_active":"2024-03-30T01:42:11.878974+0000","last_peered":"2024-03-30T01:42:11.878974+0000","last_clean":"2024-03-30T01:42:11.878974+0000","last_became_active":"2024-03-30T01:42:09.822826+0000","last_became_peered":"2024-03-30T01:42:09.822826+0000","last_unstale":"2024-03-30T01:42:11.878974+0000","last_undegraded":"2024-03-30T01:42:11.878974+0000","last_fullsized":"2024-03-30T01:42:11.878974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:17.799215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877695+0000","last_change":"2024-03-30T01:42:10.285053+0000","last_active":"2024-03-30T01:42:11.877695+0000","last_peered":"2024-03-30T01:42:11.877695+0000","last_clean":"2024-03-30T01:42:11.877695+0000","last_became_active":"2024-03-30T01:42:10.284814+0000","last_became_peered":"2024-03-30T01:42:10.284814+0000","last_unstale":"2024-03-30T01:42:11.877695+0000","last_undegraded":"2024-03-30T01:42:11.877695+0000","last_fullsized":"2024-03-30T01:42:11.877695+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:28.875197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878965+0000","last_change":"2024-03-30T01:42:09.832318+0000","last_active":"2024-03-30T01:42:11.878965+0000","last_peered":"2024-03-30T01:42:11.878965+0000","last_clean":"2024-03-30T01:42:11.878965+0000","last_became_active":"2024-03-30T01:42:09.831964+0000","last_became_peered":"2024-03-30T01:42:09.831964+0000","last_unstale":"2024-03-30T01:42:11.878965+0000","last_undegraded":"2024-03-30T01:42:11.878965+0000","last_fullsized":"2024-03-30T01:42:11.878965+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:05.700970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878486+0000","last_change":"2024-03-30T01:42:10.288326+0000","last_active":"2024-03-30T01:42:11.878486+0000","last_peered":"2024-03-30T01:42:11.878486+0000","last_clean":"2024-03-30T01:42:11.878486+0000","last_became_active":"2024-03-30T01:42:10.288050+0000","last_became_peered":"2024-03-30T01:42:10.288050+0000","last_unstale":"2024-03-30T01:42:11.878486+0000","last_undegraded":"2024-03-30T01:42:11.878486+0000","last_fullsized":"2024-03-30T01:42:11.878486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878573+0000","last_change":"2024-03-30T01:42:09.829194+0000","last_active":"2024-03-30T01:42:11.878573+0000","last_peered":"2024-03-30T01:42:11.878573+0000","last_clean":"2024-03-30T01:42:11.878573+0000","last_became_active":"2024-03-30T01:42:09.828257+0000","last_became_peered":"2024-03-30T01:42:09.828257+0000","last_unstale":"2024-03-30T01:42:11.878573+0000","last_undegraded":"2024-03-30T01:42:11.878573+0000","last_fullsized":"2024-03-30T01:42:11.878573+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877821+0000","last_change":"2024-03-30T01:42:10.279681+0000","last_active":"2024-03-30T01:42:11.877821+0000","last_peered":"2024-03-30T01:42:11.877821+0000","last_clean":"2024-03-30T01:42:11.877821+0000","last_became_active":"2024-03-30T01:42:10.279333+0000","last_became_peered":"2024-03-30T01:42:10.279333+0000","last_unstale":"2024-03-30T01:42:11.877821+0000","last_undegraded":"2024-03-30T01:42:11.877821+0000","last_fullsized":"2024-03-30T01:42:11.877821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277537+0000","last_change":"2024-03-30T01:42:09.820451+0000","last_active":"2024-03-30T01:42:11.277537+0000","last_peered":"2024-03-30T01:42:11.277537+0000","last_clean":"2024-03-30T01:42:11.277537+0000","last_became_active":"2024-03-30T01:42:09.820152+0000","last_became_peered":"2024-03-30T01:42:09.820152+0000","last_unstale":"2024-03-30T01:42:11.277537+0000","last_undegraded":"2024-03-30T01:42:11.277537+0000","last_fullsized":"2024-03-30T01:42:11.277537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:04.153303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878324+0000","last_change":"2024-03-30T01:42:10.288399+0000","last_active":"2024-03-30T01:42:11.878324+0000","last_peered":"2024-03-30T01:42:11.878324+0000","last_clean":"2024-03-30T01:42:11.878324+0000","last_became_active":"2024-03-30T01:42:10.287890+0000","last_became_peered":"2024-03-30T01:42:10.287890+0000","last_unstale":"2024-03-30T01:42:11.878324+0000","last_undegraded":"2024-03-30T01:42:11.878324+0000","last_fullsized":"2024-03-30T01:42:11.878324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877618+0000","last_change":"2024-03-30T01:42:09.831869+0000","last_active":"2024-03-30T01:42:11.877618+0000","last_peered":"2024-03-30T01:42:11.877618+0000","last_clean":"2024-03-30T01:42:11.877618+0000","last_became_active":"2024-03-30T01:42:09.831562+0000","last_became_peered":"2024-03-30T01:42:09.831562+0000","last_unstale":"2024-03-30T01:42:11.877618+0000","last_undegraded":"2024-03-30T01:42:11.877618+0000","last_fullsized":"2024-03-30T01:42:11.877618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:35.938001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878831+0000","last_change":"2024-03-30T01:42:10.291564+0000","last_active":"2024-03-30T01:42:11.878831+0000","last_peered":"2024-03-30T01:42:11.878831+0000","last_clean":"2024-03-30T01:42:11.878831+0000","last_became_active":"2024-03-30T01:42:10.291285+0000","last_became_peered":"2024-03-30T01:42:10.291285+0000","last_unstale":"2024-03-30T01:42:11.878831+0000","last_undegraded":"2024-03-30T01:42:11.878831+0000","last_fullsized":"2024-03-30T01:42:11.878831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279538+0000","last_change":"2024-03-30T01:42:09.826895+0000","last_active":"2024-03-30T01:42:11.279538+0000","last_peered":"2024-03-30T01:42:11.279538+0000","last_clean":"2024-03-30T01:42:11.279538+0000","last_became_active":"2024-03-30T01:42:09.826609+0000","last_became_peered":"2024-03-30T01:42:09.826609+0000","last_unstale":"2024-03-30T01:42:11.279538+0000","last_undegraded":"2024-03-30T01:42:11.279538+0000","last_fullsized":"2024-03-30T01:42:11.279538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:42:59.036073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279123+0000","last_change":"2024-03-30T01:42:10.295516+0000","last_active":"2024-03-30T01:42:11.279123+0000","last_peered":"2024-03-30T01:42:11.279123+0000","last_clean":"2024-03-30T01:42:11.279123+0000","last_became_active":"2024-03-30T01:42:10.295133+0000","last_became_peered":"2024-03-30T01:42:10.295133+0000","last_unstale":"2024-03-30T01:42:11.279123+0000","last_undegraded":"2024-03-30T01:42:11.279123+0000","last_fullsized":"2024-03-30T01:42:11.279123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878440+0000","last_change":"2024-03-30T01:42:09.829713+0000","last_active":"2024-03-30T01:42:11.878440+0000","last_peered":"2024-03-30T01:42:11.878440+0000","last_clean":"2024-03-30T01:42:11.878440+0000","last_became_active":"2024-03-30T01:42:09.829375+0000","last_became_peered":"2024-03-30T01:42:09.829375+0000","last_unstale":"2024-03-30T01:42:11.878440+0000","last_undegraded":"2024-03-30T01:42:11.878440+0000","last_fullsized":"2024-03-30T01:42:11.878440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:49:51.601663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878538+0000","last_change":"2024-03-30T01:42:10.304085+0000","last_active":"2024-03-30T01:42:11.878538+0000","last_peered":"2024-03-30T01:42:11.878538+0000","last_clean":"2024-03-30T01:42:11.878538+0000","last_became_active":"2024-03-30T01:42:10.303817+0000","last_became_peered":"2024-03-30T01:42:10.303817+0000","last_unstale":"2024-03-30T01:42:11.878538+0000","last_undegraded":"2024-03-30T01:42:11.878538+0000","last_fullsized":"2024-03-30T01:42:11.878538+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278117+0000","last_change":"2024-03-30T01:42:09.823221+0000","last_active":"2024-03-30T01:42:11.278117+0000","last_peered":"2024-03-30T01:42:11.278117+0000","last_clean":"2024-03-30T01:42:11.278117+0000","last_became_active":"2024-03-30T01:42:09.822882+0000","last_became_peered":"2024-03-30T01:42:09.822882+0000","last_unstale":"2024-03-30T01:42:11.278117+0000","last_undegraded":"2024-03-30T01:42:11.278117+0000","last_fullsized":"2024-03-30T01:42:11.278117+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:00:31.185685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278271+0000","last_change":"2024-03-30T01:42:10.286900+0000","last_active":"2024-03-30T01:42:11.278271+0000","last_peered":"2024-03-30T01:42:11.278271+0000","last_clean":"2024-03-30T01:42:11.278271+0000","last_became_active":"2024-03-30T01:42:10.286598+0000","last_became_peered":"2024-03-30T01:42:10.286598+0000","last_unstale":"2024-03-30T01:42:11.278271+0000","last_undegraded":"2024-03-30T01:42:11.278271+0000","last_fullsized":"2024-03-30T01:42:11.278271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:46.592007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878240+0000","last_change":"2024-03-30T01:42:09.828572+0000","last_active":"2024-03-30T01:42:11.878240+0000","last_peered":"2024-03-30T01:42:11.878240+0000","last_clean":"2024-03-30T01:42:11.878240+0000","last_became_active":"2024-03-30T01:42:09.828225+0000","last_became_peered":"2024-03-30T01:42:09.828225+0000","last_unstale":"2024-03-30T01:42:11.878240+0000","last_undegraded":"2024-03-30T01:42:11.878240+0000","last_fullsized":"2024-03-30T01:42:11.878240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877568+0000","last_change":"2024-03-30T01:42:10.279913+0000","last_active":"2024-03-30T01:42:11.877568+0000","last_peered":"2024-03-30T01:42:11.877568+0000","last_clean":"2024-03-30T01:42:11.877568+0000","last_became_active":"2024-03-30T01:42:10.279450+0000","last_became_peered":"2024-03-30T01:42:10.279450+0000","last_unstale":"2024-03-30T01:42:11.877568+0000","last_undegraded":"2024-03-30T01:42:11.877568+0000","last_fullsized":"2024-03-30T01:42:11.877568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278159+0000","last_change":"2024-03-30T01:42:09.830452+0000","last_active":"2024-03-30T01:42:11.278159+0000","last_peered":"2024-03-30T01:42:11.278159+0000","last_clean":"2024-03-30T01:42:11.278159+0000","last_became_active":"2024-03-30T01:42:09.830173+0000","last_became_peered":"2024-03-30T01:42:09.830173+0000","last_unstale":"2024-03-30T01:42:11.278159+0000","last_undegraded":"2024-03-30T01:42:11.278159+0000","last_fullsized":"2024-03-30T01:42:11.278159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:12:42.807604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878090+0000","last_change":"2024-03-30T01:42:10.290469+0000","last_active":"2024-03-30T01:42:11.878090+0000","last_peered":"2024-03-30T01:42:11.878090+0000","last_clean":"2024-03-30T01:42:11.878090+0000","last_became_active":"2024-03-30T01:42:10.290148+0000","last_became_peered":"2024-03-30T01:42:10.290148+0000","last_unstale":"2024-03-30T01:42:11.878090+0000","last_undegraded":"2024-03-30T01:42:11.878090+0000","last_fullsized":"2024-03-30T01:42:11.878090+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.278053+0000","last_change":"2024-03-30T01:42:09.826256+0000","last_active":"2024-03-30T01:42:11.278053+0000","last_peered":"2024-03-30T01:42:11.278053+0000","last_clean":"2024-03-30T01:42:11.278053+0000","last_became_active":"2024-03-30T01:42:09.825956+0000","last_became_peered":"2024-03-30T01:42:09.825956+0000","last_unstale":"2024-03-30T01:42:11.278053+0000","last_undegraded":"2024-03-30T01:42:11.278053+0000","last_fullsized":"2024-03-30T01:42:11.278053+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:59:41.446927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279087+0000","last_change":"2024-03-30T01:42:10.285649+0000","last_active":"2024-03-30T01:42:11.279087+0000","last_peered":"2024-03-30T01:42:11.279087+0000","last_clean":"2024-03-30T01:42:11.279087+0000","last_became_active":"2024-03-30T01:42:10.285297+0000","last_became_peered":"2024-03-30T01:42:10.285297+0000","last_unstale":"2024-03-30T01:42:11.279087+0000","last_undegraded":"2024-03-30T01:42:11.279087+0000","last_fullsized":"2024-03-30T01:42:11.279087+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878281+0000","last_change":"2024-03-30T01:42:09.829777+0000","last_active":"2024-03-30T01:42:11.878281+0000","last_peered":"2024-03-30T01:42:11.878281+0000","last_clean":"2024-03-30T01:42:11.878281+0000","last_became_active":"2024-03-30T01:42:09.829483+0000","last_became_peered":"2024-03-30T01:42:09.829483+0000","last_unstale":"2024-03-30T01:42:11.878281+0000","last_undegraded":"2024-03-30T01:42:11.878281+0000","last_fullsized":"2024-03-30T01:42:11.878281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:45.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878808+0000","last_change":"2024-03-30T01:42:10.292043+0000","last_active":"2024-03-30T01:42:11.878808+0000","last_peered":"2024-03-30T01:42:11.878808+0000","last_clean":"2024-03-30T01:42:11.878808+0000","last_became_active":"2024-03-30T01:42:10.291598+0000","last_became_peered":"2024-03-30T01:42:10.291598+0000","last_unstale":"2024-03-30T01:42:11.878808+0000","last_undegraded":"2024-03-30T01:42:11.878808+0000","last_fullsized":"2024-03-30T01:42:11.878808+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277994+0000","last_change":"2024-03-30T01:42:09.829638+0000","last_active":"2024-03-30T01:42:11.277994+0000","last_peered":"2024-03-30T01:42:11.277994+0000","last_clean":"2024-03-30T01:42:11.277994+0000","last_became_active":"2024-03-30T01:42:09.829317+0000","last_became_peered":"2024-03-30T01:42:09.829317+0000","last_unstale":"2024-03-30T01:42:11.277994+0000","last_undegraded":"2024-03-30T01:42:11.277994+0000","last_fullsized":"2024-03-30T01:42:11.277994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:33.639097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878108+0000","last_change":"2024-03-30T01:42:10.298216+0000","last_active":"2024-03-30T01:42:11.878108+0000","last_peered":"2024-03-30T01:42:11.878108+0000","last_clean":"2024-03-30T01:42:11.878108+0000","last_became_active":"2024-03-30T01:42:10.297976+0000","last_became_peered":"2024-03-30T01:42:10.297976+0000","last_unstale":"2024-03-30T01:42:11.878108+0000","last_undegraded":"2024-03-30T01:42:11.878108+0000","last_fullsized":"2024-03-30T01:42:11.878108+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279604+0000","last_change":"2024-03-30T01:42:09.833189+0000","last_active":"2024-03-30T01:42:11.279604+0000","last_peered":"2024-03-30T01:42:11.279604+0000","last_clean":"2024-03-30T01:42:11.279604+0000","last_became_active":"2024-03-30T01:42:09.832891+0000","last_became_peered":"2024-03-30T01:42:09.832891+0000","last_unstale":"2024-03-30T01:42:11.279604+0000","last_undegraded":"2024-03-30T01:42:11.279604+0000","last_fullsized":"2024-03-30T01:42:11.279604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878290+0000","last_change":"2024-03-30T01:42:10.284232+0000","last_active":"2024-03-30T01:42:11.878290+0000","last_peered":"2024-03-30T01:42:11.878290+0000","last_clean":"2024-03-30T01:42:11.878290+0000","last_became_active":"2024-03-30T01:42:10.283909+0000","last_became_peered":"2024-03-30T01:42:10.283909+0000","last_unstale":"2024-03-30T01:42:11.878290+0000","last_undegraded":"2024-03-30T01:42:11.878290+0000","last_fullsized":"2024-03-30T01:42:11.878290+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:33.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.959456+0000","last_change":"2024-03-30T01:42:09.824373+0000","last_active":"2024-03-30T01:42:11.959456+0000","last_peered":"2024-03-30T01:42:11.959456+0000","last_clean":"2024-03-30T01:42:11.959456+0000","last_became_active":"2024-03-30T01:42:09.824089+0000","last_became_peered":"2024-03-30T01:42:09.824089+0000","last_unstale":"2024-03-30T01:42:11.959456+0000","last_undegraded":"2024-03-30T01:42:11.959456+0000","last_fullsized":"2024-03-30T01:42:11.959456+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:37.434399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878956+0000","last_change":"2024-03-30T01:42:10.299986+0000","last_active":"2024-03-30T01:42:11.878956+0000","last_peered":"2024-03-30T01:42:11.878956+0000","last_clean":"2024-03-30T01:42:11.878956+0000","last_became_active":"2024-03-30T01:42:10.299776+0000","last_became_peered":"2024-03-30T01:42:10.299776+0000","last_unstale":"2024-03-30T01:42:11.878956+0000","last_undegraded":"2024-03-30T01:42:11.878956+0000","last_fullsized":"2024-03-30T01:42:11.878956+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878497+0000","last_change":"2024-03-30T01:42:09.830490+0000","last_active":"2024-03-30T01:42:11.878497+0000","last_peered":"2024-03-30T01:42:11.878497+0000","last_clean":"2024-03-30T01:42:11.878497+0000","last_became_active":"2024-03-30T01:42:09.830197+0000","last_became_peered":"2024-03-30T01:42:09.830197+0000","last_unstale":"2024-03-30T01:42:11.878497+0000","last_undegraded":"2024-03-30T01:42:11.878497+0000","last_fullsized":"2024-03-30T01:42:11.878497+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:46.433516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877767+0000","last_change":"2024-03-30T01:42:10.288334+0000","last_active":"2024-03-30T01:42:11.877767+0000","last_peered":"2024-03-30T01:42:11.877767+0000","last_clean":"2024-03-30T01:42:11.877767+0000","last_became_active":"2024-03-30T01:42:10.288093+0000","last_became_peered":"2024-03-30T01:42:10.288093+0000","last_unstale":"2024-03-30T01:42:11.877767+0000","last_undegraded":"2024-03-30T01:42:11.877767+0000","last_fullsized":"2024-03-30T01:42:11.877767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878858+0000","last_change":"2024-03-30T01:42:09.827116+0000","last_active":"2024-03-30T01:42:11.878858+0000","last_peered":"2024-03-30T01:42:11.878858+0000","last_clean":"2024-03-30T01:42:11.878858+0000","last_became_active":"2024-03-30T01:42:09.826876+0000","last_became_peered":"2024-03-30T01:42:09.826876+0000","last_unstale":"2024-03-30T01:42:11.878858+0000","last_undegraded":"2024-03-30T01:42:11.878858+0000","last_fullsized":"2024-03-30T01:42:11.878858+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:38.792762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878701+0000","last_change":"2024-03-30T01:42:10.291029+0000","last_active":"2024-03-30T01:42:11.878701+0000","last_peered":"2024-03-30T01:42:11.878701+0000","last_clean":"2024-03-30T01:42:11.878701+0000","last_became_active":"2024-03-30T01:42:10.290728+0000","last_became_peered":"2024-03-30T01:42:10.290728+0000","last_unstale":"2024-03-30T01:42:11.878701+0000","last_undegraded":"2024-03-30T01:42:11.878701+0000","last_fullsized":"2024-03-30T01:42:11.878701+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.277776+0000","last_change":"2024-03-30T01:42:09.822074+0000","last_active":"2024-03-30T01:42:11.277776+0000","last_peered":"2024-03-30T01:42:11.277776+0000","last_clean":"2024-03-30T01:42:11.277776+0000","last_became_active":"2024-03-30T01:42:09.821786+0000","last_became_peered":"2024-03-30T01:42:09.821786+0000","last_unstale":"2024-03-30T01:42:11.277776+0000","last_undegraded":"2024-03-30T01:42:11.277776+0000","last_fullsized":"2024-03-30T01:42:11.277776+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:18.840885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878277+0000","last_change":"2024-03-30T01:42:10.299170+0000","last_active":"2024-03-30T01:42:11.878277+0000","last_peered":"2024-03-30T01:42:11.878277+0000","last_clean":"2024-03-30T01:42:11.878277+0000","last_became_active":"2024-03-30T01:42:10.298846+0000","last_became_peered":"2024-03-30T01:42:10.298846+0000","last_unstale":"2024-03-30T01:42:11.878277+0000","last_undegraded":"2024-03-30T01:42:11.878277+0000","last_fullsized":"2024-03-30T01:42:11.878277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878295+0000","last_change":"2024-03-30T01:42:09.832991+0000","last_active":"2024-03-30T01:42:11.878295+0000","last_peered":"2024-03-30T01:42:11.878295+0000","last_clean":"2024-03-30T01:42:11.878295+0000","last_became_active":"2024-03-30T01:42:09.832682+0000","last_became_peered":"2024-03-30T01:42:09.832682+0000","last_unstale":"2024-03-30T01:42:11.878295+0000","last_undegraded":"2024-03-30T01:42:11.878295+0000","last_fullsized":"2024-03-30T01:42:11.878295+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:37.427434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877952+0000","last_change":"2024-03-30T01:42:10.284566+0000","last_active":"2024-03-30T01:42:11.877952+0000","last_peered":"2024-03-30T01:42:11.877952+0000","last_clean":"2024-03-30T01:42:11.877952+0000","last_became_active":"2024-03-30T01:42:10.284192+0000","last_became_peered":"2024-03-30T01:42:10.284192+0000","last_unstale":"2024-03-30T01:42:11.877952+0000","last_undegraded":"2024-03-30T01:42:11.877952+0000","last_fullsized":"2024-03-30T01:42:11.877952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:05.460383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878466+0000","last_change":"2024-03-30T01:42:09.829174+0000","last_active":"2024-03-30T01:42:11.878466+0000","last_peered":"2024-03-30T01:42:11.878466+0000","last_clean":"2024-03-30T01:42:11.878466+0000","last_became_active":"2024-03-30T01:42:09.828730+0000","last_became_peered":"2024-03-30T01:42:09.828730+0000","last_unstale":"2024-03-30T01:42:11.878466+0000","last_undegraded":"2024-03-30T01:42:11.878466+0000","last_fullsized":"2024-03-30T01:42:11.878466+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.279516+0000","last_change":"2024-03-30T01:42:10.289721+0000","last_active":"2024-03-30T01:42:11.279516+0000","last_peered":"2024-03-30T01:42:11.279516+0000","last_clean":"2024-03-30T01:42:11.279516+0000","last_became_active":"2024-03-30T01:42:10.289413+0000","last_became_peered":"2024-03-30T01:42:10.289413+0000","last_unstale":"2024-03-30T01:42:11.279516+0000","last_undegraded":"2024-03-30T01:42:11.279516+0000","last_fullsized":"2024-03-30T01:42:11.279516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.878595+0000","last_change":"2024-03-30T01:42:09.836532+0000","last_active":"2024-03-30T01:42:11.878595+0000","last_peered":"2024-03-30T01:42:11.878595+0000","last_clean":"2024-03-30T01:42:11.878595+0000","last_became_active":"2024-03-30T01:42:09.836262+0000","last_became_peered":"2024-03-30T01:42:09.836262+0000","last_unstale":"2024-03-30T01:42:11.878595+0000","last_undegraded":"2024-03-30T01:42:11.878595+0000","last_fullsized":"2024-03-30T01:42:11.878595+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:08.249431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-30T01:42:10.271303+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:42:10.271303+0000","last_peered":"2024-03-30T01:42:10.271303+0000","last_clean":"2024-03-30T01:42:10.271303+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:42:10.271303+0000","last_undegraded":"2024-03-30T01:42:10.271303+0000","last_fullsized":"2024-03-30T01:42:10.271303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":786117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.624}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.72699999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.2009999999999996,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.441}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65600000000000003}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.90600000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.82499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.70799999999999996}]},{"osd":1,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.3250000000000002,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.68300000000000005}]},{"osd":2,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.66300000000000003}]},{"osd":3,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.503},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61199999999999999}]},{"osd":5,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.72299999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":15.038,"5min":15.038,"15min":15.038},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":11.858000000000001,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.33500000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.434},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1289999999999996,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.377}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":11.99,"5min":11.99,"15min":11.99},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":15.006,"5min":15.006,"15min":15.006},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":11.926,"5min":11.926,"15min":11.926},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":14.974,"5min":14.974,"15min":14.974},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.960000000000001,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":14.923999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.48899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":783548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.946999999999999,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.33400000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":12.983000000000001,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.2279999999999998,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.59599999999999997}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.3869999999999996,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.75800000000000001}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.627},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":5.468,"5min":5.468,"15min":5.468},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":12.478999999999999,"5min":12.478999999999999,"15min":12.478999999999999},"last":2.1890000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":2.1469999999999998}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":12.425000000000001,"5min":12.425000000000001,"15min":12.425000000000001},"last":2.1309999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":13.217000000000001,"5min":13.217000000000001,"15min":13.217000000000001},"last":1.8759999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":8.7400000000000002,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":1.956},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":8.5640000000000001,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":2.0630000000000002}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":8.7759999999999998,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":1.9159999999999999}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":8.5190000000000001,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":2.1139999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":3.1829999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.503},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":8.5939999999999994,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":3.105}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":195713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.375}]},{"osd":1,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.40200000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.442}]},{"osd":4,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.48099999999999998}]},{"osd":5,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.5},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.63400000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.504},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":9.3970000000000002,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.627},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.42899999999999999}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":5.5289999999999999,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":9.452,"5min":9.452,"15min":9.452},"last":0.57099999999999995}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47799999999999998}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.45400000000000001}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":6.6760000000000002,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.88200000000000001}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7480000000000002,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.7709999999999999,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.94099999999999995}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.8410000000000002,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.77000000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.72499999999999998}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.85299999999999998}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64900000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:42:14.942 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T01:42:14.942 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T01:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.119+0000 7fe4bb782700 1 Processor -- start 2024-03-30T01:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.119+0000 7fe4bb782700 1 -- start start 2024-03-30T01:42:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.120+0000 7fe4bb782700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4b414fb40 0x7fe4b40c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.120+0000 7fe4bb782700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b40a7600 0x7fe4b4152f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.120+0000 7fe4bb782700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b40c4620 con 0x7fe4b414ff20 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.120+0000 7fe4bb782700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b4152570 con 0x7fe4b414fb40 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.120+0000 7fe4bb782700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4152ad0 con 0x7fe4b40a7600 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.120+0000 7fe4b9d1f700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe4b414ff20 0x7fe4b40c47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41686/0 (socket says 172.21.15.31:41686) 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.120+0000 7fe4b9d1f700 1 -- 172.21.15.31:0/2078826617 learned_addr learned my addr 172.21.15.31:0/2078826617 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.120+0000 7fe4b951e700 1 --2- 172.21.15.31:0/2078826617 >> v2:172.21.15.59:3300/0 conn(0x7fe4b414fb40 0x7fe4b40c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.120+0000 7fe4b8d1d700 1 --2- 172.21.15.31:0/2078826617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b40a7600 0x7fe4b4152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4a7fff700 1 -- 172.21.15.31:0/2078826617 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561815179 0 0) 0x7fe4ac001490 con 0x7fe4b414ff20 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4a7fff700 1 -- 172.21.15.31:0/2078826617 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a00035b0 con 0x7fe4b414ff20 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4b8d1d700 1 -- 172.21.15.31:0/2078826617 >> v2:172.21.15.59:3300/0 conn(0x7fe4b414fb40 msgr2=0x7fe4b40c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4b8d1d700 1 --2- 172.21.15.31:0/2078826617 >> v2:172.21.15.59:3300/0 conn(0x7fe4b414fb40 0x7fe4b40c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4b8d1d700 1 -- 172.21.15.31:0/2078826617 >> v1:172.21.15.31:6789/0 conn(0x7fe4b414ff20 legacy=0x7fe4b40c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4b8d1d700 1 -- 172.21.15.31:0/2078826617 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b40c2ca0 con 0x7fe4b40a7600 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4b951e700 1 --2- 172.21.15.31:0/2078826617 >> v2:172.21.15.59:3300/0 conn(0x7fe4b414fb40 0x7fe4b40c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4b8d1d700 1 --2- 172.21.15.31:0/2078826617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b40a7600 0x7fe4b4152f60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009840 tx=0x7fe4a8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc4ba37662c0092 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4a7fff700 1 -- 172.21.15.31:0/2078826617 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8011830 con 0x7fe4b40a7600 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4a7fff700 1 -- 172.21.15.31:0/2078826617 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a8011990 con 0x7fe4b40a7600 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.121+0000 7fe4a7fff700 1 -- 172.21.15.31:0/2078826617 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8011d20 con 0x7fe4b40a7600 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.122+0000 7fe4bb782700 1 -- 172.21.15.31:0/2078826617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b40a7600 msgr2=0x7fe4b4152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.122+0000 7fe4bb782700 1 --2- 172.21.15.31:0/2078826617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b40a7600 0x7fe4b4152f60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009840 tx=0x7fe4a8009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.122+0000 7fe4bb782700 1 -- 172.21.15.31:0/2078826617 shutdown_connections 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.122+0000 7fe4bb782700 1 --2- 172.21.15.31:0/2078826617 >> v2:172.21.15.59:3300/0 conn(0x7fe4b414fb40 0x7fe4b40c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.122+0000 7fe4bb782700 1 --2- 172.21.15.31:0/2078826617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b40a7600 0x7fe4b4152f60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.122+0000 7fe4bb782700 1 -- 172.21.15.31:0/2078826617 >> 172.21.15.31:0/2078826617 conn(0x7fe4b40c7d60 msgr2=0x7fe4b40c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.122+0000 7fe4bb782700 1 -- 172.21.15.31:0/2078826617 shutdown_connections 2024-03-30T01:42:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.122+0000 7fe4bb782700 1 -- 172.21.15.31:0/2078826617 wait complete. 2024-03-30T01:42:15.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.123+0000 7fe4bb782700 1 Processor -- start 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.123+0000 7fe4bb782700 1 -- start start 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4bb782700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40a7600 0x7fe4b4150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4bb782700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b414fb40 0x7fe4b406ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4bb782700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b414ff20 0x7fe4b4149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4bb782700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b406c250 con 0x7fe4b414fb40 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4bb782700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40c1750 con 0x7fe4b40a7600 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4bb782700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b4151060 con 0x7fe4b414ff20 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4b8d1d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b414fb40 0x7fe4b406ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4b8d1d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b414fb40 0x7fe4b406ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47356/0 (socket says 172.21.15.31:47356) 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4b8d1d700 1 -- 172.21.15.31:0/3682073138 learned_addr learned my addr 172.21.15.31:0/3682073138 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:15.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4b951e700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40a7600 0x7fe4b4150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.124+0000 7fe4b9d1f700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b414ff20 0x7fe4b4149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4b8d1d700 1 -- 172.21.15.31:0/3682073138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b414ff20 msgr2=0x7fe4b4149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4b8d1d700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b414ff20 0x7fe4b4149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4b8d1d700 1 -- 172.21.15.31:0/3682073138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40a7600 msgr2=0x7fe4b4150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4b8d1d700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40a7600 0x7fe4b4150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4b8d1d700 1 -- 172.21.15.31:0/3682073138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b00094a0 con 0x7fe4b414fb40 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4b8d1d700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b414fb40 0x7fe4b406ce70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80090b0 tx=0x7fe4a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4a5ffb700 1 -- 172.21.15.31:0/3682073138 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80036a0 con 0x7fe4b414fb40 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4a5ffb700 1 -- 172.21.15.31:0/3682073138 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a801f070 con 0x7fe4b414fb40 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a80094a0 con 0x7fe4b414fb40 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4a5ffb700 1 -- 172.21.15.31:0/3682073138 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8003a40 con 0x7fe4b414fb40 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.125+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b406bb90 con 0x7fe4b414fb40 2024-03-30T01:42:15.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.126+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b40747d0 con 0x7fe4b414fb40 2024-03-30T01:42:15.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.127+0000 7fe4a5ffb700 1 -- 172.21.15.31:0/3682073138 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe4a8003ba0 con 0x7fe4b414fb40 2024-03-30T01:42:15.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.127+0000 7fe4a5ffb700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe49806ac90 0x7fe49806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.127+0000 7fe4a5ffb700 1 -- 172.21.15.31:0/3682073138 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe4a8021070 con 0x7fe4b414fb40 2024-03-30T01:42:15.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.127+0000 7fe4b951e700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe49806ac90 0x7fe49806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.128+0000 7fe4b951e700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe49806ac90 0x7fe49806d150 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009d70 tx=0x7fe4b0009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:15.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.132+0000 7fe4a5ffb700 1 -- 172.21.15.31:0/3682073138 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe4a8055d00 con 0x7fe4b414fb40 2024-03-30T01:42:15.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.431+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe4b40c6bb0 con 0x7fe4b414fb40 2024-03-30T01:42:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.435+0000 7fe4a5ffb700 1 -- 172.21.15.31:0/3682073138 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe4a8059320 con 0x7fe4b414fb40 2024-03-30T01:42:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe49806ac90 msgr2=0x7fe49806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe49806ac90 0x7fe49806d150 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009d70 tx=0x7fe4b0009210 comp rx=0 tx=0).stop 2024-03-30T01:42:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b414fb40 msgr2=0x7fe4b406ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b414fb40 0x7fe4b406ce70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80090b0 tx=0x7fe4a8004750 comp rx=0 tx=0).stop 2024-03-30T01:42:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 shutdown_connections 2024-03-30T01:42:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe49806ac90 0x7fe49806d150 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4b414ff20 0x7fe4b4149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4b40a7600 0x7fe4b4150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 --2- 172.21.15.31:0/3682073138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4b414fb40 0x7fe4b406ce70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 >> 172.21.15.31:0/3682073138 conn(0x7fe4b40c7d60 msgr2=0x7fe4b40c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:15.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 shutdown_connections 2024-03-30T01:42:15.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.437+0000 7fe4bb782700 1 -- 172.21.15.31:0/3682073138 wait complete. 2024-03-30T01:42:15.445 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-03-30T01:42:15.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:15.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.586+0000 7f9e9c679700 1 Processor -- start 2024-03-30T01:42:15.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.586+0000 7f9e9c679700 1 -- start start 2024-03-30T01:42:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.587+0000 7f9e9c679700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e94154d20 0x7f9e94155100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.587+0000 7f9e9c679700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e940a88c0 con 0x7f9e940c3530 2024-03-30T01:42:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.587+0000 7f9e9c679700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e940a85e0 con 0x7f9e940722b0 2024-03-30T01:42:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.587+0000 7f9e9c679700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940c3110 con 0x7f9e94154d20 2024-03-30T01:42:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.587+0000 7f9e9ac16700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9e940c3530 0x7f9e941479d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45936/0 (socket says 172.21.15.31:45936) 2024-03-30T01:42:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.587+0000 7f9e9ac16700 1 -- 172.21.15.31:0/2631825145 learned_addr learned my addr 172.21.15.31:0/2631825145 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.587+0000 7f9e9a415700 1 --2- 172.21.15.31:0/2631825145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e94154d20 0x7f9e94155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e99413700 1 -- 172.21.15.31:0/2631825145 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112372368 0 0) 0x7f9e940a88c0 con 0x7f9e940c3530 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e99413700 1 -- 172.21.15.31:0/2631825145 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e800035b0 con 0x7f9e940c3530 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e99413700 1 -- 172.21.15.31:0/2631825145 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420238816 0 0) 0x7f9e940a85e0 con 0x7f9e940722b0 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e99413700 1 -- 172.21.15.31:0/2631825145 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e940a88c0 con 0x7f9e940722b0 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e9a415700 1 -- 172.21.15.31:0/2631825145 >> v1:172.21.15.59:6789/0 conn(0x7f9e940722b0 legacy=0x7f9e9406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e9a415700 1 -- 172.21.15.31:0/2631825145 >> v1:172.21.15.31:6789/0 conn(0x7f9e940c3530 legacy=0x7f9e941479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e9a415700 1 -- 172.21.15.31:0/2631825145 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940c2a50 con 0x7f9e94154d20 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e9a415700 1 --2- 172.21.15.31:0/2631825145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e94154d20 0x7f9e94155100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009870 tx=0x7f9e90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf1f2f54543f50df server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e99413700 1 -- 172.21.15.31:0/2631825145 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e90011830 con 0x7f9e94154d20 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e99413700 1 -- 172.21.15.31:0/2631825145 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e90011990 con 0x7f9e94154d20 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.588+0000 7f9e99413700 1 -- 172.21.15.31:0/2631825145 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e90011d20 con 0x7f9e94154d20 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.589+0000 7f9e9c679700 1 -- 172.21.15.31:0/2631825145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e94154d20 msgr2=0x7f9e94155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.589+0000 7f9e9c679700 1 --2- 172.21.15.31:0/2631825145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e94154d20 0x7f9e94155100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009870 tx=0x7f9e90009b80 comp rx=0 tx=0).stop 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.589+0000 7f9e9c679700 1 -- 172.21.15.31:0/2631825145 shutdown_connections 2024-03-30T01:42:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.589+0000 7f9e9c679700 1 --2- 172.21.15.31:0/2631825145 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e94154d20 0x7f9e94155100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.589+0000 7f9e9c679700 1 -- 172.21.15.31:0/2631825145 >> 172.21.15.31:0/2631825145 conn(0x7f9e940c3a60 msgr2=0x7f9e940c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.589+0000 7f9e9c679700 1 -- 172.21.15.31:0/2631825145 shutdown_connections 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.589+0000 7f9e9c679700 1 -- 172.21.15.31:0/2631825145 wait complete. 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.590+0000 7f9e9c679700 1 Processor -- start 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.590+0000 7f9e9c679700 1 -- start start 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9c679700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e940722b0 0x7f9e941e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9c679700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e940c3530 0x7f9e940c0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9c679700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e94154d20 0x7f9e94142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9c679700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940c12b0 con 0x7f9e940722b0 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9c679700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940aa1d0 con 0x7f9e94154d20 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9c679700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940aa4b0 con 0x7f9e940c3530 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9a415700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e940722b0 0x7f9e941e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9a415700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e940722b0 0x7f9e941e9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48720/0 (socket says 172.21.15.31:48720) 2024-03-30T01:42:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9ac16700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e94154d20 0x7f9e94142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9a415700 1 -- 172.21.15.31:0/2442772105 learned_addr learned my addr 172.21.15.31:0/2442772105 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e99c14700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e940c3530 0x7f9e940c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9a415700 1 -- 172.21.15.31:0/2442772105 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e940c3530 msgr2=0x7f9e940c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9a415700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e940c3530 0x7f9e940c0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9a415700 1 -- 172.21.15.31:0/2442772105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e94154d20 msgr2=0x7f9e94142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9a415700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e94154d20 0x7f9e94142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.591+0000 7f9e9a415700 1 -- 172.21.15.31:0/2442772105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e900094a0 con 0x7f9e940722b0 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.592+0000 7f9e9a415700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e940722b0 0x7f9e941e9c10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9e90014040 tx=0x7f9e90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.592+0000 7f9e8affd700 1 -- 172.21.15.31:0/2442772105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9001a8f0 con 0x7f9e940722b0 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.592+0000 7f9e8affd700 1 -- 172.21.15.31:0/2442772105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9001aa50 con 0x7f9e940722b0 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.592+0000 7f9e8affd700 1 -- 172.21.15.31:0/2442772105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9001adc0 con 0x7f9e940722b0 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.592+0000 7f9e9c679700 1 -- 172.21.15.31:0/2442772105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e940aa790 con 0x7f9e940722b0 2024-03-30T01:42:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.592+0000 7f9e9c679700 1 -- 172.21.15.31:0/2442772105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e940c0b40 con 0x7f9e940722b0 2024-03-30T01:42:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.593+0000 7f9e88ff9700 1 -- 172.21.15.31:0/2442772105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e700052f0 con 0x7f9e940722b0 2024-03-30T01:42:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.594+0000 7f9e8affd700 1 -- 172.21.15.31:0/2442772105 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9e9002d450 con 0x7f9e940722b0 2024-03-30T01:42:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.594+0000 7f9e8affd700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9e7806ac90 0x7f9e7806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.594+0000 7f9e8affd700 1 -- 172.21.15.31:0/2442772105 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9e9008a540 con 0x7f9e940722b0 2024-03-30T01:42:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.594+0000 7f9e99c14700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9e7806ac90 0x7f9e7806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.595+0000 7f9e99c14700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9e7806ac90 0x7f9e7806d150 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9e94151610 tx=0x7f9e84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.599+0000 7f9e8affd700 1 -- 172.21.15.31:0/2442772105 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9e90055670 con 0x7f9e940722b0 2024-03-30T01:42:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.863+0000 7f9e88ff9700 1 -- 172.21.15.31:0/2442772105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e700055a0 con 0x7f9e940722b0 2024-03-30T01:42:15.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.863+0000 7f9e8affd700 1 -- 172.21.15.31:0/2442772105 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f9e90015090 con 0x7f9e940722b0 2024-03-30T01:42:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:10.264695+0000","modified":"2024-03-30T01:42:13.278071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/1616247357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1616247357},{"type":"v1","addr":"172.21.15.31:6837","nonce":1616247357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-30T01:42:15.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.865+0000 7f9e88ff9700 1 -- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9e7806ac90 msgr2=0x7f9e7806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.865+0000 7f9e88ff9700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9e7806ac90 0x7f9e7806d150 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9e94151610 tx=0x7f9e84009040 comp rx=0 tx=0).stop 2024-03-30T01:42:15.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 -- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e940722b0 msgr2=0x7f9e941e9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e940722b0 0x7f9e941e9c10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9e90014040 tx=0x7f9e90004750 comp rx=0 tx=0).stop 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 -- 172.21.15.31:0/2442772105 shutdown_connections 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e940722b0 0x7f9e941e9c10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9e7806ac90 0x7f9e7806d150 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e94154d20 0x7f9e94142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 --2- 172.21.15.31:0/2442772105 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e940c3530 0x7f9e940c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 -- 172.21.15.31:0/2442772105 >> 172.21.15.31:0/2442772105 conn(0x7f9e940c3a60 msgr2=0x7f9e94153360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 -- 172.21.15.31:0/2442772105 shutdown_connections 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:15.866+0000 7f9e88ff9700 1 -- 172.21.15.31:0/2442772105 wait complete. 2024-03-30T01:42:15.873 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-03-30T01:42:16.289 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T01:42:10.264695+0000', 'modified': '2024-03-30T01:42:13.278071+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5238}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5238': {'gid': 5238, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.31:6837/1616247357', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6835', 'nonce': 1616247357}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 1616247357}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T01:42:16.289 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T01:42:16.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T01:42:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.431+0000 7f939d295700 1 Processor -- start 2024-03-30T01:42:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.431+0000 7f939d295700 1 -- start start 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.431+0000 7f939d295700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93980c4010 0x7f93981517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.431+0000 7f939d295700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f939806eed0 0x7f9398151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.431+0000 7f939d295700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939806dd90 con 0x7f9398071260 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.431+0000 7f939d295700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939806e070 con 0x7f939806eed0 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.431+0000 7f939d295700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939806e350 con 0x7f93980c4010 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f93977fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9398071260 0x7f9398157d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45946/0 (socket says 172.21.15.31:45946) 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f93977fe700 1 -- 172.21.15.31:0/3142780399 learned_addr learned my addr 172.21.15.31:0/3142780399 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f93967fc700 1 --2- 172.21.15.31:0/3142780399 >> v2:172.21.15.59:3300/0 conn(0x7f939806eed0 0x7f9398151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f9396ffd700 1 --2- 172.21.15.31:0/3142780399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93980c4010 0x7f93981517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f9395ffb700 1 -- 172.21.15.31:0/3142780399 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614871098 0 0) 0x7f939806dd90 con 0x7f9398071260 2024-03-30T01:42:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f9395ffb700 1 -- 172.21.15.31:0/3142780399 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93800035b0 con 0x7f9398071260 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f9396ffd700 1 -- 172.21.15.31:0/3142780399 >> v2:172.21.15.59:3300/0 conn(0x7f939806eed0 msgr2=0x7f9398151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f9396ffd700 1 --2- 172.21.15.31:0/3142780399 >> v2:172.21.15.59:3300/0 conn(0x7f939806eed0 0x7f9398151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f9396ffd700 1 -- 172.21.15.31:0/3142780399 >> v1:172.21.15.31:6789/0 conn(0x7f9398071260 legacy=0x7f9398157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f9396ffd700 1 -- 172.21.15.31:0/3142780399 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939806e630 con 0x7f93980c4010 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.432+0000 7f9396ffd700 1 --2- 172.21.15.31:0/3142780399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93980c4010 0x7f93981517b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9388009840 tx=0x7f9388009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1ebd731bd4de036 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f9395ffb700 1 -- 172.21.15.31:0/3142780399 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388011830 con 0x7f93980c4010 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f9395ffb700 1 -- 172.21.15.31:0/3142780399 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9388011990 con 0x7f93980c4010 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f9395ffb700 1 -- 172.21.15.31:0/3142780399 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388011d20 con 0x7f93980c4010 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f939d295700 1 -- 172.21.15.31:0/3142780399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93980c4010 msgr2=0x7f93981517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f939d295700 1 --2- 172.21.15.31:0/3142780399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93980c4010 0x7f93981517b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9388009840 tx=0x7f9388009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f939d295700 1 -- 172.21.15.31:0/3142780399 shutdown_connections 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f939d295700 1 --2- 172.21.15.31:0/3142780399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93980c4010 0x7f93981517b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f939d295700 1 --2- 172.21.15.31:0/3142780399 >> v2:172.21.15.59:3300/0 conn(0x7f939806eed0 0x7f9398151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f939d295700 1 -- 172.21.15.31:0/3142780399 >> 172.21.15.31:0/3142780399 conn(0x7f939806bc30 msgr2=0x7f939806c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f939d295700 1 -- 172.21.15.31:0/3142780399 shutdown_connections 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.433+0000 7f939d295700 1 -- 172.21.15.31:0/3142780399 wait complete. 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.435+0000 7f939d295700 1 Processor -- start 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.435+0000 7f939d295700 1 -- start start 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.435+0000 7f939d295700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f939806eed0 0x7f93981ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.435+0000 7f939d295700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9398071260 0x7f9398143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.435+0000 7f939d295700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93980c4010 0x7f9398143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.435+0000 7f939d295700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93980c5c90 con 0x7f9398071260 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.435+0000 7f939d295700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939814f2f0 con 0x7f93980c4010 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.435+0000 7f939d295700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93981ed760 con 0x7f939806eed0 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93967fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9398071260 0x7f9398143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93967fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9398071260 0x7f9398143780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48732/0 (socket says 172.21.15.31:48732) 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93967fc700 1 -- 172.21.15.31:0/4245223748 learned_addr learned my addr 172.21.15.31:0/4245223748 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f9396ffd700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f939806eed0 0x7f93981ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93977fe700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93980c4010 0x7f9398143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93967fc700 1 -- 172.21.15.31:0/4245223748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f939806eed0 msgr2=0x7f93981ecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93967fc700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f939806eed0 0x7f93981ecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93967fc700 1 -- 172.21.15.31:0/4245223748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93980c4010 msgr2=0x7f9398143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93967fc700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93980c4010 0x7f9398143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93967fc700 1 -- 172.21.15.31:0/4245223748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880094a0 con 0x7f9398071260 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f93967fc700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9398071260 0x7f9398143780 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f938c00e9e0 tx=0x7f938c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f937f7fe700 1 -- 172.21.15.31:0/4245223748 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c009710 con 0x7f9398071260 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f937f7fe700 1 -- 172.21.15.31:0/4245223748 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c00c850 con 0x7f9398071260 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f939d295700 1 -- 172.21.15.31:0/4245223748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93981eda00 con 0x7f9398071260 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f937f7fe700 1 -- 172.21.15.31:0/4245223748 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c00cbc0 con 0x7f9398071260 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.436+0000 7f939d295700 1 -- 172.21.15.31:0/4245223748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93980c6170 con 0x7f9398071260 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.438+0000 7f937f7fe700 1 -- 172.21.15.31:0/4245223748 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f938c00cd20 con 0x7f9398071260 2024-03-30T01:42:16.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.438+0000 7f937f7fe700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f938406ac90 0x7f938406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.439+0000 7f937f7fe700 1 -- 172.21.15.31:0/4245223748 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f938c089530 con 0x7f9398071260 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.439+0000 7f9396ffd700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f938406ac90 0x7f938406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.439+0000 7f9396ffd700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f938406ac90 0x7f938406d150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9388000c00 tx=0x7f938801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.441+0000 7f93a942bb80 1 Processor -- start 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.441+0000 7f93a942bb80 1 -- start start 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d6399ff0 0x5639d6395690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639b090 0x5639d63d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d639a3d0 0x5639d63b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f93a942bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d62f1940 con 0x5639d6399ff0 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f93a942bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d62ef300 con 0x5639d639b090 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f93a942bb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639d62eabf0 con 0x5639d639a3d0 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f9396ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d6399ff0 0x5639d6395690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f9396ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d6399ff0 0x5639d6395690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48740/0 (socket says 172.21.15.31:48740) 2024-03-30T01:42:16.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f9396ffd700 1 -- 172.21.15.31:0/1258644255 learned_addr learned my addr 172.21.15.31:0/1258644255 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f93967fc700 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639b090 0x5639d63d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f93977fe700 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d639a3d0 0x5639d63b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f9396ffd700 1 -- 172.21.15.31:0/1258644255 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d639a3d0 msgr2=0x5639d63b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f9396ffd700 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d639a3d0 0x5639d63b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f9396ffd700 1 -- 172.21.15.31:0/1258644255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639b090 msgr2=0x5639d63d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f9396ffd700 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639b090 0x5639d63d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f9396ffd700 1 -- 172.21.15.31:0/1258644255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639d6118b10 con 0x5639d6399ff0 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f93977fe700 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d639a3d0 0x5639d63b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f9396ffd700 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d6399ff0 0x5639d6395690 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9388011960 tx=0x7f9388010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81418526e41580c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.442+0000 7f937cff9700 1 -- 172.21.15.31:0/1258644255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388011d00 con 0x5639d6399ff0 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f937cff9700 1 -- 172.21.15.31:0/1258644255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93880036a0 con 0x5639d6399ff0 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f937cff9700 1 -- 172.21.15.31:0/1258644255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388003a10 con 0x5639d6399ff0 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f93a942bb80 1 -- 172.21.15.31:0/1258644255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d6399ff0 msgr2=0x5639d6395690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d6399ff0 0x5639d6395690 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9388011960 tx=0x7f9388010000 comp rx=0 tx=0).stop 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f93a942bb80 1 -- 172.21.15.31:0/1258644255 shutdown_connections 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d6399ff0 0x5639d6395690 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d639a3d0 0x5639d63b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1258644255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639b090 0x5639d63d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f93a942bb80 1 -- 172.21.15.31:0/1258644255 >> 172.21.15.31:0/1258644255 conn(0x5639d62f8650 msgr2=0x5639d6397ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f93a942bb80 1 -- 172.21.15.31:0/1258644255 shutdown_connections 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.443+0000 7f93a942bb80 1 -- 172.21.15.31:0/1258644255 wait complete. 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.444+0000 7f93a942bb80 1 -- start start 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.444+0000 7f93a942bb80 1 Processor -- start 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 0x5639d63e7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63e90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d63e7b40 0x5639d63e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93a942bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d6286d30 con 0x5639d639a3d0 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93a942bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d62869d0 con 0x5639d63e7b40 2024-03-30T01:42:16.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93a942bb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639d616e650 con 0x5639d6399ff0 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93a942bb80 10 client.0 _ll_register_callbacks cb 0x5639d6290420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93967fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63e90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f9396ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 0x5639d63e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93967fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63e90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48756/0 (socket says 172.21.15.31:48756) 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93967fc700 1 -- 172.21.15.31:0/1061120488 learned_addr learned my addr 172.21.15.31:0/1061120488 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93977fe700 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d63e7b40 0x5639d63e95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93967fc700 1 -- 172.21.15.31:0/1061120488 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 msgr2=0x5639d63e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93967fc700 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 0x5639d63e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93967fc700 1 -- 172.21.15.31:0/1061120488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d63e7b40 msgr2=0x5639d63e95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93967fc700 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d63e7b40 0x5639d63e95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.445+0000 7f93967fc700 1 -- 172.21.15.31:0/1061120488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939806e630 con 0x5639d639a3d0 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93967fc700 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63e90b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f938c057f80 tx=0x7f938c058a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb26f0fce6b046b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f9375ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f9375ffb700 1 -- 172.21.15.31:0/1061120488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c0892c0 con 0x5639d639a3d0 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f9375ffb700 1 -- 172.21.15.31:0/1061120488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f938c05c0b0 con 0x5639d639a3d0 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93a942bb80 1 -- 172.21.15.31:0/1061120488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639d62ab330 con 0x5639d639a3d0 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f9375ffb700 1 -- 172.21.15.31:0/1061120488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938c060670 con 0x5639d639a3d0 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93a942bb80 10 client.5397 fetch_fsmap learned FSMap version 17 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93a942bb80 1 -- 172.21.15.31:0/1061120488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5639d62b0e50 con 0x5639d639a3d0 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f9375ffb700 1 -- 172.21.15.31:0/1061120488 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639d62ab330 con 0x5639d639a3d0 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f9375ffb700 1 -- 172.21.15.31:0/1061120488 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f938c057770 con 0x5639d639a3d0 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93a942bb80 10 client.5397 fetch_fsmap finished waiting for FSMap version 17 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93a942bb80 10 client.5397 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93a942bb80 20 client.5397 populate_metadata read hostname 'smithi031' 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] conn(0x5639d63d8710 0x5639d63d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93a942bb80 4 client.5397 mds_command: new command op to 5238 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f93a942bb80 1 -- 172.21.15.31:0/1061120488 --> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639d62e36b0 con 0x5639d63d8710 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.446+0000 7f9396ffd700 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] conn(0x5639d63d8710 0x5639d63d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.447+0000 7f9375ffb700 10 client.5397 ms_handle_connect on v2:172.21.15.31:6835/1616247357 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.447+0000 7f9396ffd700 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] conn(0x5639d63d8710 0x5639d63d8af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99df0dec1e9c753e server_cookie=490757915df01611 in_seq=0 out_seq=0 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.447+0000 7f9375ffb700 1 -- 172.21.15.31:0/1061120488 <== mds.0 v2:172.21.15.31:6835/1616247357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5639d62e36b0 con 0x5639d63d8710 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.447+0000 7f9375ffb700 10 client.5397 handle_command_reply: tid=0 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.447+0000 7f93a942bb80 1 client.5397 shutdown 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 -- 172.21.15.31:0/1061120488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 msgr2=0x5639d63e90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63e90b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f938c057f80 tx=0x7f938c058a70 comp rx=0 tx=0).stop 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 -- 172.21.15.31:0/1061120488 shutdown_connections 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] conn(0x5639d63d8710 0x5639d63d8af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d63e7b40 0x5639d63e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 0x5639d63e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1061120488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63e90b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f9375ffb700 0 client.5397 ms_handle_reset on v2:172.21.15.31:6835/1616247357 2024-03-30T01:42:16.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 -- 172.21.15.31:0/1061120488 >> 172.21.15.31:0/1061120488 conn(0x5639d62f8650 msgr2=0x5639d63bbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:16.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 -- 172.21.15.31:0/1061120488 shutdown_connections 2024-03-30T01:42:16.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 1 -- 172.21.15.31:0/1061120488 wait complete. 2024-03-30T01:42:16.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.448+0000 7f93a942bb80 20 client.5397 trim_cache size 0 max 16384 2024-03-30T01:42:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.467+0000 7f93a942bb80 1 Processor -- start 2024-03-30T01:42:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.467+0000 7f93a942bb80 1 -- start start 2024-03-30T01:42:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.467+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 0x5639d63e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.467+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639a3d0 0x5639d63d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.467+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d63e7b40 0x5639d63d80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.467+0000 7f93a942bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d6397920 con 0x5639d63e7b40 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.467+0000 7f93a942bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d63d1230 con 0x5639d639a3d0 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.467+0000 7f93a942bb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639d63cd870 con 0x5639d6399ff0 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.467+0000 7f93977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d63e7b40 0x5639d63d80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f93977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d63e7b40 0x5639d63d80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48770/0 (socket says 172.21.15.31:48770) 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f93977fe700 1 -- 172.21.15.31:0/1343337979 learned_addr learned my addr 172.21.15.31:0/1343337979 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f9396ffd700 1 --2- 172.21.15.31:0/1343337979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 0x5639d63e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f93967fc700 1 --2- 172.21.15.31:0/1343337979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639a3d0 0x5639d63d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f93977fe700 1 -- 172.21.15.31:0/1343337979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 msgr2=0x5639d63e8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f93977fe700 1 --2- 172.21.15.31:0/1343337979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 0x5639d63e8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f93977fe700 1 -- 172.21.15.31:0/1343337979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639a3d0 msgr2=0x5639d63d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f93977fe700 1 --2- 172.21.15.31:0/1343337979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639a3d0 0x5639d63d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f93977fe700 1 -- 172.21.15.31:0/1343337979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639d6170c80 con 0x5639d63e7b40 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f93977fe700 1 --2- 172.21.15.31:0/1343337979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d63e7b40 0x5639d63d80b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9390001fd0 tx=0x7f93900125a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debb73934e575818 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f937d7fa700 1 -- 172.21.15.31:0/1343337979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900020c0 con 0x5639d63e7b40 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f937d7fa700 1 -- 172.21.15.31:0/1343337979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939000d070 con 0x5639d63e7b40 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.468+0000 7f937d7fa700 1 -- 172.21.15.31:0/1343337979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390012eb0 con 0x5639d63e7b40 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 -- 172.21.15.31:0/1343337979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d63e7b40 msgr2=0x5639d63d80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1343337979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d63e7b40 0x5639d63d80b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9390001fd0 tx=0x7f93900125a0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 -- 172.21.15.31:0/1343337979 shutdown_connections 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1343337979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d63e7b40 0x5639d63d80b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1343337979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d6399ff0 0x5639d63e8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1343337979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d639a3d0 0x5639d63d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 -- 172.21.15.31:0/1343337979 >> 172.21.15.31:0/1343337979 conn(0x5639d62f8650 msgr2=0x5639d63f8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 -- 172.21.15.31:0/1343337979 shutdown_connections 2024-03-30T01:42:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 -- 172.21.15.31:0/1343337979 wait complete. 2024-03-30T01:42:16.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.469+0000 7f93a942bb80 1 -- start start 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.470+0000 7f93a942bb80 1 Processor -- start 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d6399ff0 0x5639d63b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63ba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93a942bb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d63e7b40 0x5639d63d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93a942bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d6441310 con 0x5639d639a3d0 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93a942bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639d63d4720 con 0x5639d6399ff0 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93a942bb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639d63b5e50 con 0x5639d63e7b40 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93a942bb80 10 client.0 _ll_register_callbacks cb 0x5639d6290420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93967fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93967fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63ba540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48786/0 (socket says 172.21.15.31:48786) 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93967fc700 1 -- 172.21.15.31:0/1848221643 learned_addr learned my addr 172.21.15.31:0/1848221643 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f9396ffd700 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d6399ff0 0x5639d63b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93977fe700 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d63e7b40 0x5639d63d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93967fc700 1 -- 172.21.15.31:0/1848221643 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d63e7b40 msgr2=0x5639d63d3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93967fc700 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d63e7b40 0x5639d63d3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93967fc700 1 -- 172.21.15.31:0/1848221643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d6399ff0 msgr2=0x5639d63b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93967fc700 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d6399ff0 0x5639d63b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93967fc700 1 -- 172.21.15.31:0/1848221643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93800035b0 con 0x5639d639a3d0 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.471+0000 7f93967fc700 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63ba540 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f938c060140 tx=0x7f938c0877e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b418ccc33ff600f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f9375ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f9375ffb700 1 -- 172.21.15.31:0/1848221643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c060670 con 0x5639d639a3d0 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f9375ffb700 1 -- 172.21.15.31:0/1848221643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f938c058980 con 0x5639d639a3d0 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f93a942bb80 1 -- 172.21.15.31:0/1848221643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93880036a0 con 0x5639d639a3d0 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f9375ffb700 1 -- 172.21.15.31:0/1848221643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938c058050 con 0x5639d639a3d0 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f93a942bb80 10 client.5403 fetch_fsmap learned FSMap version 17 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f93a942bb80 1 -- 172.21.15.31:0/1848221643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5639d6116fe0 con 0x5639d639a3d0 2024-03-30T01:42:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f9375ffb700 1 -- 172.21.15.31:0/1848221643 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93880036a0 con 0x5639d639a3d0 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f9375ffb700 1 -- 172.21.15.31:0/1848221643 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f938c0813d0 con 0x5639d639a3d0 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f93a942bb80 10 client.5403 fetch_fsmap finished waiting for FSMap version 17 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f93a942bb80 10 client.5403 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f93a942bb80 20 client.5403 populate_metadata read hostname 'smithi031' 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] conn(0x5639d643a5f0 0x5639d643a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f93a942bb80 4 client.5403 mds_command: new command op to 5238 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f93a942bb80 1 -- 172.21.15.31:0/1848221643 --> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5639d6139ca0 con 0x5639d643a5f0 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.472+0000 7f9396ffd700 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] conn(0x5639d643a5f0 0x5639d643a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f9375ffb700 10 client.5403 ms_handle_connect on v2:172.21.15.31:6835/1616247357 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f9396ffd700 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] conn(0x5639d643a5f0 0x5639d643a9d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c45e97132ab506f server_cookie=b7af22cdb7724327 in_seq=0 out_seq=0 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f9375ffb700 1 -- 172.21.15.31:0/1848221643 <== mds.0 v2:172.21.15.31:6835/1616247357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5639d6139ca0 con 0x5639d643a5f0 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f9375ffb700 10 client.5403 handle_command_reply: tid=0 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 client.5403 shutdown 2024-03-30T01:42:16.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 -- 172.21.15.31:0/1848221643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 msgr2=0x5639d63ba540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63ba540 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f938c060140 tx=0x7f938c0877e0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 -- 172.21.15.31:0/1848221643 shutdown_connections 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.31:6835/1616247357,v1:172.21.15.31:6837/1616247357] conn(0x5639d643a5f0 0x5639d643a9d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5639d63e7b40 0x5639d63d3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5639d6399ff0 0x5639d63b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f9375ffb700 0 client.5403 ms_handle_reset on v2:172.21.15.31:6835/1616247357 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 --2- 172.21.15.31:0/1848221643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639d639a3d0 0x5639d63ba540 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 -- 172.21.15.31:0/1848221643 >> 172.21.15.31:0/1848221643 conn(0x5639d62f8650 msgr2=0x5639d63d8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 -- 172.21.15.31:0/1848221643 shutdown_connections 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 1 -- 172.21.15.31:0/1848221643 wait complete. 2024-03-30T01:42:16.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.473+0000 7f93a942bb80 20 client.5403 trim_cache size 0 max 16384 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9739890c-57ec-4503-9fd1-d4b0145023f4", 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5238, 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.2080003770000003, 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 17, 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 39, 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 39, 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 14.519615021 2024-03-30T01:42:16.486 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-03-30T01:42:16.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 -- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f938406ac90 msgr2=0x7f938406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f938406ac90 0x7f938406d150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9388000c00 tx=0x7f938801c040 comp rx=0 tx=0).stop 2024-03-30T01:42:16.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 -- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9398071260 msgr2=0x7f9398143780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9398071260 0x7f9398143780 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f938c00e9e0 tx=0x7f938c00eda0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 -- 172.21.15.31:0/4245223748 shutdown_connections 2024-03-30T01:42:16.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f938406ac90 0x7f938406d150 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93980c4010 0x7f9398143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f939806eed0 0x7f93981ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 --2- 172.21.15.31:0/4245223748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9398071260 0x7f9398143780 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 -- 172.21.15.31:0/4245223748 >> 172.21.15.31:0/4245223748 conn(0x7f939806bc30 msgr2=0x7f93980c1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:16.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 -- 172.21.15.31:0/4245223748 shutdown_connections 2024-03-30T01:42:16.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.474+0000 7f939d295700 1 -- 172.21.15.31:0/4245223748 wait complete. 2024-03-30T01:42:16.494 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:42:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.623+0000 7f6803d77700 1 Processor -- start 2024-03-30T01:42:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.623+0000 7f6803d77700 1 -- start start 2024-03-30T01:42:16.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.624+0000 7f6803d77700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67fc06b170 0x7f67fc155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.624+0000 7f6803d77700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67fc0c8d10 con 0x7f67fc154760 2024-03-30T01:42:16.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.624+0000 7f6803d77700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67fc0c98b0 con 0x7f67fc0bd7f0 2024-03-30T01:42:16.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.624+0000 7f6803d77700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc06bef0 con 0x7f67fc06b170 2024-03-30T01:42:16.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.624+0000 7f6802314700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67fc154760 0x7f67fc15f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45960/0 (socket says 172.21.15.31:45960) 2024-03-30T01:42:16.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.624+0000 7f6802314700 1 -- 172.21.15.31:0/4220334916 learned_addr learned my addr 172.21.15.31:0/4220334916 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.624+0000 7f6801b13700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67fc06b170 0x7f67fc155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.624+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604340974 0 0) 0x7f67fc0c8d10 con 0x7f67fc154760 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67e80035b0 con 0x7f67fc154760 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1593228955 0 0) 0x7f67e80035b0 con 0x7f67fc154760 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67fc06b170 msgr2=0x7f67fc155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67fc06b170 0x7f67fc155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 >> v1:172.21.15.59:6789/0 conn(0x7f67fc0bd7f0 legacy=0x7f67fc0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc0c8100 con 0x7f67fc154760 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f67f40019f0 con 0x7f67fc154760 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 >> v1:172.21.15.31:6789/0 conn(0x7f67fc154760 legacy=0x7f67fc15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67fc06b170 0x7f67e8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e8003f70 0x7f67e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67e8004d90 0x7f67e8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e800bbd0 con 0x7f67e8003f70 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e800bd40 con 0x7f67e8004d90 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e800bf00 con 0x7f67fc06b170 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6801312700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e8003f70 0x7f67e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6801b13700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67fc06b170 0x7f67e8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6802314700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67e8004d90 0x7f67e8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6801312700 1 -- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67fc06b170 msgr2=0x7f67e8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6801312700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67fc06b170 0x7f67e8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6801312700 1 -- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67e8004d90 msgr2=0x7f67e8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6801312700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67e8004d90 0x7f67e8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.625+0000 7f6801312700 1 -- 172.21.15.31:0/4220334916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e80035b0 con 0x7f67e8003f70 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6801312700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e8003f70 0x7f67e8004850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f67ec001880 tx=0x7f67e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ac75d682298db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec001370 con 0x7f67e8003f70 2024-03-30T01:42:16.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec012640 con 0x7f67e8003f70 2024-03-30T01:42:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6800b11700 1 -- 172.21.15.31:0/4220334916 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec0129f0 con 0x7f67e8003f70 2024-03-30T01:42:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6803d77700 1 -- 172.21.15.31:0/4220334916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e8003f70 msgr2=0x7f67e8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6803d77700 1 --2- 172.21.15.31:0/4220334916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e8003f70 0x7f67e8004850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f67ec001880 tx=0x7f67e8005880 comp rx=0 tx=0).stop 2024-03-30T01:42:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6802314700 1 -- 172.21.15.31:0/4220334916 reap_dead start 2024-03-30T01:42:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6803d77700 1 -- 172.21.15.31:0/4220334916 shutdown_connections 2024-03-30T01:42:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6803d77700 1 -- 172.21.15.31:0/4220334916 >> 172.21.15.31:0/4220334916 conn(0x7f67fc0ca0d0 msgr2=0x7f67fc0ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6803d77700 1 -- 172.21.15.31:0/4220334916 shutdown_connections 2024-03-30T01:42:16.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.626+0000 7f6803d77700 1 -- 172.21.15.31:0/4220334916 wait complete. 2024-03-30T01:42:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.628+0000 7f6803d77700 1 Processor -- start 2024-03-30T01:42:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.628+0000 7f6803d77700 1 -- start start 2024-03-30T01:42:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.628+0000 7f6803d77700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e8003f70 0x7f67fc1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.628+0000 7f6803d77700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67fc0bdc00 0x7f67fc0c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.628+0000 7f6803d77700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67fc0c0c20 0x7f67fc0c1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.628+0000 7f6803d77700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc0bff50 con 0x7f67fc0bdc00 2024-03-30T01:42:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.628+0000 7f6803d77700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc150bb0 con 0x7f67fc0c0c20 2024-03-30T01:42:16.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.628+0000 7f6803d77700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc0bf900 con 0x7f67e8003f70 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801312700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67fc0bdc00 0x7f67fc0c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801312700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67fc0bdc00 0x7f67fc0c06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48798/0 (socket says 172.21.15.31:48798) 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801312700 1 -- 172.21.15.31:0/2045923139 learned_addr learned my addr 172.21.15.31:0/2045923139 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801b13700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e8003f70 0x7f67fc1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6802314700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67fc0c0c20 0x7f67fc0c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801312700 1 -- 172.21.15.31:0/2045923139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e8003f70 msgr2=0x7f67fc1505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801312700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e8003f70 0x7f67fc1505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801312700 1 -- 172.21.15.31:0/2045923139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67fc0c0c20 msgr2=0x7f67fc0c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801312700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67fc0c0c20 0x7f67fc0c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801312700 1 -- 172.21.15.31:0/2045923139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc0bfbe0 con 0x7f67fc0bdc00 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6801312700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67fc0bdc00 0x7f67fc0c06e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f67e800bd10 tx=0x7f67ec001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f67f27fc700 1 -- 172.21.15.31:0/2045923139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec001370 con 0x7f67fc0bdc00 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f67f27fc700 1 -- 172.21.15.31:0/2045923139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec021070 con 0x7f67fc0bdc00 2024-03-30T01:42:16.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc0ae920 con 0x7f67fc0bdc00 2024-03-30T01:42:16.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f67f27fc700 1 -- 172.21.15.31:0/2045923139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec006ca0 con 0x7f67fc0bdc00 2024-03-30T01:42:16.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.629+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc0c1d50 con 0x7f67fc0bdc00 2024-03-30T01:42:16.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.630+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc0bf770 con 0x7f67fc0bdc00 2024-03-30T01:42:16.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.631+0000 7f67f27fc700 1 -- 172.21.15.31:0/2045923139 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f67ec023030 con 0x7f67fc0bdc00 2024-03-30T01:42:16.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.631+0000 7f67f27fc700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67e006ac90 0x7f67e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:16.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.631+0000 7f67f27fc700 1 -- 172.21.15.31:0/2045923139 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f67ec08a100 con 0x7f67fc0bdc00 2024-03-30T01:42:16.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.631+0000 7f6801b13700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67e006ac90 0x7f67e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:16.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.632+0000 7f6801b13700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67e006ac90 0x7f67e006d150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009c60 tx=0x7f67f8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:16.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.636+0000 7f67f27fc700 1 -- 172.21.15.31:0/2045923139 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f67ec055230 con 0x7f67fc0bdc00 2024-03-30T01:42:16.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.898+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67fc0bfd70 con 0x7f67fc0bdc00 2024-03-30T01:42:16.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.898+0000 7f67f27fc700 1 -- 172.21.15.31:0/2045923139 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f67ec02b060 con 0x7f67fc0bdc00 2024-03-30T01:42:16.991 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T01:42:16.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67e006ac90 msgr2=0x7f67e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67e006ac90 0x7f67e006d150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009c60 tx=0x7f67f8009210 comp rx=0 tx=0).stop 2024-03-30T01:42:16.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67fc0bdc00 msgr2=0x7f67fc0c06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:16.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67fc0bdc00 0x7f67fc0c06e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f67e800bd10 tx=0x7f67ec001110 comp rx=0 tx=0).stop 2024-03-30T01:42:16.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 shutdown_connections 2024-03-30T01:42:16.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f67e006ac90 0x7f67e006d150 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67fc0c0c20 0x7f67fc0c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67e8003f70 0x7f67fc1505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 --2- 172.21.15.31:0/2045923139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67fc0bdc00 0x7f67fc0c06e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:16.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 >> 172.21.15.31:0/2045923139 conn(0x7f67fc0ca0d0 msgr2=0x7f67fc151680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:16.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 shutdown_connections 2024-03-30T01:42:16.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:16.901+0000 7f6803d77700 1 -- 172.21.15.31:0/2045923139 wait complete. 2024-03-30T01:42:16.994 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T01:42:16.994 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:42:16.994 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-03-30T01:42:16.994 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T01:42:16.994 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T01:42:16.994 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T01:42:16.994 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-30T01:42:16.995 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T01:42:16.995 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:42:16.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:16.995 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-03-30T01:42:17.016 INFO:teuthology.orchestra.run.smithi031.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42188sec preferred_lft 42188sec 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T01:42:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:37:22:42:e7:ee brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T01:42:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::2cb6:4cff:fefa:d1a5/64 scope link 2024-03-30T01:42:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:17.018 INFO:teuthology.orchestra.run.smithi031.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T01:42:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:37:22:42:e7:ee brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T01:42:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c37:22ff:fe42:e7ee/64 scope link 2024-03-30T01:42:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:17.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:17.018 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:42:17.090 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-03-30T01:42:17.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:17.092 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-03-30T01:42:17.157 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-03-30T01:42:17.157 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:42:17.157 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:42:17.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:42:17.158 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:42:17.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:17.158 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:42:17.158 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:42:17.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:42:17.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:42:17.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:42:17.158 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:42:17.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:17.282 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-03-30T01:42:17.303 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T01:42:17.303 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:42:17.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:17.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-03-30T01:42:17.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:42:17.379 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T01:42:17.441 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:42:17.442 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:42:17.442 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T01:42:17.504 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-30T01:42:17.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:17.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:42:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:42:17.602 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-30T01:42:17.602 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:42:17.612 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:42:17.612 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-30T01:42:17.612 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:42:17.612 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:42:17.612 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:42:17.632 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:42:17.632 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-30T01:42:17.632 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:42:17.632 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:42:17.632 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:42:17.632 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790" 2024-03-30T01:42:17.632 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790:/ 2024-03-30T01:42:17.632 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9739890c-57ec-4503-9fd1-d4b0145023f4" will pass to kernel 2024-03-30T01:42:17.635 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T01:42:17.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:17.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T01:42:17.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:17.668 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:42:17.765 INFO:teuthology.orchestra.run.smithi031.stdout:client5335 2024-03-30T01:42:17.766 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:42:17.766 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:17.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:42:17.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.968+0000 7f35b3f70700 1 Processor -- start 2024-03-30T01:42:17.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.968+0000 7f35b3f70700 1 -- start start 2024-03-30T01:42:17.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b3f70700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35ac1561c0 0x7f35ac158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:17.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b3f70700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35ac06b170 0x7f35ac06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:17.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b3f70700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac0c8340 con 0x7f35ac0c93e0 2024-03-30T01:42:17.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b3f70700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac06bc80 con 0x7f35ac1561c0 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b3f70700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac0c8620 con 0x7f35ac06b170 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b1d0c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35ac1561c0 0x7f35ac158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b150b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f35ac0c93e0 0x7f35ac15a100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45966/0 (socket says 172.21.15.31:45966) 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b150b700 1 -- 172.21.15.31:0/1049895871 learned_addr learned my addr 172.21.15.31:0/1049895871 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b1d0c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f35ac1561c0 0x7f35ac158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48804/0 (socket says 172.21.15.31:48804) 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b250d700 1 --2- 172.21.15.31:0/1049895871 >> v2:172.21.15.59:3300/0 conn(0x7f35ac06b170 0x7f35ac06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b1d0c700 1 -- 172.21.15.31:0/1049895871 >> v2:172.21.15.59:3300/0 conn(0x7f35ac06b170 msgr2=0x7f35ac06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b1d0c700 1 --2- 172.21.15.31:0/1049895871 >> v2:172.21.15.59:3300/0 conn(0x7f35ac06b170 0x7f35ac06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b1d0c700 1 -- 172.21.15.31:0/1049895871 >> v1:172.21.15.31:6789/0 conn(0x7f35ac0c93e0 legacy=0x7f35ac15a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b1d0c700 1 -- 172.21.15.31:0/1049895871 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac0c8900 con 0x7f35ac1561c0 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.969+0000 7f35b0d0a700 1 -- 172.21.15.31:0/1049895871 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247532108 0 0) 0x7f35ac0c8340 con 0x7f35ac0c93e0 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.970+0000 7f35b1d0c700 1 --2- 172.21.15.31:0/1049895871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac1561c0 0x7f35ac158c90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0accc0 tx=0x7f35a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7382397f096dc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.970+0000 7f35b0d0a700 1 -- 172.21.15.31:0/1049895871 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8011830 con 0x7f35ac1561c0 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.970+0000 7f35b0d0a700 1 -- 172.21.15.31:0/1049895871 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a8011990 con 0x7f35ac1561c0 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.970+0000 7f35b0d0a700 1 -- 172.21.15.31:0/1049895871 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8011d20 con 0x7f35ac1561c0 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.970+0000 7f35b3f70700 1 -- 172.21.15.31:0/1049895871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac1561c0 msgr2=0x7f35ac158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.970+0000 7f35b3f70700 1 --2- 172.21.15.31:0/1049895871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac1561c0 0x7f35ac158c90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0accc0 tx=0x7f35a8009b20 comp rx=0 tx=0).stop 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.971+0000 7f35b3f70700 1 -- 172.21.15.31:0/1049895871 shutdown_connections 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.971+0000 7f35b3f70700 1 --2- 172.21.15.31:0/1049895871 >> v2:172.21.15.59:3300/0 conn(0x7f35ac06b170 0x7f35ac06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.971+0000 7f35b3f70700 1 --2- 172.21.15.31:0/1049895871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac1561c0 0x7f35ac158c90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.971+0000 7f35b3f70700 1 -- 172.21.15.31:0/1049895871 >> 172.21.15.31:0/1049895871 conn(0x7f35ac0ca0d0 msgr2=0x7f35ac0ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.971+0000 7f35b3f70700 1 -- 172.21.15.31:0/1049895871 shutdown_connections 2024-03-30T01:42:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.971+0000 7f35b3f70700 1 -- 172.21.15.31:0/1049895871 wait complete. 2024-03-30T01:42:17.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.972+0000 7f35b3f70700 1 Processor -- start 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.972+0000 7f35b3f70700 1 -- start start 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b3f70700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac06b170 0x7f35ac0c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b3f70700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35ac0c93e0 0x7f35ac0c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b3f70700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35ac1561c0 0x7f35ac152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b3f70700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0c2fc0 con 0x7f35ac06b170 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b3f70700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0c7440 con 0x7f35ac0c93e0 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b3f70700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0c7990 con 0x7f35ac1561c0 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b1d0c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac06b170 0x7f35ac0c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b1d0c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac06b170 0x7f35ac0c3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48814/0 (socket says 172.21.15.31:48814) 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b1d0c700 1 -- 172.21.15.31:0/2314433629 learned_addr learned my addr 172.21.15.31:0/2314433629 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b250d700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35ac1561c0 0x7f35ac152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b150b700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35ac0c93e0 0x7f35ac0c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b1d0c700 1 -- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35ac1561c0 msgr2=0x7f35ac152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b1d0c700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35ac1561c0 0x7f35ac152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b1d0c700 1 -- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35ac0c93e0 msgr2=0x7f35ac0c2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b1d0c700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35ac0c93e0 0x7f35ac0c2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b1d0c700 1 -- 172.21.15.31:0/2314433629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80094a0 con 0x7f35ac06b170 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b250d700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35ac1561c0 0x7f35ac152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b150b700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35ac0c93e0 0x7f35ac0c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35b1d0c700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac06b170 0x7f35ac0c3a70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f35a80090b0 tx=0x7f35a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.973+0000 7f35a27fc700 1 -- 172.21.15.31:0/2314433629 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a80038a0 con 0x7f35ac06b170 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.974+0000 7f35a27fc700 1 -- 172.21.15.31:0/2314433629 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a801e070 con 0x7f35ac06b170 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.974+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac0c76b0 con 0x7f35ac06b170 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.974+0000 7f35a27fc700 1 -- 172.21.15.31:0/2314433629 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8003c20 con 0x7f35ac06b170 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.974+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac0c3700 con 0x7f35ac06b170 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.975+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ac0c28d0 con 0x7f35ac06b170 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.975+0000 7f35a27fc700 1 -- 172.21.15.31:0/2314433629 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f35a801a420 con 0x7f35ac06b170 2024-03-30T01:42:17.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.976+0000 7f35a27fc700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f359006ac90 0x7f359006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:17.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.976+0000 7f35b150b700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f359006ac90 0x7f359006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:17.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.976+0000 7f35a27fc700 1 -- 172.21.15.31:0/2314433629 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f35a808a520 con 0x7f35ac06b170 2024-03-30T01:42:17.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.976+0000 7f35b150b700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f359006ac90 0x7f359006d150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f359c000f60 tx=0x7f359c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:17.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:17.980+0000 7f35a27fc700 1 -- 172.21.15.31:0/2314433629 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f35a8055650 con 0x7f35ac06b170 2024-03-30T01:42:18.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.243+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f35ac04ea80 con 0x7f35ac06b170 2024-03-30T01:42:18.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.244+0000 7f35a27fc700 1 -- 172.21.15.31:0/2314433629 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f35a8020060 con 0x7f35ac06b170 2024-03-30T01:42:18.250 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T01:42:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f359006ac90 msgr2=0x7f359006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f359006ac90 0x7f359006d150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f359c000f60 tx=0x7f359c009040 comp rx=0 tx=0).stop 2024-03-30T01:42:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac06b170 msgr2=0x7f35ac0c3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac06b170 0x7f35ac0c3a70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f35a80090b0 tx=0x7f35a8004750 comp rx=0 tx=0).stop 2024-03-30T01:42:18.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 shutdown_connections 2024-03-30T01:42:18.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac06b170 0x7f35ac0c3a70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:18.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f359006ac90 0x7f359006d150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:18.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35ac1561c0 0x7f35ac152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:18.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 --2- 172.21.15.31:0/2314433629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35ac0c93e0 0x7f35ac0c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:18.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 >> 172.21.15.31:0/2314433629 conn(0x7f35ac0ca0d0 msgr2=0x7f35ac0aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:18.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 shutdown_connections 2024-03-30T01:42:18.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:18.246+0000 7f35b3f70700 1 -- 172.21.15.31:0/2314433629 wait complete. 2024-03-30T01:42:18.266 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T01:42:18.266 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:42:18.266 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-03-30T01:42:18.266 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T01:42:18.266 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:18.266 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T01:42:18.266 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-30T01:42:18.266 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T01:42:18.266 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:42:18.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:18.267 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-03-30T01:42:18.291 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:42:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:42:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:42:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-03-30T01:42:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:18.292 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-03-30T01:42:18.292 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-03-30T01:42:18.293 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:42:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-03-30T01:42:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:42:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42203sec preferred_lft 42203sec 2024-03-30T01:42:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2024-03-30T01:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-03-30T01:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T01:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether e6:5a:8b:06:76:44 brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T01:42:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::5055:d3ff:fee5:dc2a/64 scope link 2024-03-30T01:42:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:18.295 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T01:42:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether e6:5a:8b:06:76:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T01:42:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::e45a:8bff:fe06:7644/64 scope link 2024-03-30T01:42:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:18.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:18.296 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:42:18.367 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-03-30T01:42:18.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:18.369 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-03-30T01:42:18.434 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-03-30T01:42:18.434 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:42:18.434 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:42:18.434 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:42:18.434 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:42:18.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:18.435 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2024-03-30T01:42:18.435 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-03-30T01:42:18.435 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:42:18.435 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:42:18.435 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:42:18.435 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-03-30T01:42:18.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:18.557 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-03-30T01:42:18.577 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T01:42:18.577 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:42:18.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:18.578 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-03-30T01:42:18.653 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:42:18.653 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:18.715 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:42:18.716 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:42:18.716 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:18.776 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-30T01:42:18.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:18.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:42:18.875 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:42:18.875 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-30T01:42:18.875 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:42:18.886 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:42:18.886 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-30T01:42:18.886 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:42:18.886 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:42:18.886 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:42:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:42:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-30T01:42:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:42:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:42:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:42:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790" 2024-03-30T01:42:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790:/ 2024-03-30T01:42:18.906 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9739890c-57ec-4503-9fd1-d4b0145023f4" will pass to kernel 2024-03-30T01:42:18.909 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T01:42:18.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:18.909 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:18.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:18.941 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:42:19.023 INFO:teuthology.orchestra.run.smithi059.stdout:client5421 2024-03-30T01:42:19.023 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:42:19.023 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:19.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth rm client.testuser 2024-03-30T01:42:19.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.227+0000 7f8e5470d700 1 Processor -- start 2024-03-30T01:42:19.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.227+0000 7f8e5470d700 1 -- start start 2024-03-30T01:42:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.228+0000 7f8e5470d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c14fd30 0x7f8e4c156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.228+0000 7f8e5470d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8e4c150110 0x7f8e4c0c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.228+0000 7f8e5470d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e4c0c88c0 0x7f8e4c0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.228+0000 7f8e5470d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c0c4d60 con 0x7f8e4c0c88c0 2024-03-30T01:42:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.228+0000 7f8e5470d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c0c5040 con 0x7f8e4c150110 2024-03-30T01:42:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.228+0000 7f8e5470d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0c5320 con 0x7f8e4c14fd30 2024-03-30T01:42:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.228+0000 7f8e52caa700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e4c0c88c0 0x7f8e4c0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.228+0000 7f8e52caa700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8e4c0c88c0 0x7f8e4c0ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48824/0 (socket says 172.21.15.31:48824) 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.228+0000 7f8e52caa700 1 -- 172.21.15.31:0/2388204042 learned_addr learned my addr 172.21.15.31:0/2388204042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e51ca8700 1 --2- 172.21.15.31:0/2388204042 >> v2:172.21.15.59:3300/0 conn(0x7f8e4c150110 0x7f8e4c0c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e524a9700 1 --2- 172.21.15.31:0/2388204042 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c14fd30 0x7f8e4c156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e52caa700 1 -- 172.21.15.31:0/2388204042 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c14fd30 msgr2=0x7f8e4c156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e52caa700 1 --2- 172.21.15.31:0/2388204042 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c14fd30 0x7f8e4c156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e52caa700 1 -- 172.21.15.31:0/2388204042 >> v2:172.21.15.59:3300/0 conn(0x7f8e4c150110 msgr2=0x7f8e4c0c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e52caa700 1 --2- 172.21.15.31:0/2388204042 >> v2:172.21.15.59:3300/0 conn(0x7f8e4c150110 0x7f8e4c0c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e52caa700 1 -- 172.21.15.31:0/2388204042 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0c5600 con 0x7f8e4c0c88c0 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e51ca8700 1 --2- 172.21.15.31:0/2388204042 >> v2:172.21.15.59:3300/0 conn(0x7f8e4c150110 0x7f8e4c0c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e524a9700 1 --2- 172.21.15.31:0/2388204042 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c14fd30 0x7f8e4c156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e52caa700 1 --2- 172.21.15.31:0/2388204042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c0c88c0 0x7f8e4c0ac630 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0c7e50 tx=0x7f8e4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cbc9989311fcba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e514a7700 1 -- 172.21.15.31:0/2388204042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4400e070 con 0x7f8e4c0c88c0 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e514a7700 1 -- 172.21.15.31:0/2388204042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4400b9c0 con 0x7f8e4c0c88c0 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.229+0000 7f8e514a7700 1 -- 172.21.15.31:0/2388204042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4400bd70 con 0x7f8e4c0c88c0 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.230+0000 7f8e5470d700 1 -- 172.21.15.31:0/2388204042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c0c88c0 msgr2=0x7f8e4c0ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.230+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2388204042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c0c88c0 0x7f8e4c0ac630 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0c7e50 tx=0x7f8e4400b330 comp rx=0 tx=0).stop 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.230+0000 7f8e5470d700 1 -- 172.21.15.31:0/2388204042 shutdown_connections 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.230+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2388204042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c0c88c0 0x7f8e4c0ac630 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.230+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2388204042 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c14fd30 0x7f8e4c156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.230+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2388204042 >> v2:172.21.15.59:3300/0 conn(0x7f8e4c150110 0x7f8e4c0c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.230+0000 7f8e5470d700 1 -- 172.21.15.31:0/2388204042 >> 172.21.15.31:0/2388204042 conn(0x7f8e4c06ab20 msgr2=0x7f8e4c06af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.230+0000 7f8e5470d700 1 -- 172.21.15.31:0/2388204042 shutdown_connections 2024-03-30T01:42:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.230+0000 7f8e5470d700 1 -- 172.21.15.31:0/2388204042 wait complete. 2024-03-30T01:42:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.231+0000 7f8e5470d700 1 Processor -- start 2024-03-30T01:42:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.231+0000 7f8e5470d700 1 -- start start 2024-03-30T01:42:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e5470d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c0c88c0 0x7f8e4c0c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e5470d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c14fd30 0x7f8e4c1e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e5470d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c150110 0x7f8e4c14c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e5470d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c152910 con 0x7f8e4c14fd30 2024-03-30T01:42:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e5470d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1e7980 con 0x7f8e4c0c88c0 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e5470d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0c1d10 con 0x7f8e4c150110 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e51ca8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c14fd30 0x7f8e4c1e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e51ca8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c14fd30 0x7f8e4c1e80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48828/0 (socket says 172.21.15.31:48828) 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e51ca8700 1 -- 172.21.15.31:0/2369845898 learned_addr learned my addr 172.21.15.31:0/2369845898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e52caa700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c150110 0x7f8e4c14c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e524a9700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c0c88c0 0x7f8e4c0c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e51ca8700 1 -- 172.21.15.31:0/2369845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c150110 msgr2=0x7f8e4c14c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e51ca8700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c150110 0x7f8e4c14c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e51ca8700 1 -- 172.21.15.31:0/2369845898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c0c88c0 msgr2=0x7f8e4c0c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e51ca8700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c0c88c0 0x7f8e4c0c0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.232+0000 7f8e51ca8700 1 -- 172.21.15.31:0/2369845898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44009b30 con 0x7f8e4c14fd30 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.233+0000 7f8e51ca8700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c14fd30 0x7f8e4c1e80b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c7f0 tx=0x7f8e3c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.233+0000 7f8e42ffd700 1 -- 172.21.15.31:0/2369845898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c011800 con 0x7f8e4c14fd30 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.233+0000 7f8e42ffd700 1 -- 172.21.15.31:0/2369845898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c011960 con 0x7f8e4c14fd30 2024-03-30T01:42:19.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.233+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c151790 con 0x7f8e4c14fd30 2024-03-30T01:42:19.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.233+0000 7f8e42ffd700 1 -- 172.21.15.31:0/2369845898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c011cf0 con 0x7f8e4c14fd30 2024-03-30T01:42:19.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.233+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c0c1a30 con 0x7f8e4c14fd30 2024-03-30T01:42:19.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.234+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c073760 con 0x7f8e4c14fd30 2024-03-30T01:42:19.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.234+0000 7f8e42ffd700 1 -- 172.21.15.31:0/2369845898 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8e3c018360 con 0x7f8e4c14fd30 2024-03-30T01:42:19.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.235+0000 7f8e42ffd700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8e3006ac90 0x7f8e3006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.235+0000 7f8e42ffd700 1 -- 172.21.15.31:0/2369845898 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8e3c0891c0 con 0x7f8e4c14fd30 2024-03-30T01:42:19.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.235+0000 7f8e524a9700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8e3006ac90 0x7f8e3006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.235+0000 7f8e524a9700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8e3006ac90 0x7f8e3006d150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e48006fd0 tx=0x7f8e48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:19.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.240+0000 7f8e42ffd700 1 -- 172.21.15.31:0/2369845898 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8e3c0542f0 con 0x7f8e4c14fd30 2024-03-30T01:42:19.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.545+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.testuser"} v 0) v1 -- 0x7f8e4c0c7790 con 0x7f8e4c14fd30 2024-03-30T01:42:19.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.545+0000 7f8e42ffd700 1 -- 172.21.15.31:0/2369845898 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.testuser"}]=0 v13) v1 ==== 84+0+0 (secure 0 0 0) 0x7f8e3c057910 con 0x7f8e4c14fd30 2024-03-30T01:42:19.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8e3006ac90 msgr2=0x7f8e3006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8e3006ac90 0x7f8e3006d150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e48006fd0 tx=0x7f8e48008040 comp rx=0 tx=0).stop 2024-03-30T01:42:19.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c14fd30 msgr2=0x7f8e4c1e80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c14fd30 0x7f8e4c1e80b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c7f0 tx=0x7f8e3c00cb00 comp rx=0 tx=0).stop 2024-03-30T01:42:19.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 shutdown_connections 2024-03-30T01:42:19.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8e3006ac90 0x7f8e3006d150 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e4c150110 0x7f8e4c14c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c0c88c0 0x7f8e4c0c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 --2- 172.21.15.31:0/2369845898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e4c14fd30 0x7f8e4c1e80b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.547+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 >> 172.21.15.31:0/2369845898 conn(0x7f8e4c06ab20 msgr2=0x7f8e4c06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:19.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.548+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 shutdown_connections 2024-03-30T01:42:19.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.548+0000 7f8e5470d700 1 -- 172.21.15.31:0/2369845898 wait complete. 2024-03-30T01:42:19.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:19.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.698+0000 7f5146e7a700 1 Processor -- start 2024-03-30T01:42:19.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.698+0000 7f5146e7a700 1 -- start start 2024-03-30T01:42:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.699+0000 7f5146e7a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f514006b170 0x7f5140155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.699+0000 7f5146e7a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51400c8d10 con 0x7f5140154760 2024-03-30T01:42:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.699+0000 7f5146e7a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51400c98b0 con 0x7f51400bd7f0 2024-03-30T01:42:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.699+0000 7f5146e7a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514006bef0 con 0x7f514006b170 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.699+0000 7f5145417700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5140154760 0x7f514015f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45982/0 (socket says 172.21.15.31:45982) 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.699+0000 7f5145417700 1 -- 172.21.15.31:0/1770259147 learned_addr learned my addr 172.21.15.31:0/1770259147 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.699+0000 7f5144c16700 1 --2- 172.21.15.31:0/1770259147 >> v2:172.21.15.59:3300/0 conn(0x7f514006b170 0x7f5140155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849967877 0 0) 0x7f51400c8d10 con 0x7f5140154760 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51280035b0 con 0x7f5140154760 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 142698998 0 0) 0x7f51280035b0 con 0x7f5140154760 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 >> v2:172.21.15.59:3300/0 conn(0x7f514006b170 msgr2=0x7f5140155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 --2- 172.21.15.31:0/1770259147 >> v2:172.21.15.59:3300/0 conn(0x7f514006b170 0x7f5140155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 >> v1:172.21.15.59:6789/0 conn(0x7f51400bd7f0 legacy=0x7f51400c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51400c8100 con 0x7f5140154760 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f51380019f0 con 0x7f5140154760 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 >> v1:172.21.15.31:6789/0 conn(0x7f5140154760 legacy=0x7f514015f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5128003a00 0x7f5128003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f5128004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5128005330 0x7f512800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512800c1d0 con 0x7f5128004850 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512800c340 con 0x7f5128003a00 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512800c500 con 0x7f5128005330 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f5144c16700 1 --2- 172.21.15.31:0/1770259147 >> v2:172.21.15.59:3300/0 conn(0x7f514006b170 0x7f5140155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513ffff700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f5128004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f5145417700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5128005330 0x7f512800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f5144c16700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5128003a00 0x7f5128003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513ffff700 1 -- 172.21.15.31:0/1770259147 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5128005330 msgr2=0x7f512800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513ffff700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5128005330 0x7f512800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513ffff700 1 -- 172.21.15.31:0/1770259147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5128003a00 msgr2=0x7f5128003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513ffff700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5128003a00 0x7f5128003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f513ffff700 1 -- 172.21.15.31:0/1770259147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280035b0 con 0x7f5128004850 2024-03-30T01:42:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f5145417700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5128005330 0x7f512800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.700+0000 7f5145417700 1 -- 172.21.15.31:0/1770259147 reap_dead start 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f513ffff700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f5128004cd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5134007fd0 tx=0x7f5128005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c215ad9f3239896 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51340052d0 con 0x7f5128004850 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5134005430 con 0x7f5128004850 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f513f7fe700 1 -- 172.21.15.31:0/1770259147 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51340038c0 con 0x7f5128004850 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f5146e7a700 1 -- 172.21.15.31:0/1770259147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 msgr2=0x7f5128004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f5146e7a700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f5128004cd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5134007fd0 tx=0x7f5128005880 comp rx=0 tx=0).stop 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f5146e7a700 1 -- 172.21.15.31:0/1770259147 shutdown_connections 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f5146e7a700 1 --2- 172.21.15.31:0/1770259147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f5128004cd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f5146e7a700 1 -- 172.21.15.31:0/1770259147 >> 172.21.15.31:0/1770259147 conn(0x7f51400ca0d0 msgr2=0x7f51400ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f5146e7a700 1 -- 172.21.15.31:0/1770259147 shutdown_connections 2024-03-30T01:42:19.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.701+0000 7f5146e7a700 1 -- 172.21.15.31:0/1770259147 wait complete. 2024-03-30T01:42:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.703+0000 7f5146e7a700 1 Processor -- start 2024-03-30T01:42:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.703+0000 7f5146e7a700 1 -- start start 2024-03-30T01:42:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.703+0000 7f5146e7a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f51401e9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5146e7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51400c0820 0x7f51400c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5146e7a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51401ed820 0x7f514014f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5146e7a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51400c3a30 con 0x7f5128004850 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5146e7a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51400c6540 con 0x7f51400c0820 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5146e7a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51401e9ec0 con 0x7f51401ed820 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5144c16700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f51401e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5144c16700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f51401e9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48856/0 (socket says 172.21.15.31:48856) 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5144c16700 1 -- 172.21.15.31:0/1200798078 learned_addr learned my addr 172.21.15.31:0/1200798078 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f513ffff700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51400c0820 0x7f51400c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5145417700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51401ed820 0x7f514014f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5144c16700 1 -- 172.21.15.31:0/1200798078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51401ed820 msgr2=0x7f514014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5144c16700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51401ed820 0x7f514014f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5144c16700 1 -- 172.21.15.31:0/1200798078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51400c0820 msgr2=0x7f51400c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5144c16700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51400c0820 0x7f51400c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5144c16700 1 -- 172.21.15.31:0/1200798078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51400c5f80 con 0x7f5128004850 2024-03-30T01:42:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5145417700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51401ed820 0x7f514014f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5144c16700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f51401e9980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5130011900 tx=0x7f5130011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f513d7fa700 1 -- 172.21.15.31:0/1200798078 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513000fae0 con 0x7f5128004850 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f513d7fa700 1 -- 172.21.15.31:0/1200798078 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513000fc40 con 0x7f5128004850 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.704+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51400c6260 con 0x7f5128004850 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.705+0000 7f513d7fa700 1 -- 172.21.15.31:0/1200798078 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130011ea0 con 0x7f5128004850 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.705+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51401ed300 con 0x7f5128004850 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.706+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51401ed4c0 con 0x7f5128004850 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.706+0000 7f513d7fa700 1 -- 172.21.15.31:0/1200798078 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5130015460 con 0x7f5128004850 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.706+0000 7f513d7fa700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512c06ac90 0x7f512c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.707+0000 7f513d7fa700 1 -- 172.21.15.31:0/1200798078 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5130089dc0 con 0x7f5128004850 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.707+0000 7f513ffff700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512c06ac90 0x7f512c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.707+0000 7f513ffff700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512c06ac90 0x7f512c06d150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5134007fd0 tx=0x7f5134009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:19.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.711+0000 7f513d7fa700 1 -- 172.21.15.31:0/1200798078 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5130054ef0 con 0x7f5128004850 2024-03-30T01:42:19.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.928+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f514004ea80 con 0x7f5128004850 2024-03-30T01:42:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.929+0000 7f513d7fa700 1 -- 172.21.15.31:0/1200798078 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16258 (secure 0 0 0) 0x7f5130058510 con 0x7f5128004850 2024-03-30T01:42:19.936 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:19.936 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":39,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:11.266485+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.931+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512c06ac90 msgr2=0x7f512c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.931+0000 7f5146e7a700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512c06ac90 0x7f512c06d150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5134007fd0 tx=0x7f5134009030 comp rx=0 tx=0).stop 2024-03-30T01:42:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.931+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 msgr2=0x7f51401e9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.931+0000 7f5146e7a700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f51401e9980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5130011900 tx=0x7f5130011cc0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.932+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 shutdown_connections 2024-03-30T01:42:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.932+0000 7f5146e7a700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128004850 0x7f51401e9980 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.932+0000 7f5146e7a700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512c06ac90 0x7f512c06d150 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.932+0000 7f5146e7a700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51401ed820 0x7f514014f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.932+0000 7f5146e7a700 1 --2- 172.21.15.31:0/1200798078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51400c0820 0x7f51400c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.932+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 >> 172.21.15.31:0/1200798078 conn(0x7f51400ca0d0 msgr2=0x7f5140151e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.932+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 shutdown_connections 2024-03-30T01:42:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:19.932+0000 7f5146e7a700 1 -- 172.21.15.31:0/1200798078 wait complete. 2024-03-30T01:42:19.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:42:07.881738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:42:09.236870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:42:19.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:42:20.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.085+0000 7f6f24c2b700 1 Processor -- start 2024-03-30T01:42:20.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.085+0000 7f6f24c2b700 1 -- start start 2024-03-30T01:42:20.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.086+0000 7f6f24c2b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f200c46c0 0x7f6f2006ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.086+0000 7f6f24c2b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f200c5d10 0x7f6f200c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.086+0000 7f6f24c2b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f2006bc80 0x7f6f200c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.086+0000 7f6f24c2b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f200c4fe0 con 0x7f6f200c46c0 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.086+0000 7f6f24c2b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f20070610 con 0x7f6f200c5d10 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.086+0000 7f6f24c2b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200708f0 con 0x7f6f2006bc80 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.086+0000 7f6f1e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f200c46c0 0x7f6f2006ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.086+0000 7f6f1e59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f200c46c0 0x7f6f2006ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48862/0 (socket says 172.21.15.31:48862) 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.086+0000 7f6f1e59c700 1 -- 172.21.15.31:0/4107226229 learned_addr learned my addr 172.21.15.31:0/4107226229 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1dd9b700 1 --2- 172.21.15.31:0/4107226229 >> v2:172.21.15.59:3300/0 conn(0x7f6f200c5d10 0x7f6f200c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1ed9d700 1 --2- 172.21.15.31:0/4107226229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f2006bc80 0x7f6f200c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1e59c700 1 -- 172.21.15.31:0/4107226229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f2006bc80 msgr2=0x7f6f200c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:20.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1e59c700 1 --2- 172.21.15.31:0/4107226229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f2006bc80 0x7f6f200c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1e59c700 1 -- 172.21.15.31:0/4107226229 >> v2:172.21.15.59:3300/0 conn(0x7f6f200c5d10 msgr2=0x7f6f200c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:20.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1e59c700 1 --2- 172.21.15.31:0/4107226229 >> v2:172.21.15.59:3300/0 conn(0x7f6f200c5d10 0x7f6f200c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1e59c700 1 -- 172.21.15.31:0/4107226229 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200c3210 con 0x7f6f200c46c0 2024-03-30T01:42:20.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1e59c700 1 --2- 172.21.15.31:0/4107226229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f200c46c0 0x7f6f2006ce80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6f200c5c80 tx=0x7f6f10009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc9c91b3e3e4931 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:20.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1d59a700 1 -- 172.21.15.31:0/4107226229 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000b870 con 0x7f6f200c46c0 2024-03-30T01:42:20.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1d59a700 1 -- 172.21.15.31:0/4107226229 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1000b9d0 con 0x7f6f200c46c0 2024-03-30T01:42:20.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.087+0000 7f6f1d59a700 1 -- 172.21.15.31:0/4107226229 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000bd60 con 0x7f6f200c46c0 2024-03-30T01:42:20.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.088+0000 7f6f24c2b700 1 -- 172.21.15.31:0/4107226229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f200c46c0 msgr2=0x7f6f2006ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.088+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/4107226229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f200c46c0 0x7f6f2006ce80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6f200c5c80 tx=0x7f6f10009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.088+0000 7f6f24c2b700 1 -- 172.21.15.31:0/4107226229 shutdown_connections 2024-03-30T01:42:20.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.088+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/4107226229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f2006bc80 0x7f6f200c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.088+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/4107226229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f200c46c0 0x7f6f2006ce80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.088+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/4107226229 >> v2:172.21.15.59:3300/0 conn(0x7f6f200c5d10 0x7f6f200c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.088+0000 7f6f24c2b700 1 -- 172.21.15.31:0/4107226229 >> 172.21.15.31:0/4107226229 conn(0x7f6f200c7cf0 msgr2=0x7f6f200c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:20.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.088+0000 7f6f24c2b700 1 -- 172.21.15.31:0/4107226229 shutdown_connections 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.088+0000 7f6f24c2b700 1 -- 172.21.15.31:0/4107226229 wait complete. 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.089+0000 7f6f24c2b700 1 Processor -- start 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.089+0000 7f6f24c2b700 1 -- start start 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.090+0000 7f6f24c2b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f2006bc80 0x7f6f201dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.090+0000 7f6f24c2b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f200c46c0 0x7f6f201d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.090+0000 7f6f24c2b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f200c5d10 0x7f6f201d84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.090+0000 7f6f24c2b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f201d7b90 con 0x7f6f2006bc80 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.090+0000 7f6f24c2b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200a6f00 con 0x7f6f200c46c0 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.090+0000 7f6f24c2b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f201deb50 con 0x7f6f200c5d10 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.090+0000 7f6f1e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f2006bc80 0x7f6f201dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.090+0000 7f6f1e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f2006bc80 0x7f6f201dfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48866/0 (socket says 172.21.15.31:48866) 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.090+0000 7f6f1e59c700 1 -- 172.21.15.31:0/2247542962 learned_addr learned my addr 172.21.15.31:0/2247542962 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1dd9b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f200c46c0 0x7f6f201d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1ed9d700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f200c5d10 0x7f6f201d84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1e59c700 1 -- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f200c5d10 msgr2=0x7f6f201d84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1e59c700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f200c5d10 0x7f6f201d84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1e59c700 1 -- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f200c46c0 msgr2=0x7f6f201d7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1e59c700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f200c46c0 0x7f6f201d7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1e59c700 1 -- 172.21.15.31:0/2247542962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f10009550 con 0x7f6f2006bc80 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1dd9b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f200c46c0 0x7f6f201d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1ed9d700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f200c5d10 0x7f6f201d84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f1e59c700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f2006bc80 0x7f6f201dfba0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6f10000c00 tx=0x7f6f10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f0effd700 1 -- 172.21.15.31:0/2247542962 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10004540 con 0x7f6f2006bc80 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f0effd700 1 -- 172.21.15.31:0/2247542962 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f100036a0 con 0x7f6f2006bc80 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f201dee30 con 0x7f6f2006bc80 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f0effd700 1 -- 172.21.15.31:0/2247542962 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10003a50 con 0x7f6f2006bc80 2024-03-30T01:42:20.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.091+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f201df3f0 con 0x7f6f2006bc80 2024-03-30T01:42:20.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.092+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f201df580 con 0x7f6f2006bc80 2024-03-30T01:42:20.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.093+0000 7f6f0effd700 1 -- 172.21.15.31:0/2247542962 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6f10021070 con 0x7f6f2006bc80 2024-03-30T01:42:20.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.093+0000 7f6f0effd700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f0006ac90 0x7f6f0006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.093+0000 7f6f0effd700 1 -- 172.21.15.31:0/2247542962 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6f1008ab40 con 0x7f6f2006bc80 2024-03-30T01:42:20.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.093+0000 7f6f1dd9b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f0006ac90 0x7f6f0006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.094+0000 7f6f1dd9b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f0006ac90 0x7f6f0006d150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6f20070560 tx=0x7f6f14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:20.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.098+0000 7f6f0effd700 1 -- 172.21.15.31:0/2247542962 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6f201df580 con 0x7f6f2006bc80 2024-03-30T01:42:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.311+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f201df580 con 0x7f6f2006bc80 2024-03-30T01:42:20.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.311+0000 7f6f0effd700 1 -- 172.21.15.31:0/2247542962 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f201df580 con 0x7f6f2006bc80 2024-03-30T01:42:20.318 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f0006ac90 msgr2=0x7f6f0006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f0006ac90 0x7f6f0006d150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6f20070560 tx=0x7f6f14008040 comp rx=0 tx=0).stop 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f2006bc80 msgr2=0x7f6f201dfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f2006bc80 0x7f6f201dfba0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6f10000c00 tx=0x7f6f10004750 comp rx=0 tx=0).stop 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 shutdown_connections 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f2006bc80 0x7f6f201dfba0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f0006ac90 0x7f6f0006d150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f200c5d10 0x7f6f201d84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 --2- 172.21.15.31:0/2247542962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f200c46c0 0x7f6f201d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 >> 172.21.15.31:0/2247542962 conn(0x7f6f200c7cf0 msgr2=0x7f6f200c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 shutdown_connections 2024-03-30T01:42:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.314+0000 7f6f24c2b700 1 -- 172.21.15.31:0/2247542962 wait complete. 2024-03-30T01:42:20.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:42:20.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.468+0000 7f1e3dfec700 1 Processor -- start 2024-03-30T01:42:20.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.468+0000 7f1e3dfec700 1 -- start start 2024-03-30T01:42:20.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e3dfec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e3814ec60 0x7f1e3806a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e3dfec700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e380c1620 0x7f1e3806ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e3dfec700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e38071300 0x7f1e380717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e3dfec700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e381558e0 con 0x7f1e38071300 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e3dfec700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3806b180 con 0x7f1e380c1620 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e3dfec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380c2080 con 0x7f1e3814ec60 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e37fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e38071300 0x7f1e380717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e37fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1e38071300 0x7f1e380717b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48874/0 (socket says 172.21.15.31:48874) 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e37fff700 1 -- 172.21.15.31:0/10174080 learned_addr learned my addr 172.21.15.31:0/10174080 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e377fe700 1 --2- 172.21.15.31:0/10174080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e3814ec60 0x7f1e3806a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e36ffd700 1 --2- 172.21.15.31:0/10174080 >> v2:172.21.15.59:3300/0 conn(0x7f1e380c1620 0x7f1e3806ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e37fff700 1 -- 172.21.15.31:0/10174080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e3814ec60 msgr2=0x7f1e3806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e37fff700 1 --2- 172.21.15.31:0/10174080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e3814ec60 0x7f1e3806a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e37fff700 1 -- 172.21.15.31:0/10174080 >> v2:172.21.15.59:3300/0 conn(0x7f1e380c1620 msgr2=0x7f1e3806ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e37fff700 1 --2- 172.21.15.31:0/10174080 >> v2:172.21.15.59:3300/0 conn(0x7f1e380c1620 0x7f1e3806ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.469+0000 7f1e37fff700 1 -- 172.21.15.31:0/10174080 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38154b20 con 0x7f1e38071300 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e37fff700 1 --2- 172.21.15.31:0/10174080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38071300 0x7f1e380717b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1e3806eb00 tx=0x7f1e3000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8264bcc33de06a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:20.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e367fc700 1 -- 172.21.15.31:0/10174080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30010070 con 0x7f1e38071300 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e367fc700 1 -- 172.21.15.31:0/10174080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000dce0 con 0x7f1e38071300 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e367fc700 1 -- 172.21.15.31:0/10174080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000b310 con 0x7f1e38071300 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e3dfec700 1 -- 172.21.15.31:0/10174080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38071300 msgr2=0x7f1e380717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/10174080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38071300 0x7f1e380717b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1e3806eb00 tx=0x7f1e3000d430 comp rx=0 tx=0).stop 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e3dfec700 1 -- 172.21.15.31:0/10174080 shutdown_connections 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/10174080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38071300 0x7f1e380717b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/10174080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e3814ec60 0x7f1e3806a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/10174080 >> v2:172.21.15.59:3300/0 conn(0x7f1e380c1620 0x7f1e3806ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e3dfec700 1 -- 172.21.15.31:0/10174080 >> 172.21.15.31:0/10174080 conn(0x7f1e380c65c0 msgr2=0x7f1e380c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e3dfec700 1 -- 172.21.15.31:0/10174080 shutdown_connections 2024-03-30T01:42:20.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.470+0000 7f1e3dfec700 1 -- 172.21.15.31:0/10174080 wait complete. 2024-03-30T01:42:20.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.472+0000 7f1e3dfec700 1 Processor -- start 2024-03-30T01:42:20.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.472+0000 7f1e3dfec700 1 -- start start 2024-03-30T01:42:20.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e3dfec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e38071300 0x7f1e38145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e3dfec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e380c1620 0x7f1e381450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e3dfec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38146720 0x7f1e381455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e3dfec700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e381476a0 con 0x7f1e38146720 2024-03-30T01:42:20.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e3dfec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e38152760 con 0x7f1e380c1620 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e3dfec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e381529b0 con 0x7f1e38071300 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38146720 0x7f1e381455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38146720 0x7f1e381455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48890/0 (socket says 172.21.15.31:48890) 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e37fff700 1 -- 172.21.15.31:0/3369050337 learned_addr learned my addr 172.21.15.31:0/3369050337 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e36ffd700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e380c1620 0x7f1e381450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e377fe700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e38071300 0x7f1e38145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e37fff700 1 -- 172.21.15.31:0/3369050337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e38071300 msgr2=0x7f1e38145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e37fff700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e38071300 0x7f1e38145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e37fff700 1 -- 172.21.15.31:0/3369050337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e380c1620 msgr2=0x7f1e381450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e37fff700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e380c1620 0x7f1e381450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e37fff700 1 -- 172.21.15.31:0/3369050337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3000bd40 con 0x7f1e38146720 2024-03-30T01:42:20.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e36ffd700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e380c1620 0x7f1e381450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e37fff700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38146720 0x7f1e381455e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1e38066800 tx=0x7f1e30004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.473+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3369050337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30009a10 con 0x7f1e38146720 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.474+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3369050337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30009b70 con 0x7f1e38146720 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.474+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e38152f50 con 0x7f1e38146720 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.474+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3369050337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30004040 con 0x7f1e38146720 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.474+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e38153230 con 0x7f1e38146720 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.475+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e38066d90 con 0x7f1e38146720 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.475+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3369050337 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1e300041a0 con 0x7f1e38146720 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.476+0000 7f1e2ffff700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1e1806ac90 0x7f1e1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.476+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3369050337 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1e3008cdb0 con 0x7f1e38146720 2024-03-30T01:42:20.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.476+0000 7f1e377fe700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1e1806ac90 0x7f1e1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.476+0000 7f1e377fe700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1e1806ac90 0x7f1e1806d150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009b70 tx=0x7f1e28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:20.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.481+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3369050337 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1e30057ee0 con 0x7f1e38146720 2024-03-30T01:42:20.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.692+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e38152db0 con 0x7f1e38146720 2024-03-30T01:42:20.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.692+0000 7f1e2ffff700 1 -- 172.21.15.31:0/3369050337 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e38152db0 con 0x7f1e38146720 2024-03-30T01:42:20.702 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1e1806ac90 msgr2=0x7f1e1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1e1806ac90 0x7f1e1806d150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009b70 tx=0x7f1e28008040 comp rx=0 tx=0).stop 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38146720 msgr2=0x7f1e381455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38146720 0x7f1e381455e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1e38066800 tx=0x7f1e30004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 shutdown_connections 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e38146720 0x7f1e381455e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1e1806ac90 0x7f1e1806d150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e38071300 0x7f1e38145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 --2- 172.21.15.31:0/3369050337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e380c1620 0x7f1e381450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 >> 172.21.15.31:0/3369050337 conn(0x7f1e380c65c0 msgr2=0x7f1e380c69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:20.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 shutdown_connections 2024-03-30T01:42:20.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.695+0000 7f1e3dfec700 1 -- 172.21.15.31:0/3369050337 wait complete. 2024-03-30T01:42:20.715 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:42:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.849+0000 7f391d863700 1 Processor -- start 2024-03-30T01:42:20.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.849+0000 7f391d863700 1 -- start start 2024-03-30T01:42:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f391d863700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3918155530 0x7f39180a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f391d863700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39180c7810 0x7f39180c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f391d863700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39180c31c0 con 0x7f39180c4a00 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f391d863700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39180c41a0 con 0x7f39180c7810 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f391d863700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39180c34a0 con 0x7f3918155530 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f39167fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39180c7810 0x7f39180c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f39167fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f39180c7810 0x7f39180c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48902/0 (socket says 172.21.15.31:48902) 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f39167fc700 1 -- 172.21.15.31:0/1471115783 learned_addr learned my addr 172.21.15.31:0/1471115783 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f39167fc700 1 -- 172.21.15.31:0/1471115783 >> v2:172.21.15.59:3300/0 conn(0x7f3918155530 msgr2=0x7f39180a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f3916ffd700 1 --2- 172.21.15.31:0/1471115783 >> v2:172.21.15.59:3300/0 conn(0x7f3918155530 0x7f39180a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f39167fc700 1 --2- 172.21.15.31:0/1471115783 >> v2:172.21.15.59:3300/0 conn(0x7f3918155530 0x7f39180a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f39167fc700 1 -- 172.21.15.31:0/1471115783 >> v1:172.21.15.59:6789/0 conn(0x7f39180c4a00 legacy=0x7f3918157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f39167fc700 1 -- 172.21.15.31:0/1471115783 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39180c2890 con 0x7f39180c7810 2024-03-30T01:42:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.850+0000 7f39167fc700 1 --2- 172.21.15.31:0/1471115783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c7810 0x7f39180c44c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f39180c5670 tx=0x7f3900009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597c91f99cb682a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f3915ffb700 1 -- 172.21.15.31:0/1471115783 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900004230 con 0x7f39180c7810 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f3915ffb700 1 -- 172.21.15.31:0/1471115783 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900004390 con 0x7f39180c7810 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f3915ffb700 1 -- 172.21.15.31:0/1471115783 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39000036a0 con 0x7f39180c7810 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f391d863700 1 -- 172.21.15.31:0/1471115783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c7810 msgr2=0x7f39180c44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f391d863700 1 --2- 172.21.15.31:0/1471115783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c7810 0x7f39180c44c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f39180c5670 tx=0x7f3900009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f391d863700 1 -- 172.21.15.31:0/1471115783 shutdown_connections 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f391d863700 1 --2- 172.21.15.31:0/1471115783 >> v2:172.21.15.59:3300/0 conn(0x7f3918155530 0x7f39180a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f391d863700 1 --2- 172.21.15.31:0/1471115783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c7810 0x7f39180c44c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f391d863700 1 -- 172.21.15.31:0/1471115783 >> 172.21.15.31:0/1471115783 conn(0x7f39180c7d60 msgr2=0x7f39180c8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f391d863700 1 -- 172.21.15.31:0/1471115783 shutdown_connections 2024-03-30T01:42:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.851+0000 7f391d863700 1 -- 172.21.15.31:0/1471115783 wait complete. 2024-03-30T01:42:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.853+0000 7f391d863700 1 Processor -- start 2024-03-30T01:42:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.853+0000 7f391d863700 1 -- start start 2024-03-30T01:42:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f391d863700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c4a00 0x7f39181575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f391d863700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39180c7810 0x7f391806f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f391d863700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3918155530 0x7f391806f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f391d863700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39180acf50 con 0x7f39180c4a00 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f391d863700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39180a6300 con 0x7f3918155530 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f391d863700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39181e98c0 con 0x7f39180c7810 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c4a00 0x7f39181575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f3916ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c4a00 0x7f39181575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48910/0 (socket says 172.21.15.31:48910) 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f3916ffd700 1 -- 172.21.15.31:0/351347436 learned_addr learned my addr 172.21.15.31:0/351347436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f39167fc700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39180c7810 0x7f391806f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.854+0000 7f39177fe700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3918155530 0x7f391806f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f3916ffd700 1 -- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39180c7810 msgr2=0x7f391806f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f3916ffd700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39180c7810 0x7f391806f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f3916ffd700 1 -- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3918155530 msgr2=0x7f391806f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f3916ffd700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3918155530 0x7f391806f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f3916ffd700 1 -- 172.21.15.31:0/351347436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900009550 con 0x7f39180c4a00 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f39167fc700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39180c7810 0x7f391806f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f39177fe700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3918155530 0x7f391806f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f3916ffd700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c4a00 0x7f39181575c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f390800c7f0 tx=0x7f390800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f390ffff700 1 -- 172.21.15.31:0/351347436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908011800 con 0x7f39180c4a00 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f390ffff700 1 -- 172.21.15.31:0/351347436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3908011960 con 0x7f39180c4a00 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39180c82c0 con 0x7f39180c4a00 2024-03-30T01:42:20.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f390ffff700 1 -- 172.21.15.31:0/351347436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908011cf0 con 0x7f39180c4a00 2024-03-30T01:42:20.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.855+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39180a67f0 con 0x7f39180c4a00 2024-03-30T01:42:20.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.856+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39181e9ae0 con 0x7f39180c4a00 2024-03-30T01:42:20.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.857+0000 7f390ffff700 1 -- 172.21.15.31:0/351347436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3908018360 con 0x7f39180c4a00 2024-03-30T01:42:20.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.857+0000 7f390ffff700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38f806ac90 0x7f38f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:20.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.857+0000 7f390ffff700 1 -- 172.21.15.31:0/351347436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3908089290 con 0x7f39180c4a00 2024-03-30T01:42:20.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.857+0000 7f39167fc700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38f806ac90 0x7f38f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:20.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.858+0000 7f39167fc700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38f806ac90 0x7f38f806d150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3900009130 tx=0x7f3900008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:20.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:20.862+0000 7f390ffff700 1 -- 172.21.15.31:0/351347436 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f39181e9ae0 con 0x7f39180c4a00 2024-03-30T01:42:21.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.077+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39180747d0 con 0x7f39180c4a00 2024-03-30T01:42:21.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.078+0000 7f390ffff700 1 -- 172.21.15.31:0/351347436 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39080543c0 con 0x7f39180c4a00 2024-03-30T01:42:21.084 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38f806ac90 msgr2=0x7f38f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38f806ac90 0x7f38f806d150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3900009130 tx=0x7f3900008030 comp rx=0 tx=0).stop 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c4a00 msgr2=0x7f39181575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c4a00 0x7f39181575c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f390800c7f0 tx=0x7f390800cbb0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 shutdown_connections 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39180c4a00 0x7f39181575c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38f806ac90 0x7f38f806d150 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3918155530 0x7f391806f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 --2- 172.21.15.31:0/351347436 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39180c7810 0x7f391806f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 >> 172.21.15.31:0/351347436 conn(0x7f39180c7d60 msgr2=0x7f39180c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 shutdown_connections 2024-03-30T01:42:21.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.080+0000 7f391d863700 1 -- 172.21.15.31:0/351347436 wait complete. 2024-03-30T01:42:21.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T01:42:21.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.231+0000 7efd5a660700 1 Processor -- start 2024-03-30T01:42:21.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.231+0000 7efd5a660700 1 -- start start 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.231+0000 7efd5a660700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd540c4f60 0x7efd540c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.231+0000 7efd5a660700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efd540c62b0 0x7efd5415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.231+0000 7efd5a660700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd540c7fd0 con 0x7efd5406b400 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.231+0000 7efd5a660700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd54154830 con 0x7efd540c62b0 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.231+0000 7efd5a660700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd540c5380 con 0x7efd540c4f60 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.231+0000 7efd53fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efd5406b400 0x7efd54151a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45994/0 (socket says 172.21.15.31:45994) 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.231+0000 7efd53fff700 1 -- 172.21.15.31:0/3662824252 learned_addr learned my addr 172.21.15.31:0/3662824252 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd537fe700 1 --2- 172.21.15.31:0/3662824252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd540c4f60 0x7efd540c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd58bfd700 1 --2- 172.21.15.31:0/3662824252 >> v2:172.21.15.59:3300/0 conn(0x7efd540c62b0 0x7efd5415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd52ffd700 1 -- 172.21.15.31:0/3662824252 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072286781 0 0) 0x7efd540c7fd0 con 0x7efd5406b400 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd52ffd700 1 -- 172.21.15.31:0/3662824252 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd3c0035b0 con 0x7efd5406b400 2024-03-30T01:42:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd58bfd700 1 -- 172.21.15.31:0/3662824252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd540c4f60 msgr2=0x7efd540c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:21.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd58bfd700 1 --2- 172.21.15.31:0/3662824252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd540c4f60 0x7efd540c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd58bfd700 1 -- 172.21.15.31:0/3662824252 >> v1:172.21.15.31:6789/0 conn(0x7efd5406b400 legacy=0x7efd54151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd58bfd700 1 -- 172.21.15.31:0/3662824252 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54154a20 con 0x7efd540c62b0 2024-03-30T01:42:21.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd537fe700 1 --2- 172.21.15.31:0/3662824252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd540c4f60 0x7efd540c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:21.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd58bfd700 1 --2- 172.21.15.31:0/3662824252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 0x7efd5415f1b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009d90 tx=0x7efd4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a10fcb51a3c047c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:21.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd52ffd700 1 -- 172.21.15.31:0/3662824252 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00e070 con 0x7efd540c62b0 2024-03-30T01:42:21.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd52ffd700 1 -- 172.21.15.31:0/3662824252 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c00bb20 con 0x7efd540c62b0 2024-03-30T01:42:21.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.232+0000 7efd52ffd700 1 -- 172.21.15.31:0/3662824252 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c012430 con 0x7efd540c62b0 2024-03-30T01:42:21.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.233+0000 7efd5a660700 1 -- 172.21.15.31:0/3662824252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 msgr2=0x7efd5415f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.233+0000 7efd5a660700 1 --2- 172.21.15.31:0/3662824252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 0x7efd5415f1b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009d90 tx=0x7efd4c00b330 comp rx=0 tx=0).stop 2024-03-30T01:42:21.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.233+0000 7efd5a660700 1 -- 172.21.15.31:0/3662824252 shutdown_connections 2024-03-30T01:42:21.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.233+0000 7efd5a660700 1 --2- 172.21.15.31:0/3662824252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 0x7efd5415f1b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.233+0000 7efd5a660700 1 --2- 172.21.15.31:0/3662824252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd540c4f60 0x7efd540c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.233+0000 7efd5a660700 1 -- 172.21.15.31:0/3662824252 >> 172.21.15.31:0/3662824252 conn(0x7efd540ca0d0 msgr2=0x7efd540ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.233+0000 7efd5a660700 1 -- 172.21.15.31:0/3662824252 shutdown_connections 2024-03-30T01:42:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.233+0000 7efd5a660700 1 -- 172.21.15.31:0/3662824252 wait complete. 2024-03-30T01:42:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.234+0000 7efd5a660700 1 Processor -- start 2024-03-30T01:42:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.235+0000 7efd5a660700 1 -- start start 2024-03-30T01:42:21.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.235+0000 7efd5a660700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd5406b400 0x7efd54150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.235+0000 7efd5a660700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd540c4f60 0x7efd541de940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.235+0000 7efd5a660700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 0x7efd541dee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.235+0000 7efd5a660700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5414b360 con 0x7efd540c4f60 2024-03-30T01:42:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.235+0000 7efd5a660700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd540c0d40 con 0x7efd540c62b0 2024-03-30T01:42:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.235+0000 7efd5a660700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5414f020 con 0x7efd5406b400 2024-03-30T01:42:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.235+0000 7efd537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd540c4f60 0x7efd541de940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd540c4f60 0x7efd541de940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48924/0 (socket says 172.21.15.31:48924) 2024-03-30T01:42:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd537fe700 1 -- 172.21.15.31:0/3789413225 learned_addr learned my addr 172.21.15.31:0/3789413225 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd53fff700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd5406b400 0x7efd54150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd58bfd700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 0x7efd541dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd537fe700 1 -- 172.21.15.31:0/3789413225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd5406b400 msgr2=0x7efd54150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd537fe700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd5406b400 0x7efd54150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd537fe700 1 -- 172.21.15.31:0/3789413225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 msgr2=0x7efd541dee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd537fe700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 0x7efd541dee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd537fe700 1 -- 172.21.15.31:0/3789413225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480094a0 con 0x7efd540c4f60 2024-03-30T01:42:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd58bfd700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 0x7efd541dee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd537fe700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd540c4f60 0x7efd541de940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efd4800ec10 tx=0x7efd48009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd50ff9700 1 -- 172.21.15.31:0/3789413225 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800c8c0 con 0x7efd540c4f60 2024-03-30T01:42:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd50ff9700 1 -- 172.21.15.31:0/3789413225 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4800ca20 con 0x7efd540c4f60 2024-03-30T01:42:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c009a70 con 0x7efd540c4f60 2024-03-30T01:42:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd50ff9700 1 -- 172.21.15.31:0/3789413225 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800cdd0 con 0x7efd540c4f60 2024-03-30T01:42:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.236+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd54151340 con 0x7efd540c4f60 2024-03-30T01:42:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.237+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd54073680 con 0x7efd540c4f60 2024-03-30T01:42:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.238+0000 7efd50ff9700 1 -- 172.21.15.31:0/3789413225 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efd48013470 con 0x7efd540c4f60 2024-03-30T01:42:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.238+0000 7efd50ff9700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd4006acc0 0x7efd4006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.238+0000 7efd50ff9700 1 -- 172.21.15.31:0/3789413225 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efd48089270 con 0x7efd540c4f60 2024-03-30T01:42:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.238+0000 7efd53fff700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd4006acc0 0x7efd4006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.239+0000 7efd53fff700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd4006acc0 0x7efd4006d180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efd440011e0 tx=0x7efd44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.243+0000 7efd50ff9700 1 -- 172.21.15.31:0/3789413225 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efd48054310 con 0x7efd540c4f60 2024-03-30T01:42:21.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.458+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efd5404ea80 con 0x7efd540c4f60 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.458+0000 7efd50ff9700 1 -- 172.21.15.31:0/3789413225 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) v1 ==== 84+0+207 (secure 0 0 0) 0x7efd48057930 con 0x7efd540c4f60 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 5, 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "cephfs_metadata" 2024-03-30T01:42:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:42:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:42:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 6, 2024-03-30T01:42:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "cephfs_data" 2024-03-30T01:42:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:42:21.466 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd4006acc0 msgr2=0x7efd4006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd4006acc0 0x7efd4006d180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efd440011e0 tx=0x7efd44009040 comp rx=0 tx=0).stop 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd540c4f60 msgr2=0x7efd541de940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd540c4f60 0x7efd541de940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efd4800ec10 tx=0x7efd48009e60 comp rx=0 tx=0).stop 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 shutdown_connections 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efd4006acc0 0x7efd4006d180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd540c62b0 0x7efd541dee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd5406b400 0x7efd54150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 --2- 172.21.15.31:0/3789413225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd540c4f60 0x7efd541de940 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 >> 172.21.15.31:0/3789413225 conn(0x7efd540ca0d0 msgr2=0x7efd541526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 shutdown_connections 2024-03-30T01:42:21.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.461+0000 7efd5a660700 1 -- 172.21.15.31:0/3789413225 wait complete. 2024-03-30T01:42:21.481 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-30T01:42:21.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-30T01:42:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.614+0000 7f31144a6700 1 Processor -- start 2024-03-30T01:42:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.614+0000 7f31144a6700 1 -- start start 2024-03-30T01:42:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.615+0000 7f31144a6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 0x7f310c0c1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.615+0000 7f31144a6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f310c06a700 0x7f310c06ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.615+0000 7f31144a6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c0a8a40 con 0x7f310c155ce0 2024-03-30T01:42:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.615+0000 7f31144a6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310c0a96d0 con 0x7f310c06a700 2024-03-30T01:42:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.615+0000 7f31144a6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310c0c16b0 con 0x7f310c14fe50 2024-03-30T01:42:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3111a41700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f310c06a700 0x7f310c06ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3112242700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 0x7f310c0c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3112a43700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f310c155ce0 0x7f310c156170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:59302/0 (socket says 172.21.15.31:59302) 2024-03-30T01:42:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3112a43700 1 -- 172.21.15.31:0/969698942 learned_addr learned my addr 172.21.15.31:0/969698942 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3111a41700 1 -- 172.21.15.31:0/969698942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 msgr2=0x7f310c0c1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3111a41700 1 --2- 172.21.15.31:0/969698942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 0x7f310c0c1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3111a41700 1 -- 172.21.15.31:0/969698942 >> v1:172.21.15.59:6789/0 conn(0x7f310c155ce0 legacy=0x7f310c156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3111a41700 1 -- 172.21.15.31:0/969698942 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c06f640 con 0x7f310c06a700 2024-03-30T01:42:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3111a41700 1 --2- 172.21.15.31:0/969698942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f310c06a700 0x7f310c06ab80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3108009840 tx=0x7f3108009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a35b54c0f4c7f78b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.616+0000 7f3111240700 1 -- 172.21.15.31:0/969698942 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108011830 con 0x7f310c06a700 2024-03-30T01:42:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f3111240700 1 -- 172.21.15.31:0/969698942 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3108011990 con 0x7f310c06a700 2024-03-30T01:42:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f3111240700 1 -- 172.21.15.31:0/969698942 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108011d20 con 0x7f310c06a700 2024-03-30T01:42:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f31144a6700 1 -- 172.21.15.31:0/969698942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f310c06a700 msgr2=0x7f310c06ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f31144a6700 1 --2- 172.21.15.31:0/969698942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f310c06a700 0x7f310c06ab80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3108009840 tx=0x7f3108009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f31144a6700 1 -- 172.21.15.31:0/969698942 shutdown_connections 2024-03-30T01:42:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f31144a6700 1 --2- 172.21.15.31:0/969698942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 0x7f310c0c1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f31144a6700 1 --2- 172.21.15.31:0/969698942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f310c06a700 0x7f310c06ab80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f31144a6700 1 -- 172.21.15.31:0/969698942 >> 172.21.15.31:0/969698942 conn(0x7f310c0c39f0 msgr2=0x7f310c0c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f31144a6700 1 -- 172.21.15.31:0/969698942 shutdown_connections 2024-03-30T01:42:21.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.617+0000 7f31144a6700 1 -- 172.21.15.31:0/969698942 wait complete. 2024-03-30T01:42:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.618+0000 7f31144a6700 1 Processor -- start 2024-03-30T01:42:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f31144a6700 1 -- start start 2024-03-30T01:42:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f31144a6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c06a700 0x7f310c1e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f31144a6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 0x7f310c146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f31144a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f310c155ce0 0x7f310c1e7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f31144a6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c148c80 con 0x7f310c06a700 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f31144a6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c146cd0 con 0x7f310c155ce0 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f31144a6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310c146fb0 con 0x7f310c14fe50 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f3112242700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c06a700 0x7f310c1e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f3112242700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c06a700 0x7f310c1e8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48930/0 (socket says 172.21.15.31:48930) 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f3112242700 1 -- 172.21.15.31:0/4130938909 learned_addr learned my addr 172.21.15.31:0/4130938909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.619+0000 7f3111a41700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 0x7f310c146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f3112a43700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f310c155ce0 0x7f310c1e7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f3112242700 1 -- 172.21.15.31:0/4130938909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 msgr2=0x7f310c146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f3112242700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 0x7f310c146220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f3112242700 1 -- 172.21.15.31:0/4130938909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f310c155ce0 msgr2=0x7f310c1e7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f3112242700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f310c155ce0 0x7f310c1e7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f3112242700 1 -- 172.21.15.31:0/4130938909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31080094a0 con 0x7f310c06a700 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f3112242700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c06a700 0x7f310c1e8e10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f310000e9b0 tx=0x7f310000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f30feffd700 1 -- 172.21.15.31:0/4130938909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100009710 con 0x7f310c06a700 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f30feffd700 1 -- 172.21.15.31:0/4130938909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310000ee50 con 0x7f310c06a700 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c147290 con 0x7f310c06a700 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f30feffd700 1 -- 172.21.15.31:0/4130938909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310000ca50 con 0x7f310c06a700 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.620+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c147e50 con 0x7f310c06a700 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.621+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c147600 con 0x7f310c06a700 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.622+0000 7f30feffd700 1 -- 172.21.15.31:0/4130938909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f310000cbb0 con 0x7f310c06a700 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.622+0000 7f30feffd700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30f006ac90 0x7f30f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:21.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.623+0000 7f30feffd700 1 -- 172.21.15.31:0/4130938909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3100089420 con 0x7f310c06a700 2024-03-30T01:42:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.623+0000 7f3111a41700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30f006ac90 0x7f30f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:21.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.623+0000 7f3111a41700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30f006ac90 0x7f30f006d150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f310801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:21.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.627+0000 7f30feffd700 1 -- 172.21.15.31:0/4130938909 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3100054550 con 0x7f310c06a700 2024-03-30T01:42:21.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:21.842+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f310c147420 con 0x7f310c06a700 2024-03-30T01:42:22.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.806+0000 7f30feffd700 1 -- 172.21.15.31:0/4130938909 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v40) v1 ==== 140+0+0 (secure 0 0 0) 0x7f310000ce60 con 0x7f310c06a700 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30f006ac90 msgr2=0x7f30f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30f006ac90 0x7f30f006d150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f310801c040 comp rx=0 tx=0).stop 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c06a700 msgr2=0x7f310c1e8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c06a700 0x7f310c1e8e10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f310000e9b0 tx=0x7f310000ecc0 comp rx=0 tx=0).stop 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 shutdown_connections 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f310c06a700 0x7f310c1e8e10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f30f006ac90 0x7f30f006d150 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f310801c040 comp rx=0 tx=0).stop 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f310c155ce0 0x7f310c1e7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 --2- 172.21.15.31:0/4130938909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f310c14fe50 0x7f310c146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.811+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 >> 172.21.15.31:0/4130938909 conn(0x7f310c0c39f0 msgr2=0x7f310c0c2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:22.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.812+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 shutdown_connections 2024-03-30T01:42:22.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.812+0000 7f31144a6700 1 -- 172.21.15.31:0/4130938909 wait complete. 2024-03-30T01:42:22.820 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs2_metadata' created 2024-03-30T01:42:22.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T01:42:22.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.977+0000 7fb5013f7700 1 Processor -- start 2024-03-30T01:42:22.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.977+0000 7fb5013f7700 1 -- start start 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.977+0000 7fb5013f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 0x7fb4fc0c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.977+0000 7fb5013f7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4fc06c3f0 0x7fb4fc06c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.977+0000 7fb5013f7700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc06bc80 con 0x7fb4fc0c48b0 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.977+0000 7fb5013f7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc06cda0 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.977+0000 7fb5013f7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0c7120 con 0x7fb4fc1546e0 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4fb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb4fc06c3f0 0x7fb4fc06c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4faffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb4fc0c48b0 0x7fb4fc0c6940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46002/0 (socket says 172.21.15.31:46002) 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4faffd700 1 -- 172.21.15.31:0/196884963 learned_addr learned my addr 172.21.15.31:0/196884963 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4fa7fc700 1 --2- 172.21.15.31:0/196884963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 0x7fb4fc0c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4fb7fe700 1 -- 172.21.15.31:0/196884963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 msgr2=0x7fb4fc0c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:22.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4fb7fe700 1 --2- 172.21.15.31:0/196884963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 0x7fb4fc0c4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4fb7fe700 1 -- 172.21.15.31:0/196884963 >> v1:172.21.15.31:6789/0 conn(0x7fb4fc0c48b0 legacy=0x7fb4fc0c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4f9ffb700 1 -- 172.21.15.31:0/196884963 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806734988 0 0) 0x7fb4fc06bc80 con 0x7fb4fc0c48b0 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4fb7fe700 1 -- 172.21.15.31:0/196884963 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0c7400 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4fa7fc700 1 --2- 172.21.15.31:0/196884963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 0x7fb4fc0c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4fb7fe700 1 --2- 172.21.15.31:0/196884963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 0x7fb4fc06c860 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009d90 tx=0x7fb4f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1761bbd5c73be0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4f9ffb700 1 -- 172.21.15.31:0/196884963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400e070 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4f9ffb700 1 -- 172.21.15.31:0/196884963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f400bbd0 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.978+0000 7fb4f9ffb700 1 -- 172.21.15.31:0/196884963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4012430 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.979+0000 7fb5013f7700 1 -- 172.21.15.31:0/196884963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 msgr2=0x7fb4fc06c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.979+0000 7fb5013f7700 1 --2- 172.21.15.31:0/196884963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 0x7fb4fc06c860 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009d90 tx=0x7fb4f400b330 comp rx=0 tx=0).stop 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.979+0000 7fb5013f7700 1 -- 172.21.15.31:0/196884963 shutdown_connections 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.979+0000 7fb5013f7700 1 --2- 172.21.15.31:0/196884963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 0x7fb4fc06c860 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.979+0000 7fb5013f7700 1 --2- 172.21.15.31:0/196884963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 0x7fb4fc0c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:22.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.979+0000 7fb5013f7700 1 -- 172.21.15.31:0/196884963 >> 172.21.15.31:0/196884963 conn(0x7fb4fc0abac0 msgr2=0x7fb4fc0abed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:22.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.979+0000 7fb5013f7700 1 -- 172.21.15.31:0/196884963 shutdown_connections 2024-03-30T01:42:22.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.979+0000 7fb5013f7700 1 -- 172.21.15.31:0/196884963 wait complete. 2024-03-30T01:42:22.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.981+0000 7fb5013f7700 1 Processor -- start 2024-03-30T01:42:22.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.981+0000 7fb5013f7700 1 -- start start 2024-03-30T01:42:22.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb5013f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 0x7fb4fc0af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:22.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb5013f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4fc0c48b0 0x7fb4fc0ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:22.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb5013f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 0x7fb4fc0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:22.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb5013f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0c3f60 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb5013f7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0af9e0 con 0x7fb4fc0c48b0 2024-03-30T01:42:22.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb5013f7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc1eb1d0 con 0x7fb4fc1546e0 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4faffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 0x7fb4fc0af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4faffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 0x7fb4fc0af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48938/0 (socket says 172.21.15.31:48938) 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4faffd700 1 -- 172.21.15.31:0/268318610 learned_addr learned my addr 172.21.15.31:0/268318610 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4fb7fe700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 0x7fb4fc0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4fa7fc700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4fc0c48b0 0x7fb4fc0ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4faffd700 1 -- 172.21.15.31:0/268318610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 msgr2=0x7fb4fc0c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4faffd700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 0x7fb4fc0c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4faffd700 1 -- 172.21.15.31:0/268318610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4fc0c48b0 msgr2=0x7fb4fc0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4faffd700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4fc0c48b0 0x7fb4fc0ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.982+0000 7fb4faffd700 1 -- 172.21.15.31:0/268318610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009a70 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.983+0000 7fb4faffd700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 0x7fb4fc0af2e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c550 tx=0x7fb4ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.983+0000 7fb4f37fe700 1 -- 172.21.15.31:0/268318610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec004c80 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.983+0000 7fb4f37fe700 1 -- 172.21.15.31:0/268318610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec004e00 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.983+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc1eb4b0 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.983+0000 7fb4f37fe700 1 -- 172.21.15.31:0/268318610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec007310 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.983+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc1eba70 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.984+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc04ea80 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.984+0000 7fb4f37fe700 1 -- 172.21.15.31:0/268318610 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb4ec007470 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.985+0000 7fb4f37fe700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb4dc06ac90 0x7fb4dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:22.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.985+0000 7fb4f37fe700 1 -- 172.21.15.31:0/268318610 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6400+0+0 (secure 0 0 0) 0x7fb4ec089340 con 0x7fb4fc06c3f0 2024-03-30T01:42:22.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.985+0000 7fb4fa7fc700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb4dc06ac90 0x7fb4dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:22.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.985+0000 7fb4fa7fc700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb4dc06ac90 0x7fb4dc06d150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc06c900 tx=0x7fb4e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:22.990+0000 7fb4f37fe700 1 -- 172.21.15.31:0/268318610 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb4ec0541e0 con 0x7fb4fc06c3f0 2024-03-30T01:42:23.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.203+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb4fc0747d0 con 0x7fb4fc06c3f0 2024-03-30T01:42:23.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.810+0000 7fb4f37fe700 1 -- 172.21.15.31:0/268318610 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v41) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb4ec057800 con 0x7fb4fc06c3f0 2024-03-30T01:42:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.817+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb4dc06ac90 msgr2=0x7fb4dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.817+0000 7fb5013f7700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb4dc06ac90 0x7fb4dc06d150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc06c900 tx=0x7fb4e4008040 comp rx=0 tx=0).stop 2024-03-30T01:42:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.817+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 msgr2=0x7fb4fc0af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:23.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.817+0000 7fb5013f7700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 0x7fb4fc0af2e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c550 tx=0x7fb4ec00c910 comp rx=0 tx=0).stop 2024-03-30T01:42:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.819+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 shutdown_connections 2024-03-30T01:42:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.819+0000 7fb5013f7700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc06c3f0 0x7fb4fc0af2e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.819+0000 7fb5013f7700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb4dc06ac90 0x7fb4dc06d150 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.819+0000 7fb5013f7700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4fc1546e0 0x7fb4fc0c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:23.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.819+0000 7fb5013f7700 1 --2- 172.21.15.31:0/268318610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4fc0c48b0 0x7fb4fc0ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:23.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.819+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 >> 172.21.15.31:0/268318610 conn(0x7fb4fc0abac0 msgr2=0x7fb4fc0abea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.824+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 shutdown_connections 2024-03-30T01:42:23.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.825+0000 7fb5013f7700 1 -- 172.21.15.31:0/268318610 wait complete. 2024-03-30T01:42:23.834 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs2_data' created 2024-03-30T01:42:23.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-30T01:42:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.989+0000 7fb5e7ffe700 1 Processor -- start 2024-03-30T01:42:23.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.989+0000 7fb5e7ffe700 1 -- start start 2024-03-30T01:42:23.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.990+0000 7fb5e7ffe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5e006a700 0x7fb5e006aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.990+0000 7fb5e7ffe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006b330 0x7fb5e01594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.990+0000 7fb5e7ffe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5e00c1da0 0x7fb5e00c2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.990+0000 7fb5e7ffe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e006e9a0 con 0x7fb5e006a700 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.990+0000 7fb5e7ffe700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e0159a00 con 0x7fb5e00c1da0 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.990+0000 7fb5e7ffe700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e006fb90 con 0x7fb5e006b330 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.990+0000 7fb5e5d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5e006a700 0x7fb5e006aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.990+0000 7fb5e5d9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb5e006a700 0x7fb5e006aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48950/0 (socket says 172.21.15.31:48950) 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.990+0000 7fb5e5d9a700 1 -- 172.21.15.31:0/2566734544 learned_addr learned my addr 172.21.15.31:0/2566734544 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e659b700 1 --2- 172.21.15.31:0/2566734544 >> v2:172.21.15.59:3300/0 conn(0x7fb5e00c1da0 0x7fb5e00c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e5599700 1 --2- 172.21.15.31:0/2566734544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006b330 0x7fb5e01594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e5d9a700 1 -- 172.21.15.31:0/2566734544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006b330 msgr2=0x7fb5e01594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e5d9a700 1 --2- 172.21.15.31:0/2566734544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006b330 0x7fb5e01594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:23.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e5d9a700 1 -- 172.21.15.31:0/2566734544 >> v2:172.21.15.59:3300/0 conn(0x7fb5e00c1da0 msgr2=0x7fb5e00c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e5d9a700 1 --2- 172.21.15.31:0/2566734544 >> v2:172.21.15.59:3300/0 conn(0x7fb5e00c1da0 0x7fb5e00c2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e5d9a700 1 -- 172.21.15.31:0/2566734544 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e00c2de0 con 0x7fb5e006a700 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e5d9a700 1 --2- 172.21.15.31:0/2566734544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e006a700 0x7fb5e006aae0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00c6e10 tx=0x7fb5dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd8953ce8fbb9ba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e4d98700 1 -- 172.21.15.31:0/2566734544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00b870 con 0x7fb5e006a700 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e4d98700 1 -- 172.21.15.31:0/2566734544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc00b9d0 con 0x7fb5e006a700 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.991+0000 7fb5e4d98700 1 -- 172.21.15.31:0/2566734544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00bd60 con 0x7fb5e006a700 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.992+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2566734544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e006a700 msgr2=0x7fb5e006aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.992+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2566734544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e006a700 0x7fb5e006aae0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00c6e10 tx=0x7fb5dc009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.992+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2566734544 shutdown_connections 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.992+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2566734544 >> v2:172.21.15.59:3300/0 conn(0x7fb5e00c1da0 0x7fb5e00c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.992+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2566734544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e006a700 0x7fb5e006aae0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.992+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2566734544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006b330 0x7fb5e01594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.992+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2566734544 >> 172.21.15.31:0/2566734544 conn(0x7fb5e00c3980 msgr2=0x7fb5e00c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:24.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.992+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2566734544 shutdown_connections 2024-03-30T01:42:24.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.992+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2566734544 wait complete. 2024-03-30T01:42:24.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.993+0000 7fb5e7ffe700 1 Processor -- start 2024-03-30T01:42:24.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.994+0000 7fb5e7ffe700 1 -- start start 2024-03-30T01:42:24.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.994+0000 7fb5e7ffe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006a700 0x7fb5e00c1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.994+0000 7fb5e7ffe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5e006b330 0x7fb5e00c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e7ffe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e00c1da0 0x7fb5e014ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e7ffe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e015c780 con 0x7fb5e00c1da0 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e7ffe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e015b8e0 con 0x7fb5e006b330 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e7ffe700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e014fcc0 con 0x7fb5e006a700 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e00c1da0 0x7fb5e014ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e659b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e00c1da0 0x7fb5e014ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48966/0 (socket says 172.21.15.31:48966) 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e659b700 1 -- 172.21.15.31:0/2235482863 learned_addr learned my addr 172.21.15.31:0/2235482863 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e5599700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5e006b330 0x7fb5e00c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e5d9a700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006a700 0x7fb5e00c1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e659b700 1 -- 172.21.15.31:0/2235482863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006a700 msgr2=0x7fb5e00c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e659b700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006a700 0x7fb5e00c1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e659b700 1 -- 172.21.15.31:0/2235482863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5e006b330 msgr2=0x7fb5e00c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e659b700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5e006b330 0x7fb5e00c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e659b700 1 -- 172.21.15.31:0/2235482863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc009550 con 0x7fb5e00c1da0 2024-03-30T01:42:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e659b700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e00c1da0 0x7fb5e014ba40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8007e90 tx=0x7fb5d800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5d67fc700 1 -- 172.21.15.31:0/2235482863 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800ed30 con 0x7fb5e00c1da0 2024-03-30T01:42:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5d67fc700 1 -- 172.21.15.31:0/2235482863 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d800ee90 con 0x7fb5e00c1da0 2024-03-30T01:42:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5d67fc700 1 -- 172.21.15.31:0/2235482863 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8014660 con 0x7fb5e00c1da0 2024-03-30T01:42:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.995+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e015bb30 con 0x7fb5e00c1da0 2024-03-30T01:42:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.996+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e015c130 con 0x7fb5e00c1da0 2024-03-30T01:42:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.997+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e00747d0 con 0x7fb5e00c1da0 2024-03-30T01:42:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.997+0000 7fb5d67fc700 1 -- 172.21.15.31:0/2235482863 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb5d80147c0 con 0x7fb5e00c1da0 2024-03-30T01:42:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.998+0000 7fb5d67fc700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb5c406ac90 0x7fb5c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.998+0000 7fb5d67fc700 1 -- 172.21.15.31:0/2235482863 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6783+0+0 (secure 0 0 0) 0x7fb5d8089c90 con 0x7fb5e00c1da0 2024-03-30T01:42:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.998+0000 7fb5e5d9a700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb5c406ac90 0x7fb5c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:23.998+0000 7fb5e5d9a700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb5c406ac90 0x7fb5c406d150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc000c00 tx=0x7fb5dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:24.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.003+0000 7fb5d67fc700 1 -- 172.21.15.31:0/2235482863 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb5d8054b80 con 0x7fb5e00c1da0 2024-03-30T01:42:24.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.265+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7fb5e006f600 con 0x7fb5e00c1da0 2024-03-30T01:42:24.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.272+0000 7fb5d67fc700 1 -- 172.21.15.31:0/2235482863 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 7 and data pool 8 v18) v1 ==== 175+0+0 (secure 0 0 0) 0x7fb5d80581a0 con 0x7fb5e00c1da0 2024-03-30T01:42:24.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.279+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb5c406ac90 msgr2=0x7fb5c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.279+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb5c406ac90 0x7fb5c406d150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc000c00 tx=0x7fb5dc008030 comp rx=0 tx=0).stop 2024-03-30T01:42:24.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.279+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e00c1da0 msgr2=0x7fb5e014ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.279+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e00c1da0 0x7fb5e014ba40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8007e90 tx=0x7fb5d800e3c0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.282+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 shutdown_connections 2024-03-30T01:42:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.282+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5e00c1da0 0x7fb5e014ba40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.282+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb5c406ac90 0x7fb5c406d150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.282+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5e006a700 0x7fb5e00c1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.282+0000 7fb5e7ffe700 1 --2- 172.21.15.31:0/2235482863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5e006b330 0x7fb5e00c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.282+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 >> 172.21.15.31:0/2235482863 conn(0x7fb5e00c3980 msgr2=0x7fb5e006daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.282+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 shutdown_connections 2024-03-30T01:42:24.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.283+0000 7fb5e7ffe700 1 -- 172.21.15.31:0/2235482863 wait complete. 2024-03-30T01:42:24.296 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 7 and data pool 8 2024-03-30T01:42:24.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:24.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.450+0000 7f0649976700 1 Processor -- start 2024-03-30T01:42:24.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.450+0000 7f0649976700 1 -- start start 2024-03-30T01:42:24.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f0649976700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f064406b170 0x7f0644155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f0649976700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06440bd7f0 0x7f064406b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f0649976700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 0x7f06440c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f0649976700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064406bef0 con 0x7f06440bd7f0 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f0649976700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06440c8d10 con 0x7f064406b170 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f0649976700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440c8100 con 0x7f0644154760 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f06427fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06440bd7f0 0x7f064406b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f06427fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06440bd7f0 0x7f064406b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48976/0 (socket says 172.21.15.31:48976) 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f06427fc700 1 -- 172.21.15.31:0/866647940 learned_addr learned my addr 172.21.15.31:0/866647940 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f0642ffd700 1 --2- 172.21.15.31:0/866647940 >> v2:172.21.15.59:3300/0 conn(0x7f064406b170 0x7f0644155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.451+0000 7f06437fe700 1 --2- 172.21.15.31:0/866647940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 0x7f06440c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f06427fc700 1 -- 172.21.15.31:0/866647940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 msgr2=0x7f06440c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f06427fc700 1 --2- 172.21.15.31:0/866647940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 0x7f06440c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f06427fc700 1 -- 172.21.15.31:0/866647940 >> v2:172.21.15.59:3300/0 conn(0x7f064406b170 msgr2=0x7f0644155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:24.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f06427fc700 1 --2- 172.21.15.31:0/866647940 >> v2:172.21.15.59:3300/0 conn(0x7f064406b170 0x7f0644155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f06427fc700 1 -- 172.21.15.31:0/866647940 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440c83e0 con 0x7f06440bd7f0 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f06427fc700 1 --2- 172.21.15.31:0/866647940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 0x7f064406b550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0644158e10 tx=0x7f0638009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e49c1363f478e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f0641ffb700 1 -- 172.21.15.31:0/866647940 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063800b870 con 0x7f06440bd7f0 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f0641ffb700 1 -- 172.21.15.31:0/866647940 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063800b9d0 con 0x7f06440bd7f0 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f0641ffb700 1 -- 172.21.15.31:0/866647940 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063800bd60 con 0x7f06440bd7f0 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f0649976700 1 -- 172.21.15.31:0/866647940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 msgr2=0x7f064406b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.452+0000 7f0649976700 1 --2- 172.21.15.31:0/866647940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 0x7f064406b550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0644158e10 tx=0x7f0638009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.453+0000 7f0649976700 1 -- 172.21.15.31:0/866647940 shutdown_connections 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.453+0000 7f0649976700 1 --2- 172.21.15.31:0/866647940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 0x7f06440c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.453+0000 7f0649976700 1 --2- 172.21.15.31:0/866647940 >> v2:172.21.15.59:3300/0 conn(0x7f064406b170 0x7f0644155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.453+0000 7f0649976700 1 --2- 172.21.15.31:0/866647940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 0x7f064406b550 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.453+0000 7f0649976700 1 -- 172.21.15.31:0/866647940 >> 172.21.15.31:0/866647940 conn(0x7f06440ca0d0 msgr2=0x7f06440ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.453+0000 7f0649976700 1 -- 172.21.15.31:0/866647940 shutdown_connections 2024-03-30T01:42:24.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.453+0000 7f0649976700 1 -- 172.21.15.31:0/866647940 wait complete. 2024-03-30T01:42:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.454+0000 7f0649976700 1 Processor -- start 2024-03-30T01:42:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.454+0000 7f0649976700 1 -- start start 2024-03-30T01:42:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f0649976700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f064406b170 0x7f06441ea4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f0649976700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 0x7f06440aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f0649976700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 0x7f06440ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f0649976700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06440c36f0 con 0x7f06440bd7f0 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f0649976700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441ea9e0 con 0x7f064406b170 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f0649976700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440acf00 con 0x7f0644154760 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06427fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 0x7f06440aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06427fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 0x7f06440aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48992/0 (socket says 172.21.15.31:48992) 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06427fc700 1 -- 172.21.15.31:0/2481063485 learned_addr learned my addr 172.21.15.31:0/2481063485 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f0642ffd700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f064406b170 0x7f06441ea4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06437fe700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 0x7f06440ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06427fc700 1 -- 172.21.15.31:0/2481063485 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 msgr2=0x7f06440ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06427fc700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 0x7f06440ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06427fc700 1 -- 172.21.15.31:0/2481063485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f064406b170 msgr2=0x7f06441ea4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06427fc700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f064406b170 0x7f06441ea4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06427fc700 1 -- 172.21.15.31:0/2481063485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638009550 con 0x7f06440bd7f0 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f0642ffd700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f064406b170 0x7f06441ea4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.455+0000 7f06427fc700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 0x7f06440aad90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0638013000 tx=0x7f0638004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:24.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.456+0000 7f062bfff700 1 -- 172.21.15.31:0/2481063485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638004310 con 0x7f06440bd7f0 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.456+0000 7f062bfff700 1 -- 172.21.15.31:0/2481063485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063801f070 con 0x7f06440bd7f0 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.456+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06440ae220 con 0x7f06440bd7f0 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.456+0000 7f062bfff700 1 -- 172.21.15.31:0/2481063485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380036a0 con 0x7f06440bd7f0 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.456+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06440ae7e0 con 0x7f06440bd7f0 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.457+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06440ad0b0 con 0x7f06440bd7f0 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.457+0000 7f062bfff700 1 -- 172.21.15.31:0/2481063485 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0638021030 con 0x7f06440bd7f0 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.458+0000 7f062bfff700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f063006ac90 0x7f063006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.458+0000 7f062bfff700 1 -- 172.21.15.31:0/2481063485 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f063808a890 con 0x7f06440bd7f0 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.458+0000 7f0642ffd700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f063006ac90 0x7f063006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.458+0000 7f0642ffd700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f063006ac90 0x7f063006d150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f06340099e0 tx=0x7f0634008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:24.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.463+0000 7f062bfff700 1 -- 172.21.15.31:0/2481063485 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f06440ad0b0 con 0x7f06440bd7f0 2024-03-30T01:42:24.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.679+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06440ad070 con 0x7f06440bd7f0 2024-03-30T01:42:24.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.680+0000 7f062bfff700 1 -- 172.21.15.31:0/2481063485 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+19987 (secure 0 0 0) 0x7f0638015090 con 0x7f06440bd7f0 2024-03-30T01:42:24.687 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:24.687 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":42,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:24.267155+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f063006ac90 msgr2=0x7f063006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f063006ac90 0x7f063006d150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f06340099e0 tx=0x7f0634008040 comp rx=0 tx=0).stop 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 msgr2=0x7f06440aad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 0x7f06440aad90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0638013000 tx=0x7f0638004750 comp rx=0 tx=0).stop 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 shutdown_connections 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f063006ac90 0x7f063006d150 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0644154760 0x7f06440ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f064406b170 0x7f06441ea4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 --2- 172.21.15.31:0/2481063485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06440bd7f0 0x7f06440aad90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 >> 172.21.15.31:0/2481063485 conn(0x7f06440ca0d0 msgr2=0x7f06440ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 shutdown_connections 2024-03-30T01:42:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.683+0000 7f0649976700 1 -- 172.21.15.31:0/2481063485 wait complete. 2024-03-30T01:42:24.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:24.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.834+0000 7fa20a6d8700 1 Processor -- start 2024-03-30T01:42:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.834+0000 7fa20a6d8700 1 -- start start 2024-03-30T01:42:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa20a6d8700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa2040a9560 0x7fa20406a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa20a6d8700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20406e4a0 con 0x7fa20414fbf0 2024-03-30T01:42:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa20a6d8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa204155bf0 con 0x7fa20406d9e0 2024-03-30T01:42:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa20a6d8700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa204156ac0 con 0x7fa2040a9560 2024-03-30T01:42:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa208c75700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa2040a9560 0x7fa20406a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa203fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa20414fbf0 0x7fa2041563b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46008/0 (socket says 172.21.15.31:46008) 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa203fff700 1 -- 172.21.15.31:0/2978578556 learned_addr learned my addr 172.21.15.31:0/2978578556 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa208c75700 1 -- 172.21.15.31:0/2978578556 >> v1:172.21.15.59:6789/0 conn(0x7fa20406d9e0 legacy=0x7fa2040ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa208c75700 1 -- 172.21.15.31:0/2978578556 >> v1:172.21.15.31:6789/0 conn(0x7fa20414fbf0 legacy=0x7fa2041563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa208c75700 1 -- 172.21.15.31:0/2978578556 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20406dee0 con 0x7fa2040a9560 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.835+0000 7fa208c75700 1 --2- 172.21.15.31:0/2978578556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2040a9560 0x7fa20406a700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009d90 tx=0x7fa1fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644c6bd419465668 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa202ffd700 1 -- 172.21.15.31:0/2978578556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00e070 con 0x7fa2040a9560 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa202ffd700 1 -- 172.21.15.31:0/2978578556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc00bbd0 con 0x7fa2040a9560 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa202ffd700 1 -- 172.21.15.31:0/2978578556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc012430 con 0x7fa2040a9560 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa20a6d8700 1 -- 172.21.15.31:0/2978578556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2040a9560 msgr2=0x7fa20406a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa20a6d8700 1 --2- 172.21.15.31:0/2978578556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2040a9560 0x7fa20406a700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009d90 tx=0x7fa1fc00b330 comp rx=0 tx=0).stop 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa20a6d8700 1 -- 172.21.15.31:0/2978578556 shutdown_connections 2024-03-30T01:42:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa20a6d8700 1 --2- 172.21.15.31:0/2978578556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2040a9560 0x7fa20406a700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa20a6d8700 1 -- 172.21.15.31:0/2978578556 >> 172.21.15.31:0/2978578556 conn(0x7fa2040c3190 msgr2=0x7fa2040c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:24.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa20a6d8700 1 -- 172.21.15.31:0/2978578556 shutdown_connections 2024-03-30T01:42:24.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.836+0000 7fa20a6d8700 1 -- 172.21.15.31:0/2978578556 wait complete. 2024-03-30T01:42:24.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.837+0000 7fa20a6d8700 1 Processor -- start 2024-03-30T01:42:24.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.838+0000 7fa20a6d8700 1 -- start start 2024-03-30T01:42:24.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.838+0000 7fa20a6d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa20406d9e0 0x7fa204071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.838+0000 7fa20a6d8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2040a9560 0x7fa2040bb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.838+0000 7fa20a6d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa20414fbf0 0x7fa2040c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.838+0000 7fa20a6d8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa204154800 con 0x7fa20414fbf0 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.838+0000 7fa20a6d8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2041eaac0 con 0x7fa20406d9e0 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.838+0000 7fa20a6d8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2041eada0 con 0x7fa2040a9560 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa208c75700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa20414fbf0 0x7fa2040c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa208c75700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa20414fbf0 0x7fa2040c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49000/0 (socket says 172.21.15.31:49000) 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa208c75700 1 -- 172.21.15.31:0/430268903 learned_addr learned my addr 172.21.15.31:0/430268903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa203fff700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa20406d9e0 0x7fa204071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa2037fe700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2040a9560 0x7fa2040bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa208c75700 1 -- 172.21.15.31:0/430268903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2040a9560 msgr2=0x7fa2040bb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa208c75700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2040a9560 0x7fa2040bb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa208c75700 1 -- 172.21.15.31:0/430268903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa20406d9e0 msgr2=0x7fa204071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa208c75700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa20406d9e0 0x7fa204071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa208c75700 1 -- 172.21.15.31:0/430268903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc009a70 con 0x7fa20414fbf0 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa2037fe700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2040a9560 0x7fa2040bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa208c75700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa20414fbf0 0x7fa2040c6a80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc014040 tx=0x7fa1fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa200ff9700 1 -- 172.21.15.31:0/430268903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0098f0 con 0x7fa20414fbf0 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa200ff9700 1 -- 172.21.15.31:0/430268903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc020070 con 0x7fa20414fbf0 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa200ff9700 1 -- 172.21.15.31:0/430268903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc004280 con 0x7fa20414fbf0 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa204070bc0 con 0x7fa20414fbf0 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.839+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2040c6fc0 con 0x7fa20414fbf0 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.841+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20404ea80 con 0x7fa20414fbf0 2024-03-30T01:42:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.841+0000 7fa200ff9700 1 -- 172.21.15.31:0/430268903 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa1fc0043e0 con 0x7fa20414fbf0 2024-03-30T01:42:24.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.841+0000 7fa200ff9700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa1e406ac90 0x7fa1e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:24.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.842+0000 7fa200ff9700 1 -- 172.21.15.31:0/430268903 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fa1fc022070 con 0x7fa20414fbf0 2024-03-30T01:42:24.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.842+0000 7fa203fff700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa1e406ac90 0x7fa1e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:24.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.842+0000 7fa203fff700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa1e406ac90 0x7fa1e406d150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec005910 tx=0x7fa1ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:24.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:24.846+0000 7fa200ff9700 1 -- 172.21.15.31:0/430268903 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa1fc0563a0 con 0x7fa20414fbf0 2024-03-30T01:42:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.064+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2040a8890 con 0x7fa20414fbf0 2024-03-30T01:42:25.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.065+0000 7fa200ff9700 1 -- 172.21.15.31:0/430268903 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+19987 (secure 0 0 0) 0x7fa1fc0599c0 con 0x7fa20414fbf0 2024-03-30T01:42:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":42,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:24.267155+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:25.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.067+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa1e406ac90 msgr2=0x7fa1e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:25.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.067+0000 7fa20a6d8700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa1e406ac90 0x7fa1e406d150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec005910 tx=0x7fa1ec009040 comp rx=0 tx=0).stop 2024-03-30T01:42:25.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.067+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa20414fbf0 msgr2=0x7fa2040c6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:25.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.067+0000 7fa20a6d8700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa20414fbf0 0x7fa2040c6a80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc014040 tx=0x7fa1fc004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.068+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 shutdown_connections 2024-03-30T01:42:25.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.068+0000 7fa20a6d8700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa20414fbf0 0x7fa2040c6a80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.068+0000 7fa20a6d8700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa1e406ac90 0x7fa1e406d150 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.068+0000 7fa20a6d8700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa20406d9e0 0x7fa204071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.068+0000 7fa20a6d8700 1 --2- 172.21.15.31:0/430268903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2040a9560 0x7fa2040bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.068+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 >> 172.21.15.31:0/430268903 conn(0x7fa2040c3190 msgr2=0x7fa20406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:25.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.068+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 shutdown_connections 2024-03-30T01:42:25.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.068+0000 7fa20a6d8700 1 -- 172.21.15.31:0/430268903 wait complete. 2024-03-30T01:42:25.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-30T01:42:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.221+0000 7fc83430b700 1 Processor -- start 2024-03-30T01:42:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.221+0000 7fc83430b700 1 -- start start 2024-03-30T01:42:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc83430b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc82c14fce0 0x7fc82c0c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc83430b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc82c0a91c0 0x7fc82c06f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc83430b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82c156930 con 0x7fc82c1525e0 2024-03-30T01:42:25.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc83430b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82c0aa330 con 0x7fc82c0a91c0 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc83430b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82c156180 con 0x7fc82c14fce0 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8328a8700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc82c0a91c0 0x7fc82c06f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8318a6700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc82c1525e0 0x7fc82c155a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46014/0 (socket says 172.21.15.31:46014) 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8318a6700 1 -- 172.21.15.31:0/3945555106 learned_addr learned my addr 172.21.15.31:0/3945555106 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8320a7700 1 --2- 172.21.15.31:0/3945555106 >> v2:172.21.15.59:3300/0 conn(0x7fc82c14fce0 0x7fc82c0c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8310a5700 1 -- 172.21.15.31:0/3945555106 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185357593 0 0) 0x7fc82c156930 con 0x7fc82c1525e0 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8310a5700 1 -- 172.21.15.31:0/3945555106 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8180035b0 con 0x7fc82c1525e0 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8328a8700 1 -- 172.21.15.31:0/3945555106 >> v2:172.21.15.59:3300/0 conn(0x7fc82c14fce0 msgr2=0x7fc82c0c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8328a8700 1 --2- 172.21.15.31:0/3945555106 >> v2:172.21.15.59:3300/0 conn(0x7fc82c14fce0 0x7fc82c0c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8328a8700 1 -- 172.21.15.31:0/3945555106 >> v1:172.21.15.31:6789/0 conn(0x7fc82c1525e0 legacy=0x7fc82c155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.222+0000 7fc8328a8700 1 -- 172.21.15.31:0/3945555106 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c156370 con 0x7fc82c0a91c0 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc8328a8700 1 --2- 172.21.15.31:0/3945555106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 0x7fc82c06f330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0aa5a0 tx=0x7fc82400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325f80c5ee814234 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:25.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc8310a5700 1 -- 172.21.15.31:0/3945555106 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82400e070 con 0x7fc82c0a91c0 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc8310a5700 1 -- 172.21.15.31:0/3945555106 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82400bb20 con 0x7fc82c0a91c0 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc8310a5700 1 -- 172.21.15.31:0/3945555106 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824012430 con 0x7fc82c0a91c0 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc83430b700 1 -- 172.21.15.31:0/3945555106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 msgr2=0x7fc82c06f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc83430b700 1 --2- 172.21.15.31:0/3945555106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 0x7fc82c06f330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0aa5a0 tx=0x7fc82400b330 comp rx=0 tx=0).stop 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc83430b700 1 -- 172.21.15.31:0/3945555106 shutdown_connections 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc83430b700 1 --2- 172.21.15.31:0/3945555106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 0x7fc82c06f330 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc83430b700 1 --2- 172.21.15.31:0/3945555106 >> v2:172.21.15.59:3300/0 conn(0x7fc82c14fce0 0x7fc82c0c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc83430b700 1 -- 172.21.15.31:0/3945555106 >> 172.21.15.31:0/3945555106 conn(0x7fc82c0c3910 msgr2=0x7fc82c0c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc83430b700 1 -- 172.21.15.31:0/3945555106 shutdown_connections 2024-03-30T01:42:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.223+0000 7fc83430b700 1 -- 172.21.15.31:0/3945555106 wait complete. 2024-03-30T01:42:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.225+0000 7fc83430b700 1 Processor -- start 2024-03-30T01:42:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.225+0000 7fc83430b700 1 -- start start 2024-03-30T01:42:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc83430b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 0x7fc82c0ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc83430b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc82c14fce0 0x7fc82c151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc83430b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc82c1525e0 0x7fc82c1eb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc83430b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c0c4e40 con 0x7fc82c0a91c0 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc83430b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c0c6810 con 0x7fc82c1525e0 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc83430b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82c0c5860 con 0x7fc82c14fce0 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc8320a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 0x7fc82c0ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc8320a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 0x7fc82c0ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49018/0 (socket says 172.21.15.31:49018) 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc8320a7700 1 -- 172.21.15.31:0/1660667679 learned_addr learned my addr 172.21.15.31:0/1660667679 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc8318a6700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc82c14fce0 0x7fc82c151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.226+0000 7fc8328a8700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc82c1525e0 0x7fc82c1eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc8320a7700 1 -- 172.21.15.31:0/1660667679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc82c14fce0 msgr2=0x7fc82c151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc8320a7700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc82c14fce0 0x7fc82c151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc8320a7700 1 -- 172.21.15.31:0/1660667679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc82c1525e0 msgr2=0x7fc82c1eb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc8320a7700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc82c1525e0 0x7fc82c1eb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc8320a7700 1 -- 172.21.15.31:0/1660667679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc824009a70 con 0x7fc82c0a91c0 2024-03-30T01:42:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc8320a7700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 0x7fc82c0ac460 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc82800e810 tx=0x7fc82800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc822ffd700 1 -- 172.21.15.31:0/1660667679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800c070 con 0x7fc82c0a91c0 2024-03-30T01:42:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc822ffd700 1 -- 172.21.15.31:0/1660667679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc828013610 con 0x7fc82c0a91c0 2024-03-30T01:42:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82c0c4aa0 con 0x7fc82c0a91c0 2024-03-30T01:42:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc822ffd700 1 -- 172.21.15.31:0/1660667679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8280139c0 con 0x7fc82c0a91c0 2024-03-30T01:42:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.227+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82c0c5d50 con 0x7fc82c0a91c0 2024-03-30T01:42:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.228+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82c066f90 con 0x7fc82c0a91c0 2024-03-30T01:42:25.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.229+0000 7fc822ffd700 1 -- 172.21.15.31:0/1660667679 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc828013b20 con 0x7fc82c0a91c0 2024-03-30T01:42:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.229+0000 7fc822ffd700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc81006ac90 0x7fc81006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.229+0000 7fc822ffd700 1 -- 172.21.15.31:0/1660667679 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fc828089500 con 0x7fc82c0a91c0 2024-03-30T01:42:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.229+0000 7fc8318a6700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc81006ac90 0x7fc81006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.230+0000 7fc8318a6700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc81006ac90 0x7fc81006d150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0c13f0 tx=0x7fc81c007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:25.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.234+0000 7fc822ffd700 1 -- 172.21.15.31:0/1660667679 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc828054250 con 0x7fc82c0a91c0 2024-03-30T01:42:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:25.502+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc82c0c4f80 con 0x7fc82c0a91c0 2024-03-30T01:42:26.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.274+0000 7fc822ffd700 1 -- 172.21.15.31:0/1660667679 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v20) v1 ==== 119+0+0 (secure 0 0 0) 0x7fc82c0c4f80 con 0x7fc82c0a91c0 2024-03-30T01:42:26.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.277+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc81006ac90 msgr2=0x7fc81006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:26.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.277+0000 7fc83430b700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc81006ac90 0x7fc81006d150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0c13f0 tx=0x7fc81c007dc0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.277+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 msgr2=0x7fc82c0ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:26.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.277+0000 7fc83430b700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 0x7fc82c0ac460 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc82800e810 tx=0x7fc82800eb20 comp rx=0 tx=0).stop 2024-03-30T01:42:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.278+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 shutdown_connections 2024-03-30T01:42:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.278+0000 7fc83430b700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82c0a91c0 0x7fc82c0ac460 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.278+0000 7fc83430b700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc81006ac90 0x7fc81006d150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.278+0000 7fc83430b700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc82c1525e0 0x7fc82c1eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.278+0000 7fc83430b700 1 --2- 172.21.15.31:0/1660667679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc82c14fce0 0x7fc82c151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.278+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 >> 172.21.15.31:0/1660667679 conn(0x7fc82c0c3910 msgr2=0x7fc82c071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.278+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 shutdown_connections 2024-03-30T01:42:26.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.278+0000 7fc83430b700 1 -- 172.21.15.31:0/1660667679 wait complete. 2024-03-30T01:42:26.302 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T01:42:26.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-30T01:42:26.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.441+0000 7f8385186700 1 Processor -- start 2024-03-30T01:42:26.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.441+0000 7f8385186700 1 -- start start 2024-03-30T01:42:26.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f8385186700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f838014fce0 0x7f83800c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f8385186700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83801525e0 0x7f8380155a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f8385186700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83800a91c0 0x7f838006f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f8385186700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83800aa330 con 0x7f838014fce0 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f8385186700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8380156930 con 0x7f83801525e0 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f8385186700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8380156370 con 0x7f83800a91c0 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f838014fce0 0x7f83800c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837ed9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f838014fce0 0x7f83800c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44598/0 (socket says 172.21.15.31:44598) 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837ed9d700 1 -- 172.21.15.31:0/3089814781 learned_addr learned my addr 172.21.15.31:0/3089814781 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837f59e700 1 --2- 172.21.15.31:0/3089814781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83800a91c0 0x7f838006f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837e59c700 1 --2- 172.21.15.31:0/3089814781 >> v2:172.21.15.59:3300/0 conn(0x7f83801525e0 0x7f8380155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837ed9d700 1 -- 172.21.15.31:0/3089814781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83800a91c0 msgr2=0x7f838006f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837ed9d700 1 --2- 172.21.15.31:0/3089814781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83800a91c0 0x7f838006f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837ed9d700 1 -- 172.21.15.31:0/3089814781 >> v2:172.21.15.59:3300/0 conn(0x7f83801525e0 msgr2=0x7f8380155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837ed9d700 1 --2- 172.21.15.31:0/3089814781 >> v2:172.21.15.59:3300/0 conn(0x7f83801525e0 0x7f8380155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.442+0000 7f837ed9d700 1 -- 172.21.15.31:0/3089814781 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8380156650 con 0x7f838014fce0 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f837ed9d700 1 --2- 172.21.15.31:0/3089814781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f838014fce0 0x7f83800c1f80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f83800c1c90 tx=0x7f8370009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c389663bd5e280 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f837dd9b700 1 -- 172.21.15.31:0/3089814781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000b870 con 0x7f838014fce0 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f837dd9b700 1 -- 172.21.15.31:0/3089814781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837000b9d0 con 0x7f838014fce0 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f837dd9b700 1 -- 172.21.15.31:0/3089814781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000bd60 con 0x7f838014fce0 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f8385186700 1 -- 172.21.15.31:0/3089814781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f838014fce0 msgr2=0x7f83800c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f8385186700 1 --2- 172.21.15.31:0/3089814781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f838014fce0 0x7f83800c1f80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f83800c1c90 tx=0x7f8370009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f8385186700 1 -- 172.21.15.31:0/3089814781 shutdown_connections 2024-03-30T01:42:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f8385186700 1 --2- 172.21.15.31:0/3089814781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83800a91c0 0x7f838006f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f8385186700 1 --2- 172.21.15.31:0/3089814781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f838014fce0 0x7f83800c1f80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f8385186700 1 --2- 172.21.15.31:0/3089814781 >> v2:172.21.15.59:3300/0 conn(0x7f83801525e0 0x7f8380155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f8385186700 1 -- 172.21.15.31:0/3089814781 >> 172.21.15.31:0/3089814781 conn(0x7f83800c3910 msgr2=0x7f83800c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f8385186700 1 -- 172.21.15.31:0/3089814781 shutdown_connections 2024-03-30T01:42:26.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.443+0000 7f8385186700 1 -- 172.21.15.31:0/3089814781 wait complete. 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.445+0000 7f8385186700 1 Processor -- start 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.445+0000 7f8385186700 1 -- start start 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f8385186700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83800a91c0 0x7f83801ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f8385186700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f838014fce0 0x7f838014abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f8385186700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83801525e0 0x7f83801ea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f8385186700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838014bb60 con 0x7f83800a91c0 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f8385186700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83801ed7d0 con 0x7f83801525e0 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f8385186700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83801edab0 con 0x7f838014fce0 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83800a91c0 0x7f83801ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83800a91c0 0x7f83801ed1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44606/0 (socket says 172.21.15.31:44606) 2024-03-30T01:42:26.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837ed9d700 1 -- 172.21.15.31:0/2358065349 learned_addr learned my addr 172.21.15.31:0/2358065349 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837f59e700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83801525e0 0x7f83801ea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837ed9d700 1 -- 172.21.15.31:0/2358065349 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f838014fce0 msgr2=0x7f838014abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837e59c700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f838014fce0 0x7f838014abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837ed9d700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f838014fce0 0x7f838014abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837ed9d700 1 -- 172.21.15.31:0/2358065349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83801525e0 msgr2=0x7f83801ea2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837ed9d700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83801525e0 0x7f83801ea2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837ed9d700 1 -- 172.21.15.31:0/2358065349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8370009550 con 0x7f83800a91c0 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837f59e700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83801525e0 0x7f83801ea2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.446+0000 7f837ed9d700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83800a91c0 0x7f83801ed1f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8370000c00 tx=0x7f8370004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.447+0000 7f83777fe700 1 -- 172.21.15.31:0/2358065349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370004540 con 0x7f83800a91c0 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.447+0000 7f83777fe700 1 -- 172.21.15.31:0/2358065349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83700036a0 con 0x7f83800a91c0 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.447+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83801edd90 con 0x7f83800a91c0 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.447+0000 7f83777fe700 1 -- 172.21.15.31:0/2358065349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370003a50 con 0x7f83800a91c0 2024-03-30T01:42:26.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.447+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83801e9d90 con 0x7f83800a91c0 2024-03-30T01:42:26.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.448+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8380066f90 con 0x7f83800a91c0 2024-03-30T01:42:26.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.448+0000 7f83777fe700 1 -- 172.21.15.31:0/2358065349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8370021070 con 0x7f83800a91c0 2024-03-30T01:42:26.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.449+0000 7f83777fe700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f836006ac90 0x7f836006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:26.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.449+0000 7f83777fe700 1 -- 172.21.15.31:0/2358065349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f837008ade0 con 0x7f83800a91c0 2024-03-30T01:42:26.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.449+0000 7f837e59c700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f836006ac90 0x7f836006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:26.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.449+0000 7f837e59c700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f836006ac90 0x7f836006d150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83800c2490 tx=0x7f8368008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:26.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.454+0000 7f83777fe700 1 -- 172.21.15.31:0/2358065349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8370055bb0 con 0x7f83800a91c0 2024-03-30T01:42:26.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:26.715+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f83801e9f80 con 0x7f83800a91c0 2024-03-30T01:42:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.276+0000 7f83777fe700 1 -- 172.21.15.31:0/2358065349 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v21) v1 ==== 123+0+0 (secure 0 0 0) 0x7f83700290c0 con 0x7f83800a91c0 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.278+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f836006ac90 msgr2=0x7f836006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.278+0000 7f8385186700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f836006ac90 0x7f836006d150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83800c2490 tx=0x7f8368008040 comp rx=0 tx=0).stop 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.278+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83800a91c0 msgr2=0x7f83801ed1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.278+0000 7f8385186700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83800a91c0 0x7f83801ed1f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8370000c00 tx=0x7f8370004750 comp rx=0 tx=0).stop 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.279+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 shutdown_connections 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.279+0000 7f8385186700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83800a91c0 0x7f83801ed1f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.279+0000 7f8385186700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f836006ac90 0x7f836006d150 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.279+0000 7f8385186700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83801525e0 0x7f83801ea2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.279+0000 7f8385186700 1 --2- 172.21.15.31:0/2358065349 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f838014fce0 0x7f838014abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.279+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 >> 172.21.15.31:0/2358065349 conn(0x7f83800c3910 msgr2=0x7f838006f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.279+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 shutdown_connections 2024-03-30T01:42:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.279+0000 7f8385186700 1 -- 172.21.15.31:0/2358065349 wait complete. 2024-03-30T01:42:27.301 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-30T01:42:27.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.436+0000 7f9fec3ce700 1 Processor -- start 2024-03-30T01:42:27.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.436+0000 7f9fec3ce700 1 -- start start 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.437+0000 7f9fec3ce700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c5df0 0x7f9fe40c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.437+0000 7f9fec3ce700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe41556d0 con 0x7f9fe40c2870 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.437+0000 7f9fec3ce700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe40c1f70 con 0x7f9fe40c8310 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.437+0000 7f9fec3ce700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe41559b0 con 0x7f9fe40c5df0 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.437+0000 7f9fea96b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9fe40c2870 0x7f9fe40a6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47352/0 (socket says 172.21.15.31:47352) 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.437+0000 7f9fea96b700 1 -- 172.21.15.31:0/232818463 learned_addr learned my addr 172.21.15.31:0/232818463 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.437+0000 7f9fe9969700 1 --2- 172.21.15.31:0/232818463 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c5df0 0x7f9fe40c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9168700 1 -- 172.21.15.31:0/232818463 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633407840 0 0) 0x7f9fe41556d0 con 0x7f9fe40c2870 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9168700 1 -- 172.21.15.31:0/232818463 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd00035b0 con 0x7f9fe40c2870 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9168700 1 -- 172.21.15.31:0/232818463 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594091344 0 0) 0x7f9fe40c1f70 con 0x7f9fe40c8310 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9168700 1 -- 172.21.15.31:0/232818463 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fe41556d0 con 0x7f9fe40c8310 2024-03-30T01:42:27.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9969700 1 -- 172.21.15.31:0/232818463 >> v1:172.21.15.59:6789/0 conn(0x7f9fe40c8310 legacy=0x7f9fe4151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9969700 1 -- 172.21.15.31:0/232818463 >> v1:172.21.15.31:6789/0 conn(0x7f9fe40c2870 legacy=0x7f9fe40a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9969700 1 -- 172.21.15.31:0/232818463 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40c2250 con 0x7f9fe40c5df0 2024-03-30T01:42:27.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9969700 1 --2- 172.21.15.31:0/232818463 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c5df0 0x7f9fe40c86f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009870 tx=0x7f9fd4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70f17e9d399aac7a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:27.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9168700 1 -- 172.21.15.31:0/232818463 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4011830 con 0x7f9fe40c5df0 2024-03-30T01:42:27.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9168700 1 -- 172.21.15.31:0/232818463 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd4011990 con 0x7f9fe40c5df0 2024-03-30T01:42:27.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.438+0000 7f9fe9168700 1 -- 172.21.15.31:0/232818463 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4011d20 con 0x7f9fe40c5df0 2024-03-30T01:42:27.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.439+0000 7f9fec3ce700 1 -- 172.21.15.31:0/232818463 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c5df0 msgr2=0x7f9fe40c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.439+0000 7f9fec3ce700 1 --2- 172.21.15.31:0/232818463 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c5df0 0x7f9fe40c86f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009870 tx=0x7f9fd4009b80 comp rx=0 tx=0).stop 2024-03-30T01:42:27.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.439+0000 7f9fec3ce700 1 -- 172.21.15.31:0/232818463 shutdown_connections 2024-03-30T01:42:27.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.439+0000 7f9fec3ce700 1 --2- 172.21.15.31:0/232818463 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c5df0 0x7f9fe40c86f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.439+0000 7f9fec3ce700 1 -- 172.21.15.31:0/232818463 >> 172.21.15.31:0/232818463 conn(0x7f9fe406ab20 msgr2=0x7f9fe406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:27.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.439+0000 7f9fec3ce700 1 -- 172.21.15.31:0/232818463 shutdown_connections 2024-03-30T01:42:27.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.439+0000 7f9fec3ce700 1 -- 172.21.15.31:0/232818463 wait complete. 2024-03-30T01:42:27.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.440+0000 7f9fec3ce700 1 Processor -- start 2024-03-30T01:42:27.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.440+0000 7f9fec3ce700 1 -- start start 2024-03-30T01:42:27.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fec3ce700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c2870 0x7f9fe4155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fec3ce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe40c5df0 0x7f9fe41564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fec3ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe40c8310 0x7f9fe40a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fec3ce700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40ab790 con 0x7f9fe40c8310 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fec3ce700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4156fd0 con 0x7f9fe40c5df0 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fec3ce700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe41569f0 con 0x7f9fe40c2870 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fea96b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe40c8310 0x7f9fe40a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fea96b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe40c8310 0x7f9fe40a6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44616/0 (socket says 172.21.15.31:44616) 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fea96b700 1 -- 172.21.15.31:0/2131296133 learned_addr learned my addr 172.21.15.31:0/2131296133 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fe9969700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe40c5df0 0x7f9fe41564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fea16a700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c2870 0x7f9fe4155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.441+0000 7f9fea96b700 1 -- 172.21.15.31:0/2131296133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c2870 msgr2=0x7f9fe4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fea96b700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c2870 0x7f9fe4155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fea96b700 1 -- 172.21.15.31:0/2131296133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe40c5df0 msgr2=0x7f9fe41564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fea96b700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe40c5df0 0x7f9fe41564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fea96b700 1 -- 172.21.15.31:0/2131296133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd40094a0 con 0x7f9fe40c8310 2024-03-30T01:42:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fea96b700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe40c8310 0x7f9fe40a6150 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00cb70 tx=0x7f9fdc00ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fdaffd700 1 -- 172.21.15.31:0/2131296133 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00aa60 con 0x7f9fe40c8310 2024-03-30T01:42:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe41e9750 con 0x7f9fe40c8310 2024-03-30T01:42:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fdaffd700 1 -- 172.21.15.31:0/2131296133 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc00abc0 con 0x7f9fe40c8310 2024-03-30T01:42:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fdaffd700 1 -- 172.21.15.31:0/2131296133 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc012400 con 0x7f9fe40c8310 2024-03-30T01:42:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.442+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe40aae80 con 0x7f9fe40c8310 2024-03-30T01:42:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.443+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe4073760 con 0x7f9fe40c8310 2024-03-30T01:42:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.443+0000 7f9fdaffd700 1 -- 172.21.15.31:0/2131296133 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9fdc01b3b0 con 0x7f9fe40c8310 2024-03-30T01:42:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.444+0000 7f9fdaffd700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9fc806ac90 0x7f9fc806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.444+0000 7f9fdaffd700 1 -- 172.21.15.31:0/2131296133 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f9fdc089350 con 0x7f9fe40c8310 2024-03-30T01:42:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.444+0000 7f9fea16a700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9fc806ac90 0x7f9fc806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.444+0000 7f9fea16a700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9fc806ac90 0x7f9fc806d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00011e0 tx=0x7f9fe0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.449+0000 7f9fdaffd700 1 -- 172.21.15.31:0/2131296133 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9fdc054150 con 0x7f9fe40c8310 2024-03-30T01:42:27.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.712+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fe40ab8d0 con 0x7f9fe40c8310 2024-03-30T01:42:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.712+0000 7f9fdaffd700 1 -- 172.21.15.31:0/2131296133 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5942 (secure 0 0 0) 0x7f9fe40ab8d0 con 0x7f9fe40c8310 2024-03-30T01:42:27.719 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:27.719 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:10.264695+0000","modified":"2024-03-30T01:42:13.278071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/1616247357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1616247357},{"type":"v1","addr":"172.21.15.31:6837","nonce":1616247357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:24.267501+0000","modified":"2024-03-30T01:42:27.273615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T01:42:27.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9fc806ac90 msgr2=0x7f9fc806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9fc806ac90 0x7f9fc806d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00011e0 tx=0x7f9fe0009040 comp rx=0 tx=0).stop 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe40c8310 msgr2=0x7f9fe40a6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe40c8310 0x7f9fe40a6150 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00cb70 tx=0x7f9fdc00ce80 comp rx=0 tx=0).stop 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 shutdown_connections 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fe40c8310 0x7f9fe40a6150 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9fc806ac90 0x7f9fc806d150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fe40c2870 0x7f9fe4155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 --2- 172.21.15.31:0/2131296133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe40c5df0 0x7f9fe41564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 >> 172.21.15.31:0/2131296133 conn(0x7f9fe406ab20 msgr2=0x7f9fe40a82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 shutdown_connections 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.715+0000 7f9fec3ce700 1 -- 172.21.15.31:0/2131296133 wait complete. 2024-03-30T01:42:27.722 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-03-30T01:42:27.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:27.792 INFO:tasks.ceph.mds.a.smithi031.stderr:2024-03-30T01:42:27.785+0000 7fe21e364700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T01:42:27.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.866+0000 7fce31168700 1 Processor -- start 2024-03-30T01:42:27.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.866+0000 7fce31168700 1 -- start start 2024-03-30T01:42:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce31168700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce2c0c10c0 0x7fce2c0c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce31168700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c1524e0 con 0x7fce2c06dc50 2024-03-30T01:42:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce31168700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c158ca0 con 0x7fce2c06a700 2024-03-30T01:42:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce31168700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c0c14d0 con 0x7fce2c0c10c0 2024-03-30T01:42:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce2b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fce2c0c10c0 0x7fce2c0c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce2ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fce2c06dc50 0x7fce2c154820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47366/0 (socket says 172.21.15.31:47366) 2024-03-30T01:42:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce2ad9d700 1 -- 172.21.15.31:0/3890833451 learned_addr learned my addr 172.21.15.31:0/3890833451 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce29d9b700 1 -- 172.21.15.31:0/3890833451 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651820086 0 0) 0x7fce2c1524e0 con 0x7fce2c06dc50 2024-03-30T01:42:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce2b59e700 1 -- 172.21.15.31:0/3890833451 >> v1:172.21.15.59:6789/0 conn(0x7fce2c06a700 legacy=0x7fce2c158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce2b59e700 1 -- 172.21.15.31:0/3890833451 >> v1:172.21.15.31:6789/0 conn(0x7fce2c06dc50 legacy=0x7fce2c154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce2b59e700 1 -- 172.21.15.31:0/3890833451 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c1550e0 con 0x7fce2c0c10c0 2024-03-30T01:42:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.867+0000 7fce2b59e700 1 --2- 172.21.15.31:0/3890833451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 0x7fce2c0c1b10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fce24009d90 tx=0x7fce2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c641e35806651b4a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce29d9b700 1 -- 172.21.15.31:0/3890833451 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2400e070 con 0x7fce2c0c10c0 2024-03-30T01:42:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce29d9b700 1 -- 172.21.15.31:0/3890833451 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2400bb90 con 0x7fce2c0c10c0 2024-03-30T01:42:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce29d9b700 1 -- 172.21.15.31:0/3890833451 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24003a20 con 0x7fce2c0c10c0 2024-03-30T01:42:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce31168700 1 -- 172.21.15.31:0/3890833451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 msgr2=0x7fce2c0c1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce31168700 1 --2- 172.21.15.31:0/3890833451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 0x7fce2c0c1b10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fce24009d90 tx=0x7fce2400b330 comp rx=0 tx=0).stop 2024-03-30T01:42:27.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce31168700 1 -- 172.21.15.31:0/3890833451 shutdown_connections 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce31168700 1 --2- 172.21.15.31:0/3890833451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 0x7fce2c0c1b10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce31168700 1 -- 172.21.15.31:0/3890833451 >> 172.21.15.31:0/3890833451 conn(0x7fce2c0c3190 msgr2=0x7fce2c0c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce31168700 1 -- 172.21.15.31:0/3890833451 shutdown_connections 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.868+0000 7fce31168700 1 -- 172.21.15.31:0/3890833451 wait complete. 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.870+0000 7fce31168700 1 Processor -- start 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.870+0000 7fce31168700 1 -- start start 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce31168700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce2c06a700 0x7fce2c150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce31168700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce2c06dc50 0x7fce2c14d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce31168700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 0x7fce2c0a8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce31168700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0ab560 con 0x7fce2c0c10c0 2024-03-30T01:42:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce31168700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0a9600 con 0x7fce2c06dc50 2024-03-30T01:42:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce31168700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c14dcc0 con 0x7fce2c06a700 2024-03-30T01:42:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 0x7fce2c0a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2b59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 0x7fce2c0a8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44624/0 (socket says 172.21.15.31:44624) 2024-03-30T01:42:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2b59e700 1 -- 172.21.15.31:0/2278116743 learned_addr learned my addr 172.21.15.31:0/2278116743 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2a59c700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce2c06dc50 0x7fce2c14d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2ad9d700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce2c06a700 0x7fce2c150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2b59e700 1 -- 172.21.15.31:0/2278116743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce2c06a700 msgr2=0x7fce2c150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2b59e700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce2c06a700 0x7fce2c150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2b59e700 1 -- 172.21.15.31:0/2278116743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce2c06dc50 msgr2=0x7fce2c14d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2b59e700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce2c06dc50 0x7fce2c14d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2b59e700 1 -- 172.21.15.31:0/2278116743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce24009a70 con 0x7fce2c0c10c0 2024-03-30T01:42:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2ad9d700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce2c06a700 0x7fce2c150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce2b59e700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 0x7fce2c0a8180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0c0d50 tx=0x7fce24004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.871+0000 7fce237fe700 1 -- 172.21.15.31:0/2278116743 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24012630 con 0x7fce2c0c10c0 2024-03-30T01:42:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.872+0000 7fce237fe700 1 -- 172.21.15.31:0/2278116743 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce240127b0 con 0x7fce2c0c10c0 2024-03-30T01:42:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.872+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c0ab280 con 0x7fce2c0c10c0 2024-03-30T01:42:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.872+0000 7fce237fe700 1 -- 172.21.15.31:0/2278116743 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24012b60 con 0x7fce2c0c10c0 2024-03-30T01:42:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.872+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c1ea500 con 0x7fce2c0c10c0 2024-03-30T01:42:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.872+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c1ea080 con 0x7fce2c0c10c0 2024-03-30T01:42:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.873+0000 7fce237fe700 1 -- 172.21.15.31:0/2278116743 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fce24012cc0 con 0x7fce2c0c10c0 2024-03-30T01:42:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.873+0000 7fce237fe700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce0c06ac90 0x7fce0c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.874+0000 7fce237fe700 1 -- 172.21.15.31:0/2278116743 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fce2408b980 con 0x7fce2c0c10c0 2024-03-30T01:42:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.874+0000 7fce2ad9d700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce0c06ac90 0x7fce0c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:27.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.874+0000 7fce2ad9d700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce0c06ac90 0x7fce0c06d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fce1c007fd0 tx=0x7fce1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:27.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:27.879+0000 7fce237fe700 1 -- 172.21.15.31:0/2278116743 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fce2c1ea080 con 0x7fce2c0c10c0 2024-03-30T01:42:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.095+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce2c1ea080 con 0x7fce2c0c10c0 2024-03-30T01:42:28.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.096+0000 7fce237fe700 1 -- 172.21.15.31:0/2278116743 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+19974 (secure 0 0 0) 0x7fce2c1ea080 con 0x7fce2c0c10c0 2024-03-30T01:42:28.103 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:28.103 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":43,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:25.270018+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce0c06ac90 msgr2=0x7fce0c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce0c06ac90 0x7fce0c06d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fce1c007fd0 tx=0x7fce1c009040 comp rx=0 tx=0).stop 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 msgr2=0x7fce2c0a8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 0x7fce2c0a8180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0c0d50 tx=0x7fce24004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 shutdown_connections 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce2c0c10c0 0x7fce2c0a8180 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fce0c06ac90 0x7fce0c06d150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce2c06a700 0x7fce2c150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 --2- 172.21.15.31:0/2278116743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce2c06dc50 0x7fce2c14d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 >> 172.21.15.31:0/2278116743 conn(0x7fce2c0c3190 msgr2=0x7fce2c151ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 shutdown_connections 2024-03-30T01:42:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.099+0000 7fce31168700 1 -- 172.21.15.31:0/2278116743 wait complete. 2024-03-30T01:42:28.119 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T01:42:28.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:42:28.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.249+0000 7fa2788a5700 1 Processor -- start 2024-03-30T01:42:28.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.249+0000 7fa2788a5700 1 -- start start 2024-03-30T01:42:28.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa2788a5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2740ac520 0x7fa2740ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa2788a5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa2740ace40 0x7fa2740c6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa2788a5700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa274153590 con 0x7fa2740a74e0 2024-03-30T01:42:28.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa2788a5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2740ab680 con 0x7fa2740ace40 2024-03-30T01:42:28.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa2788a5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741500c0 con 0x7fa2740ac520 2024-03-30T01:42:28.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa271d9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa2740a74e0 0x7fa2740c6000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47374/0 (socket says 172.21.15.31:47374) 2024-03-30T01:42:28.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa271d9b700 1 -- 172.21.15.31:0/2229221713 learned_addr learned my addr 172.21.15.31:0/2229221713 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa272d9d700 1 --2- 172.21.15.31:0/2229221713 >> v2:172.21.15.59:3300/0 conn(0x7fa2740ace40 0x7fa2740c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27259c700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2740ac520 0x7fa2740ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578146437 0 0) 0x7fa274153590 con 0x7fa2740a74e0 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2600035b0 con 0x7fa2740a74e0 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2598439552 0 0) 0x7fa2600035b0 con 0x7fa2740a74e0 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2740ac520 msgr2=0x7fa2740ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2740ac520 0x7fa2740ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 >> v2:172.21.15.59:3300/0 conn(0x7fa2740ace40 msgr2=0x7fa2740c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 --2- 172.21.15.31:0/2229221713 >> v2:172.21.15.59:3300/0 conn(0x7fa2740ace40 0x7fa2740c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2740c7990 con 0x7fa2740a74e0 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7fa25c001670 con 0x7fa2740a74e0 2024-03-30T01:42:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 >> v1:172.21.15.31:6789/0 conn(0x7fa2740a74e0 legacy=0x7fa2740c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa260003a00 0x7fa260003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260004850 0x7fa260004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2740ac520 0x7fa260005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2740ab680 con 0x7fa260003a00 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741500c0 con 0x7fa260004850 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26000ba10 con 0x7fa2740ac520 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27259c700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa260003a00 0x7fa260003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa271d9b700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260004850 0x7fa260004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa272d9d700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2740ac520 0x7fa260005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27259c700 1 -- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2740ac520 msgr2=0x7fa260005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27259c700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2740ac520 0x7fa260005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27259c700 1 -- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260004850 msgr2=0x7fa260004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27259c700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260004850 0x7fa260004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27259c700 1 -- 172.21.15.31:0/2229221713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600035b0 con 0x7fa260003a00 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa271d9b700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260004850 0x7fa260004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.251+0000 7fa27259c700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa260003a00 0x7fa260003e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa264009b90 tx=0x7fa260005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd286de4fe478e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26400e6c0 con 0x7fa260003a00 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26400e820 con 0x7fa260003a00 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa27159a700 1 -- 172.21.15.31:0/2229221713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26400eb90 con 0x7fa260003a00 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa2788a5700 1 -- 172.21.15.31:0/2229221713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa260003a00 msgr2=0x7fa260003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa2788a5700 1 --2- 172.21.15.31:0/2229221713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa260003a00 0x7fa260003e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa264009b90 tx=0x7fa260005880 comp rx=0 tx=0).stop 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa272d9d700 1 -- 172.21.15.31:0/2229221713 reap_dead start 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa2788a5700 1 -- 172.21.15.31:0/2229221713 shutdown_connections 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa2788a5700 1 -- 172.21.15.31:0/2229221713 >> 172.21.15.31:0/2229221713 conn(0x7fa27406ad30 msgr2=0x7fa2740a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa2788a5700 1 -- 172.21.15.31:0/2229221713 shutdown_connections 2024-03-30T01:42:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.252+0000 7fa2788a5700 1 -- 172.21.15.31:0/2229221713 wait complete. 2024-03-30T01:42:28.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.254+0000 7fa2788a5700 1 Processor -- start 2024-03-30T01:42:28.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.254+0000 7fa2788a5700 1 -- start start 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa2788a5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260003a00 0x7fa2740bdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa2788a5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2741563f0 0x7fa2741ebad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa2788a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2741e8ff0 0x7fa27414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa2788a5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2740be100 con 0x7fa2741e8ff0 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa2788a5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27414fb40 con 0x7fa260003a00 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa2788a5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741ec010 con 0x7fa2741563f0 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa272d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2741e8ff0 0x7fa27414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa272d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2741e8ff0 0x7fa27414bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44648/0 (socket says 172.21.15.31:44648) 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa272d9d700 1 -- 172.21.15.31:0/776205951 learned_addr learned my addr 172.21.15.31:0/776205951 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa271d9b700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2741563f0 0x7fa2741ebad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa27259c700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260003a00 0x7fa2740bdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa272d9d700 1 -- 172.21.15.31:0/776205951 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2741563f0 msgr2=0x7fa2741ebad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa272d9d700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2741563f0 0x7fa2741ebad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa272d9d700 1 -- 172.21.15.31:0/776205951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260003a00 msgr2=0x7fa2740bdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa272d9d700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260003a00 0x7fa2740bdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa272d9d700 1 -- 172.21.15.31:0/776205951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2741568b0 con 0x7fa2741e8ff0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa272d9d700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2741e8ff0 0x7fa27414bcc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa2740ad220 tx=0x7fa26c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa26affd700 1 -- 172.21.15.31:0/776205951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00e040 con 0x7fa2741e8ff0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa26affd700 1 -- 172.21.15.31:0/776205951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c00f040 con 0x7fa2741e8ff0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2740bce00 con 0x7fa2741e8ff0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.255+0000 7fa26affd700 1 -- 172.21.15.31:0/776205951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00fdb0 con 0x7fa2741e8ff0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.256+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2740bd3c0 con 0x7fa2741e8ff0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.257+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27404ea80 con 0x7fa2741e8ff0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.257+0000 7fa26affd700 1 -- 172.21.15.31:0/776205951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa26c015430 con 0x7fa2741e8ff0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.257+0000 7fa26affd700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa25406ac90 0x7fa25406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.258+0000 7fa26affd700 1 -- 172.21.15.31:0/776205951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fa26c08a060 con 0x7fa2741e8ff0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.258+0000 7fa27259c700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa25406ac90 0x7fa25406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.258+0000 7fa27259c700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa25406ac90 0x7fa25406d150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa2640090b0 tx=0x7fa264008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.263+0000 7fa26affd700 1 -- 172.21.15.31:0/776205951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa26c0bd580 con 0x7fa2741e8ff0 2024-03-30T01:42:28.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:42:28.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:42:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.474+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2740ad2d0 con 0x7fa25406ac90 2024-03-30T01:42:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.489+0000 7fa26affd700 1 -- 172.21.15.31:0/776205951 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+516228 (secure 0 0 0) 0x7fa26400e990 con 0x7fa25406ac90 2024-03-30T01:42:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa25406ac90 msgr2=0x7fa25406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa25406ac90 0x7fa25406d150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa2640090b0 tx=0x7fa264008030 comp rx=0 tx=0).stop 2024-03-30T01:42:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2741e8ff0 msgr2=0x7fa27414bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2741e8ff0 0x7fa27414bcc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa2740ad220 tx=0x7fa26c011f80 comp rx=0 tx=0).stop 2024-03-30T01:42:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 shutdown_connections 2024-03-30T01:42:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2741e8ff0 0x7fa27414bcc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa25406ac90 0x7fa25406d150 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa260003a00 0x7fa2740bdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 --2- 172.21.15.31:0/776205951 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2741563f0 0x7fa2741ebad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 >> 172.21.15.31:0/776205951 conn(0x7fa27406ad30 msgr2=0x7fa2740c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 shutdown_connections 2024-03-30T01:42:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.494+0000 7fa2788a5700 1 -- 172.21.15.31:0/776205951 wait complete. 2024-03-30T01:42:28.508 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:42:28.514 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-03-30T01:42:27.827998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594241,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220932,"kb_used_data":4740,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753788,"statfs":{"total":773094113280,"available":772867878912,"internally_reserved":0,"allocated":4853760,"data_stored":2740433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.023745"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877685+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:42:11.877685+0000","last_peered":"2024-03-30T01:42:11.877685+0000","last_clean":"2024-03-30T01:42:11.877685+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:42:11.877685+0000","last_undegraded":"2024-03-30T01:42:11.877685+0000","last_fullsized":"2024-03-30T01:42:11.877685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874412+0000","last_change":"2024-03-30T01:42:09.836532+0000","last_active":"2024-03-30T01:42:12.874412+0000","last_peered":"2024-03-30T01:42:12.874412+0000","last_clean":"2024-03-30T01:42:12.874412+0000","last_became_active":"2024-03-30T01:42:09.836262+0000","last_became_peered":"2024-03-30T01:42:09.836262+0000","last_unstale":"2024-03-30T01:42:12.874412+0000","last_undegraded":"2024-03-30T01:42:12.874412+0000","last_fullsized":"2024-03-30T01:42:12.874412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:08.249431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877538+0000","last_change":"2024-03-30T01:42:10.289721+0000","last_active":"2024-03-30T01:42:12.877538+0000","last_peered":"2024-03-30T01:42:12.877538+0000","last_clean":"2024-03-30T01:42:12.877538+0000","last_became_active":"2024-03-30T01:42:10.289413+0000","last_became_peered":"2024-03-30T01:42:10.289413+0000","last_unstale":"2024-03-30T01:42:12.877538+0000","last_undegraded":"2024-03-30T01:42:12.877538+0000","last_fullsized":"2024-03-30T01:42:12.877538+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814741+0000","last_change":"2024-03-30T01:42:22.814741+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814741+0000","last_undegraded":"2024-03-30T01:42:22.814741+0000","last_fullsized":"2024-03-30T01:42:22.814741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:34.573756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876734+0000","last_change":"2024-03-30T01:42:09.829174+0000","last_active":"2024-03-30T01:42:12.876734+0000","last_peered":"2024-03-30T01:42:12.876734+0000","last_clean":"2024-03-30T01:42:12.876734+0000","last_became_active":"2024-03-30T01:42:09.828730+0000","last_became_peered":"2024-03-30T01:42:09.828730+0000","last_unstale":"2024-03-30T01:42:12.876734+0000","last_undegraded":"2024-03-30T01:42:12.876734+0000","last_fullsized":"2024-03-30T01:42:12.876734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874371+0000","last_change":"2024-03-30T01:42:10.284566+0000","last_active":"2024-03-30T01:42:12.874371+0000","last_peered":"2024-03-30T01:42:12.874371+0000","last_clean":"2024-03-30T01:42:12.874371+0000","last_became_active":"2024-03-30T01:42:10.284192+0000","last_became_peered":"2024-03-30T01:42:10.284192+0000","last_unstale":"2024-03-30T01:42:12.874371+0000","last_undegraded":"2024-03-30T01:42:12.874371+0000","last_fullsized":"2024-03-30T01:42:12.874371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:05.460383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809509+0000","last_change":"2024-03-30T01:42:22.809509+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809509+0000","last_undegraded":"2024-03-30T01:42:22.809509+0000","last_fullsized":"2024-03-30T01:42:22.809509+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:56.846961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874138+0000","last_change":"2024-03-30T01:42:09.832991+0000","last_active":"2024-03-30T01:42:12.874138+0000","last_peered":"2024-03-30T01:42:12.874138+0000","last_clean":"2024-03-30T01:42:12.874138+0000","last_became_active":"2024-03-30T01:42:09.832682+0000","last_became_peered":"2024-03-30T01:42:09.832682+0000","last_unstale":"2024-03-30T01:42:12.874138+0000","last_undegraded":"2024-03-30T01:42:12.874138+0000","last_fullsized":"2024-03-30T01:42:12.874138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:37.427434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874189+0000","last_change":"2024-03-30T01:42:10.299170+0000","last_active":"2024-03-30T01:42:12.874189+0000","last_peered":"2024-03-30T01:42:12.874189+0000","last_clean":"2024-03-30T01:42:12.874189+0000","last_became_active":"2024-03-30T01:42:10.298846+0000","last_became_peered":"2024-03-30T01:42:10.298846+0000","last_unstale":"2024-03-30T01:42:12.874189+0000","last_undegraded":"2024-03-30T01:42:12.874189+0000","last_fullsized":"2024-03-30T01:42:12.874189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809736+0000","last_change":"2024-03-30T01:42:22.809736+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809736+0000","last_undegraded":"2024-03-30T01:42:22.809736+0000","last_fullsized":"2024-03-30T01:42:22.809736+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:43.792947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881014+0000","last_change":"2024-03-30T01:42:09.822074+0000","last_active":"2024-03-30T01:42:12.881014+0000","last_peered":"2024-03-30T01:42:12.881014+0000","last_clean":"2024-03-30T01:42:12.881014+0000","last_became_active":"2024-03-30T01:42:09.821786+0000","last_became_peered":"2024-03-30T01:42:09.821786+0000","last_unstale":"2024-03-30T01:42:12.881014+0000","last_undegraded":"2024-03-30T01:42:12.881014+0000","last_fullsized":"2024-03-30T01:42:12.881014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:18.840885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877139+0000","last_change":"2024-03-30T01:42:10.291029+0000","last_active":"2024-03-30T01:42:12.877139+0000","last_peered":"2024-03-30T01:42:12.877139+0000","last_clean":"2024-03-30T01:42:12.877139+0000","last_became_active":"2024-03-30T01:42:10.290728+0000","last_became_peered":"2024-03-30T01:42:10.290728+0000","last_unstale":"2024-03-30T01:42:12.877139+0000","last_undegraded":"2024-03-30T01:42:12.877139+0000","last_fullsized":"2024-03-30T01:42:12.877139+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.804768+0000","last_change":"2024-03-30T01:42:22.804768+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.804768+0000","last_undegraded":"2024-03-30T01:42:22.804768+0000","last_fullsized":"2024-03-30T01:42:22.804768+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877173+0000","last_change":"2024-03-30T01:42:09.827116+0000","last_active":"2024-03-30T01:42:12.877173+0000","last_peered":"2024-03-30T01:42:12.877173+0000","last_clean":"2024-03-30T01:42:12.877173+0000","last_became_active":"2024-03-30T01:42:09.826876+0000","last_became_peered":"2024-03-30T01:42:09.826876+0000","last_unstale":"2024-03-30T01:42:12.877173+0000","last_undegraded":"2024-03-30T01:42:12.877173+0000","last_fullsized":"2024-03-30T01:42:12.877173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:38.792762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877100+0000","last_change":"2024-03-30T01:42:10.288334+0000","last_active":"2024-03-30T01:42:12.877100+0000","last_peered":"2024-03-30T01:42:12.877100+0000","last_clean":"2024-03-30T01:42:12.877100+0000","last_became_active":"2024-03-30T01:42:10.288093+0000","last_became_peered":"2024-03-30T01:42:10.288093+0000","last_unstale":"2024-03-30T01:42:12.877100+0000","last_undegraded":"2024-03-30T01:42:12.877100+0000","last_fullsized":"2024-03-30T01:42:12.877100+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815309+0000","last_change":"2024-03-30T01:42:22.815309+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815309+0000","last_undegraded":"2024-03-30T01:42:22.815309+0000","last_fullsized":"2024-03-30T01:42:22.815309+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:44:11.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877240+0000","last_change":"2024-03-30T01:42:09.830490+0000","last_active":"2024-03-30T01:42:12.877240+0000","last_peered":"2024-03-30T01:42:12.877240+0000","last_clean":"2024-03-30T01:42:12.877240+0000","last_became_active":"2024-03-30T01:42:09.830197+0000","last_became_peered":"2024-03-30T01:42:09.830197+0000","last_unstale":"2024-03-30T01:42:12.877240+0000","last_undegraded":"2024-03-30T01:42:12.877240+0000","last_fullsized":"2024-03-30T01:42:12.877240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:46.433516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876955+0000","last_change":"2024-03-30T01:42:10.299986+0000","last_active":"2024-03-30T01:42:12.876955+0000","last_peered":"2024-03-30T01:42:12.876955+0000","last_clean":"2024-03-30T01:42:12.876955+0000","last_became_active":"2024-03-30T01:42:10.299776+0000","last_became_peered":"2024-03-30T01:42:10.299776+0000","last_unstale":"2024-03-30T01:42:12.876955+0000","last_undegraded":"2024-03-30T01:42:12.876955+0000","last_fullsized":"2024-03-30T01:42:12.876955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812125+0000","last_change":"2024-03-30T01:42:22.812125+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812125+0000","last_undegraded":"2024-03-30T01:42:22.812125+0000","last_fullsized":"2024-03-30T01:42:22.812125+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:58.845872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875741+0000","last_change":"2024-03-30T01:42:09.824373+0000","last_active":"2024-03-30T01:42:12.875741+0000","last_peered":"2024-03-30T01:42:12.875741+0000","last_clean":"2024-03-30T01:42:12.875741+0000","last_became_active":"2024-03-30T01:42:09.824089+0000","last_became_peered":"2024-03-30T01:42:09.824089+0000","last_unstale":"2024-03-30T01:42:12.875741+0000","last_undegraded":"2024-03-30T01:42:12.875741+0000","last_fullsized":"2024-03-30T01:42:12.875741+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:37.434399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874455+0000","last_change":"2024-03-30T01:42:10.284232+0000","last_active":"2024-03-30T01:42:12.874455+0000","last_peered":"2024-03-30T01:42:12.874455+0000","last_clean":"2024-03-30T01:42:12.874455+0000","last_became_active":"2024-03-30T01:42:10.283909+0000","last_became_peered":"2024-03-30T01:42:10.283909+0000","last_unstale":"2024-03-30T01:42:12.874455+0000","last_undegraded":"2024-03-30T01:42:12.874455+0000","last_fullsized":"2024-03-30T01:42:12.874455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:33.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815396+0000","last_change":"2024-03-30T01:42:22.815396+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815396+0000","last_undegraded":"2024-03-30T01:42:22.815396+0000","last_fullsized":"2024-03-30T01:42:22.815396+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:42:51.968187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877562+0000","last_change":"2024-03-30T01:42:09.833189+0000","last_active":"2024-03-30T01:42:12.877562+0000","last_peered":"2024-03-30T01:42:12.877562+0000","last_clean":"2024-03-30T01:42:12.877562+0000","last_became_active":"2024-03-30T01:42:09.832891+0000","last_became_peered":"2024-03-30T01:42:09.832891+0000","last_unstale":"2024-03-30T01:42:12.877562+0000","last_undegraded":"2024-03-30T01:42:12.877562+0000","last_fullsized":"2024-03-30T01:42:12.877562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877221+0000","last_change":"2024-03-30T01:42:10.298216+0000","last_active":"2024-03-30T01:42:12.877221+0000","last_peered":"2024-03-30T01:42:12.877221+0000","last_clean":"2024-03-30T01:42:12.877221+0000","last_became_active":"2024-03-30T01:42:10.297976+0000","last_became_peered":"2024-03-30T01:42:10.297976+0000","last_unstale":"2024-03-30T01:42:12.877221+0000","last_undegraded":"2024-03-30T01:42:12.877221+0000","last_fullsized":"2024-03-30T01:42:12.877221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.807923+0000","last_change":"2024-03-30T01:42:22.807923+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.807923+0000","last_undegraded":"2024-03-30T01:42:22.807923+0000","last_fullsized":"2024-03-30T01:42:22.807923+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875636+0000","last_change":"2024-03-30T01:42:09.829638+0000","last_active":"2024-03-30T01:42:12.875636+0000","last_peered":"2024-03-30T01:42:12.875636+0000","last_clean":"2024-03-30T01:42:12.875636+0000","last_became_active":"2024-03-30T01:42:09.829317+0000","last_became_peered":"2024-03-30T01:42:09.829317+0000","last_unstale":"2024-03-30T01:42:12.875636+0000","last_undegraded":"2024-03-30T01:42:12.875636+0000","last_fullsized":"2024-03-30T01:42:12.875636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:33.639097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876807+0000","last_change":"2024-03-30T01:42:10.292043+0000","last_active":"2024-03-30T01:42:12.876807+0000","last_peered":"2024-03-30T01:42:12.876807+0000","last_clean":"2024-03-30T01:42:12.876807+0000","last_became_active":"2024-03-30T01:42:10.291598+0000","last_became_peered":"2024-03-30T01:42:10.291598+0000","last_unstale":"2024-03-30T01:42:12.876807+0000","last_undegraded":"2024-03-30T01:42:12.876807+0000","last_fullsized":"2024-03-30T01:42:12.876807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.810206+0000","last_change":"2024-03-30T01:42:22.810206+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.810206+0000","last_undegraded":"2024-03-30T01:42:22.810206+0000","last_fullsized":"2024-03-30T01:42:22.810206+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:57.966402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874291+0000","last_change":"2024-03-30T01:42:09.829777+0000","last_active":"2024-03-30T01:42:12.874291+0000","last_peered":"2024-03-30T01:42:12.874291+0000","last_clean":"2024-03-30T01:42:12.874291+0000","last_became_active":"2024-03-30T01:42:09.829483+0000","last_became_peered":"2024-03-30T01:42:09.829483+0000","last_unstale":"2024-03-30T01:42:12.874291+0000","last_undegraded":"2024-03-30T01:42:12.874291+0000","last_fullsized":"2024-03-30T01:42:12.874291+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:45.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877410+0000","last_change":"2024-03-30T01:42:10.285649+0000","last_active":"2024-03-30T01:42:12.877410+0000","last_peered":"2024-03-30T01:42:12.877410+0000","last_clean":"2024-03-30T01:42:12.877410+0000","last_became_active":"2024-03-30T01:42:10.285297+0000","last_became_peered":"2024-03-30T01:42:10.285297+0000","last_unstale":"2024-03-30T01:42:12.877410+0000","last_undegraded":"2024-03-30T01:42:12.877410+0000","last_fullsized":"2024-03-30T01:42:12.877410+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809986+0000","last_change":"2024-03-30T01:42:22.809986+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809986+0000","last_undegraded":"2024-03-30T01:42:22.809986+0000","last_fullsized":"2024-03-30T01:42:22.809986+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:19:37.521361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875652+0000","last_change":"2024-03-30T01:42:09.826256+0000","last_active":"2024-03-30T01:42:12.875652+0000","last_peered":"2024-03-30T01:42:12.875652+0000","last_clean":"2024-03-30T01:42:12.875652+0000","last_became_active":"2024-03-30T01:42:09.825956+0000","last_became_peered":"2024-03-30T01:42:09.825956+0000","last_unstale":"2024-03-30T01:42:12.875652+0000","last_undegraded":"2024-03-30T01:42:12.875652+0000","last_fullsized":"2024-03-30T01:42:12.875652+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:59:41.446927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877149+0000","last_change":"2024-03-30T01:42:10.290469+0000","last_active":"2024-03-30T01:42:12.877149+0000","last_peered":"2024-03-30T01:42:12.877149+0000","last_clean":"2024-03-30T01:42:12.877149+0000","last_became_active":"2024-03-30T01:42:10.290148+0000","last_became_peered":"2024-03-30T01:42:10.290148+0000","last_unstale":"2024-03-30T01:42:12.877149+0000","last_undegraded":"2024-03-30T01:42:12.877149+0000","last_fullsized":"2024-03-30T01:42:12.877149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.805285+0000","last_change":"2024-03-30T01:42:22.805285+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.805285+0000","last_undegraded":"2024-03-30T01:42:22.805285+0000","last_fullsized":"2024-03-30T01:42:22.805285+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:54.923310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875675+0000","last_change":"2024-03-30T01:42:09.830452+0000","last_active":"2024-03-30T01:42:12.875675+0000","last_peered":"2024-03-30T01:42:12.875675+0000","last_clean":"2024-03-30T01:42:12.875675+0000","last_became_active":"2024-03-30T01:42:09.830173+0000","last_became_peered":"2024-03-30T01:42:09.830173+0000","last_unstale":"2024-03-30T01:42:12.875675+0000","last_undegraded":"2024-03-30T01:42:12.875675+0000","last_fullsized":"2024-03-30T01:42:12.875675+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:12:42.807604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874172+0000","last_change":"2024-03-30T01:42:10.279913+0000","last_active":"2024-03-30T01:42:12.874172+0000","last_peered":"2024-03-30T01:42:12.874172+0000","last_clean":"2024-03-30T01:42:12.874172+0000","last_became_active":"2024-03-30T01:42:10.279450+0000","last_became_peered":"2024-03-30T01:42:10.279450+0000","last_unstale":"2024-03-30T01:42:12.874172+0000","last_undegraded":"2024-03-30T01:42:12.874172+0000","last_fullsized":"2024-03-30T01:42:12.874172+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814722+0000","last_change":"2024-03-30T01:42:22.814722+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814722+0000","last_undegraded":"2024-03-30T01:42:22.814722+0000","last_fullsized":"2024-03-30T01:42:22.814722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:44.454854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877240+0000","last_change":"2024-03-30T01:42:09.828572+0000","last_active":"2024-03-30T01:42:12.877240+0000","last_peered":"2024-03-30T01:42:12.877240+0000","last_clean":"2024-03-30T01:42:12.877240+0000","last_became_active":"2024-03-30T01:42:09.828225+0000","last_became_peered":"2024-03-30T01:42:09.828225+0000","last_unstale":"2024-03-30T01:42:12.877240+0000","last_undegraded":"2024-03-30T01:42:12.877240+0000","last_fullsized":"2024-03-30T01:42:12.877240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875755+0000","last_change":"2024-03-30T01:42:10.286900+0000","last_active":"2024-03-30T01:42:12.875755+0000","last_peered":"2024-03-30T01:42:12.875755+0000","last_clean":"2024-03-30T01:42:12.875755+0000","last_became_active":"2024-03-30T01:42:10.286598+0000","last_became_peered":"2024-03-30T01:42:10.286598+0000","last_unstale":"2024-03-30T01:42:12.875755+0000","last_undegraded":"2024-03-30T01:42:12.875755+0000","last_fullsized":"2024-03-30T01:42:12.875755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:46.592007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.805879+0000","last_change":"2024-03-30T01:42:22.805879+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.805879+0000","last_undegraded":"2024-03-30T01:42:22.805879+0000","last_fullsized":"2024-03-30T01:42:22.805879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:57.756956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875729+0000","last_change":"2024-03-30T01:42:09.823221+0000","last_active":"2024-03-30T01:42:12.875729+0000","last_peered":"2024-03-30T01:42:12.875729+0000","last_clean":"2024-03-30T01:42:12.875729+0000","last_became_active":"2024-03-30T01:42:09.822882+0000","last_became_peered":"2024-03-30T01:42:09.822882+0000","last_unstale":"2024-03-30T01:42:12.875729+0000","last_undegraded":"2024-03-30T01:42:12.875729+0000","last_fullsized":"2024-03-30T01:42:12.875729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:00:31.185685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874279+0000","last_change":"2024-03-30T01:42:10.304085+0000","last_active":"2024-03-30T01:42:12.874279+0000","last_peered":"2024-03-30T01:42:12.874279+0000","last_clean":"2024-03-30T01:42:12.874279+0000","last_became_active":"2024-03-30T01:42:10.303817+0000","last_became_peered":"2024-03-30T01:42:10.303817+0000","last_unstale":"2024-03-30T01:42:12.874279+0000","last_undegraded":"2024-03-30T01:42:12.874279+0000","last_fullsized":"2024-03-30T01:42:12.874279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814798+0000","last_change":"2024-03-30T01:42:22.814798+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814798+0000","last_undegraded":"2024-03-30T01:42:22.814798+0000","last_fullsized":"2024-03-30T01:42:22.814798+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:06.958915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874246+0000","last_change":"2024-03-30T01:42:09.829713+0000","last_active":"2024-03-30T01:42:12.874246+0000","last_peered":"2024-03-30T01:42:12.874246+0000","last_clean":"2024-03-30T01:42:12.874246+0000","last_became_active":"2024-03-30T01:42:09.829375+0000","last_became_peered":"2024-03-30T01:42:09.829375+0000","last_unstale":"2024-03-30T01:42:12.874246+0000","last_undegraded":"2024-03-30T01:42:12.874246+0000","last_fullsized":"2024-03-30T01:42:12.874246+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:49:51.601663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877472+0000","last_change":"2024-03-30T01:42:10.295516+0000","last_active":"2024-03-30T01:42:12.877472+0000","last_peered":"2024-03-30T01:42:12.877472+0000","last_clean":"2024-03-30T01:42:12.877472+0000","last_became_active":"2024-03-30T01:42:10.295133+0000","last_became_peered":"2024-03-30T01:42:10.295133+0000","last_unstale":"2024-03-30T01:42:12.877472+0000","last_undegraded":"2024-03-30T01:42:12.877472+0000","last_fullsized":"2024-03-30T01:42:12.877472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812100+0000","last_change":"2024-03-30T01:42:22.812100+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812100+0000","last_undegraded":"2024-03-30T01:42:22.812100+0000","last_fullsized":"2024-03-30T01:42:22.812100+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:46.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877549+0000","last_change":"2024-03-30T01:42:09.826895+0000","last_active":"2024-03-30T01:42:12.877549+0000","last_peered":"2024-03-30T01:42:12.877549+0000","last_clean":"2024-03-30T01:42:12.877549+0000","last_became_active":"2024-03-30T01:42:09.826609+0000","last_became_peered":"2024-03-30T01:42:09.826609+0000","last_unstale":"2024-03-30T01:42:12.877549+0000","last_undegraded":"2024-03-30T01:42:12.877549+0000","last_fullsized":"2024-03-30T01:42:12.877549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:42:59.036073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876887+0000","last_change":"2024-03-30T01:42:10.291564+0000","last_active":"2024-03-30T01:42:12.876887+0000","last_peered":"2024-03-30T01:42:12.876887+0000","last_clean":"2024-03-30T01:42:12.876887+0000","last_became_active":"2024-03-30T01:42:10.291285+0000","last_became_peered":"2024-03-30T01:42:10.291285+0000","last_unstale":"2024-03-30T01:42:12.876887+0000","last_undegraded":"2024-03-30T01:42:12.876887+0000","last_fullsized":"2024-03-30T01:42:12.876887+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812173+0000","last_change":"2024-03-30T01:42:22.812173+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812173+0000","last_undegraded":"2024-03-30T01:42:22.812173+0000","last_fullsized":"2024-03-30T01:42:22.812173+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:49.116227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874199+0000","last_change":"2024-03-30T01:42:09.831869+0000","last_active":"2024-03-30T01:42:12.874199+0000","last_peered":"2024-03-30T01:42:12.874199+0000","last_clean":"2024-03-30T01:42:12.874199+0000","last_became_active":"2024-03-30T01:42:09.831562+0000","last_became_peered":"2024-03-30T01:42:09.831562+0000","last_unstale":"2024-03-30T01:42:12.874199+0000","last_undegraded":"2024-03-30T01:42:12.874199+0000","last_fullsized":"2024-03-30T01:42:12.874199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:35.938001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876785+0000","last_change":"2024-03-30T01:42:10.288399+0000","last_active":"2024-03-30T01:42:12.876785+0000","last_peered":"2024-03-30T01:42:12.876785+0000","last_clean":"2024-03-30T01:42:12.876785+0000","last_became_active":"2024-03-30T01:42:10.287890+0000","last_became_peered":"2024-03-30T01:42:10.287890+0000","last_unstale":"2024-03-30T01:42:12.876785+0000","last_undegraded":"2024-03-30T01:42:12.876785+0000","last_fullsized":"2024-03-30T01:42:12.876785+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812225+0000","last_change":"2024-03-30T01:42:22.812225+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812225+0000","last_undegraded":"2024-03-30T01:42:22.812225+0000","last_fullsized":"2024-03-30T01:42:22.812225+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:59.368657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.880961+0000","last_change":"2024-03-30T01:42:09.820451+0000","last_active":"2024-03-30T01:42:12.880961+0000","last_peered":"2024-03-30T01:42:12.880961+0000","last_clean":"2024-03-30T01:42:12.880961+0000","last_became_active":"2024-03-30T01:42:09.820152+0000","last_became_peered":"2024-03-30T01:42:09.820152+0000","last_unstale":"2024-03-30T01:42:12.880961+0000","last_undegraded":"2024-03-30T01:42:12.880961+0000","last_fullsized":"2024-03-30T01:42:12.880961+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:04.153303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874321+0000","last_change":"2024-03-30T01:42:10.279681+0000","last_active":"2024-03-30T01:42:12.874321+0000","last_peered":"2024-03-30T01:42:12.874321+0000","last_clean":"2024-03-30T01:42:12.874321+0000","last_became_active":"2024-03-30T01:42:10.279333+0000","last_became_peered":"2024-03-30T01:42:10.279333+0000","last_unstale":"2024-03-30T01:42:12.874321+0000","last_undegraded":"2024-03-30T01:42:12.874321+0000","last_fullsized":"2024-03-30T01:42:12.874321+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814765+0000","last_change":"2024-03-30T01:42:22.814765+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814765+0000","last_undegraded":"2024-03-30T01:42:22.814765+0000","last_fullsized":"2024-03-30T01:42:22.814765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.379389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877207+0000","last_change":"2024-03-30T01:42:09.829194+0000","last_active":"2024-03-30T01:42:12.877207+0000","last_peered":"2024-03-30T01:42:12.877207+0000","last_clean":"2024-03-30T01:42:12.877207+0000","last_became_active":"2024-03-30T01:42:09.828257+0000","last_became_peered":"2024-03-30T01:42:09.828257+0000","last_unstale":"2024-03-30T01:42:12.877207+0000","last_undegraded":"2024-03-30T01:42:12.877207+0000","last_fullsized":"2024-03-30T01:42:12.877207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876760+0000","last_change":"2024-03-30T01:42:10.288326+0000","last_active":"2024-03-30T01:42:12.876760+0000","last_peered":"2024-03-30T01:42:12.876760+0000","last_clean":"2024-03-30T01:42:12.876760+0000","last_became_active":"2024-03-30T01:42:10.288050+0000","last_became_peered":"2024-03-30T01:42:10.288050+0000","last_unstale":"2024-03-30T01:42:12.876760+0000","last_undegraded":"2024-03-30T01:42:12.876760+0000","last_fullsized":"2024-03-30T01:42:12.876760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809500+0000","last_change":"2024-03-30T01:42:22.809500+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809500+0000","last_undegraded":"2024-03-30T01:42:22.809500+0000","last_fullsized":"2024-03-30T01:42:22.809500+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:20.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876861+0000","last_change":"2024-03-30T01:42:09.832318+0000","last_active":"2024-03-30T01:42:12.876861+0000","last_peered":"2024-03-30T01:42:12.876861+0000","last_clean":"2024-03-30T01:42:12.876861+0000","last_became_active":"2024-03-30T01:42:09.831964+0000","last_became_peered":"2024-03-30T01:42:09.831964+0000","last_unstale":"2024-03-30T01:42:12.876861+0000","last_undegraded":"2024-03-30T01:42:12.876861+0000","last_fullsized":"2024-03-30T01:42:12.876861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:05.700970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877072+0000","last_change":"2024-03-30T01:42:10.285053+0000","last_active":"2024-03-30T01:42:12.877072+0000","last_peered":"2024-03-30T01:42:12.877072+0000","last_clean":"2024-03-30T01:42:12.877072+0000","last_became_active":"2024-03-30T01:42:10.284814+0000","last_became_peered":"2024-03-30T01:42:10.284814+0000","last_unstale":"2024-03-30T01:42:12.877072+0000","last_undegraded":"2024-03-30T01:42:12.877072+0000","last_fullsized":"2024-03-30T01:42:12.877072+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:28.875197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.807918+0000","last_change":"2024-03-30T01:42:22.807918+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.807918+0000","last_undegraded":"2024-03-30T01:42:22.807918+0000","last_fullsized":"2024-03-30T01:42:22.807918+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:29:59.201428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:18.899337+0000","last_change":"2024-03-30T01:42:09.823152+0000","last_active":"2024-03-30T01:42:18.899337+0000","last_peered":"2024-03-30T01:42:18.899337+0000","last_clean":"2024-03-30T01:42:18.899337+0000","last_became_active":"2024-03-30T01:42:09.822826+0000","last_became_peered":"2024-03-30T01:42:09.822826+0000","last_unstale":"2024-03-30T01:42:18.899337+0000","last_undegraded":"2024-03-30T01:42:18.899337+0000","last_fullsized":"2024-03-30T01:42:18.899337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:17.799215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874291+0000","last_change":"2024-03-30T01:42:10.296944+0000","last_active":"2024-03-30T01:42:12.874291+0000","last_peered":"2024-03-30T01:42:12.874291+0000","last_clean":"2024-03-30T01:42:12.874291+0000","last_became_active":"2024-03-30T01:42:10.296714+0000","last_became_peered":"2024-03-30T01:42:10.296714+0000","last_unstale":"2024-03-30T01:42:12.874291+0000","last_undegraded":"2024-03-30T01:42:12.874291+0000","last_fullsized":"2024-03-30T01:42:12.874291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815307+0000","last_change":"2024-03-30T01:42:22.815307+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815307+0000","last_undegraded":"2024-03-30T01:42:22.815307+0000","last_fullsized":"2024-03-30T01:42:22.815307+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877496+0000","last_change":"2024-03-30T01:42:09.833439+0000","last_active":"2024-03-30T01:42:12.877496+0000","last_peered":"2024-03-30T01:42:12.877496+0000","last_clean":"2024-03-30T01:42:12.877496+0000","last_became_active":"2024-03-30T01:42:09.833137+0000","last_became_peered":"2024-03-30T01:42:09.833137+0000","last_unstale":"2024-03-30T01:42:12.877496+0000","last_undegraded":"2024-03-30T01:42:12.877496+0000","last_fullsized":"2024-03-30T01:42:12.877496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877264+0000","last_change":"2024-03-30T01:42:10.292182+0000","last_active":"2024-03-30T01:42:12.877264+0000","last_peered":"2024-03-30T01:42:12.877264+0000","last_clean":"2024-03-30T01:42:12.877264+0000","last_became_active":"2024-03-30T01:42:10.291868+0000","last_became_peered":"2024-03-30T01:42:10.291868+0000","last_unstale":"2024-03-30T01:42:12.877264+0000","last_undegraded":"2024-03-30T01:42:12.877264+0000","last_fullsized":"2024-03-30T01:42:12.877264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.807910+0000","last_change":"2024-03-30T01:42:22.807910+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.807910+0000","last_undegraded":"2024-03-30T01:42:22.807910+0000","last_fullsized":"2024-03-30T01:42:22.807910+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:12.384136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877526+0000","last_change":"2024-03-30T01:42:09.826561+0000","last_active":"2024-03-30T01:42:12.877526+0000","last_peered":"2024-03-30T01:42:12.877526+0000","last_clean":"2024-03-30T01:42:12.877526+0000","last_became_active":"2024-03-30T01:42:09.826059+0000","last_became_peered":"2024-03-30T01:42:09.826059+0000","last_unstale":"2024-03-30T01:42:12.877526+0000","last_undegraded":"2024-03-30T01:42:12.877526+0000","last_fullsized":"2024-03-30T01:42:12.877526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874434+0000","last_change":"2024-03-30T01:42:10.284755+0000","last_active":"2024-03-30T01:42:12.874434+0000","last_peered":"2024-03-30T01:42:12.874434+0000","last_clean":"2024-03-30T01:42:12.874434+0000","last_became_active":"2024-03-30T01:42:10.284421+0000","last_became_peered":"2024-03-30T01:42:10.284421+0000","last_unstale":"2024-03-30T01:42:12.874434+0000","last_undegraded":"2024-03-30T01:42:12.874434+0000","last_fullsized":"2024-03-30T01:42:12.874434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.810039+0000","last_change":"2024-03-30T01:42:22.810039+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.810039+0000","last_undegraded":"2024-03-30T01:42:22.810039+0000","last_fullsized":"2024-03-30T01:42:22.810039+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:44.146196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877020+0000","last_change":"2024-03-30T01:42:09.828624+0000","last_active":"2024-03-30T01:42:12.877020+0000","last_peered":"2024-03-30T01:42:12.877020+0000","last_clean":"2024-03-30T01:42:12.877020+0000","last_became_active":"2024-03-30T01:42:09.828341+0000","last_became_peered":"2024-03-30T01:42:09.828341+0000","last_unstale":"2024-03-30T01:42:12.877020+0000","last_undegraded":"2024-03-30T01:42:12.877020+0000","last_fullsized":"2024-03-30T01:42:12.877020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:32.138427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875705+0000","last_change":"2024-03-30T01:42:10.295259+0000","last_active":"2024-03-30T01:42:12.875705+0000","last_peered":"2024-03-30T01:42:12.875705+0000","last_clean":"2024-03-30T01:42:12.875705+0000","last_became_active":"2024-03-30T01:42:10.295054+0000","last_became_peered":"2024-03-30T01:42:10.295054+0000","last_unstale":"2024-03-30T01:42:12.875705+0000","last_undegraded":"2024-03-30T01:42:12.875705+0000","last_fullsized":"2024-03-30T01:42:12.875705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:36.942583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.808461+0000","last_change":"2024-03-30T01:42:22.808461+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.808461+0000","last_undegraded":"2024-03-30T01:42:22.808461+0000","last_fullsized":"2024-03-30T01:42:22.808461+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:54:59.010522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874117+0000","last_change":"2024-03-30T01:42:09.820512+0000","last_active":"2024-03-30T01:42:12.874117+0000","last_peered":"2024-03-30T01:42:12.874117+0000","last_clean":"2024-03-30T01:42:12.874117+0000","last_became_active":"2024-03-30T01:42:09.820253+0000","last_became_peered":"2024-03-30T01:42:09.820253+0000","last_unstale":"2024-03-30T01:42:12.874117+0000","last_undegraded":"2024-03-30T01:42:12.874117+0000","last_fullsized":"2024-03-30T01:42:12.874117+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.455801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876834+0000","last_change":"2024-03-30T01:42:10.291872+0000","last_active":"2024-03-30T01:42:12.876834+0000","last_peered":"2024-03-30T01:42:12.876834+0000","last_clean":"2024-03-30T01:42:12.876834+0000","last_became_active":"2024-03-30T01:42:10.291199+0000","last_became_peered":"2024-03-30T01:42:10.291199+0000","last_unstale":"2024-03-30T01:42:12.876834+0000","last_undegraded":"2024-03-30T01:42:12.876834+0000","last_fullsized":"2024-03-30T01:42:12.876834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815299+0000","last_change":"2024-03-30T01:42:22.815299+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815299+0000","last_undegraded":"2024-03-30T01:42:22.815299+0000","last_fullsized":"2024-03-30T01:42:22.815299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877166+0000","last_change":"2024-03-30T01:42:09.826326+0000","last_active":"2024-03-30T01:42:12.877166+0000","last_peered":"2024-03-30T01:42:12.877166+0000","last_clean":"2024-03-30T01:42:12.877166+0000","last_became_active":"2024-03-30T01:42:09.826050+0000","last_became_peered":"2024-03-30T01:42:09.826050+0000","last_unstale":"2024-03-30T01:42:12.877166+0000","last_undegraded":"2024-03-30T01:42:12.877166+0000","last_fullsized":"2024-03-30T01:42:12.877166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874163+0000","last_change":"2024-03-30T01:42:10.303359+0000","last_active":"2024-03-30T01:42:12.874163+0000","last_peered":"2024-03-30T01:42:12.874163+0000","last_clean":"2024-03-30T01:42:12.874163+0000","last_became_active":"2024-03-30T01:42:10.303039+0000","last_became_peered":"2024-03-30T01:42:10.303039+0000","last_unstale":"2024-03-30T01:42:12.874163+0000","last_undegraded":"2024-03-30T01:42:12.874163+0000","last_fullsized":"2024-03-30T01:42:12.874163+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815535+0000","last_change":"2024-03-30T01:42:22.815535+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815535+0000","last_undegraded":"2024-03-30T01:42:22.815535+0000","last_fullsized":"2024-03-30T01:42:22.815535+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:36.452658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874345+0000","last_change":"2024-03-30T01:42:09.831981+0000","last_active":"2024-03-30T01:42:12.874345+0000","last_peered":"2024-03-30T01:42:12.874345+0000","last_clean":"2024-03-30T01:42:12.874345+0000","last_became_active":"2024-03-30T01:42:09.831647+0000","last_became_peered":"2024-03-30T01:42:09.831647+0000","last_unstale":"2024-03-30T01:42:12.874345+0000","last_undegraded":"2024-03-30T01:42:12.874345+0000","last_fullsized":"2024-03-30T01:42:12.874345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:02.883477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877386+0000","last_change":"2024-03-30T01:42:10.289213+0000","last_active":"2024-03-30T01:42:12.877386+0000","last_peered":"2024-03-30T01:42:12.877386+0000","last_clean":"2024-03-30T01:42:12.877386+0000","last_became_active":"2024-03-30T01:42:10.288677+0000","last_became_peered":"2024-03-30T01:42:10.288677+0000","last_unstale":"2024-03-30T01:42:12.877386+0000","last_undegraded":"2024-03-30T01:42:12.877386+0000","last_fullsized":"2024-03-30T01:42:12.877386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814739+0000","last_change":"2024-03-30T01:42:22.814739+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814739+0000","last_undegraded":"2024-03-30T01:42:22.814739+0000","last_fullsized":"2024-03-30T01:42:22.814739+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:32.897671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877446+0000","last_change":"2024-03-30T01:42:09.831496+0000","last_active":"2024-03-30T01:42:12.877446+0000","last_peered":"2024-03-30T01:42:12.877446+0000","last_clean":"2024-03-30T01:42:12.877446+0000","last_became_active":"2024-03-30T01:42:09.831197+0000","last_became_peered":"2024-03-30T01:42:09.831197+0000","last_unstale":"2024-03-30T01:42:12.877446+0000","last_undegraded":"2024-03-30T01:42:12.877446+0000","last_fullsized":"2024-03-30T01:42:12.877446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877126+0000","last_change":"2024-03-30T01:42:10.287526+0000","last_active":"2024-03-30T01:42:12.877126+0000","last_peered":"2024-03-30T01:42:12.877126+0000","last_clean":"2024-03-30T01:42:12.877126+0000","last_became_active":"2024-03-30T01:42:10.287255+0000","last_became_peered":"2024-03-30T01:42:10.287255+0000","last_unstale":"2024-03-30T01:42:12.877126+0000","last_undegraded":"2024-03-30T01:42:12.877126+0000","last_fullsized":"2024-03-30T01:42:12.877126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812195+0000","last_change":"2024-03-30T01:42:22.812195+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812195+0000","last_undegraded":"2024-03-30T01:42:22.812195+0000","last_fullsized":"2024-03-30T01:42:22.812195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:35.849177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874220+0000","last_change":"2024-03-30T01:42:09.824809+0000","last_active":"2024-03-30T01:42:12.874220+0000","last_peered":"2024-03-30T01:42:12.874220+0000","last_clean":"2024-03-30T01:42:12.874220+0000","last_became_active":"2024-03-30T01:42:09.824493+0000","last_became_peered":"2024-03-30T01:42:09.824493+0000","last_unstale":"2024-03-30T01:42:12.874220+0000","last_undegraded":"2024-03-30T01:42:12.874220+0000","last_fullsized":"2024-03-30T01:42:12.874220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877000+0000","last_change":"2024-03-30T01:42:10.291979+0000","last_active":"2024-03-30T01:42:12.877000+0000","last_peered":"2024-03-30T01:42:12.877000+0000","last_clean":"2024-03-30T01:42:12.877000+0000","last_became_active":"2024-03-30T01:42:10.291708+0000","last_became_peered":"2024-03-30T01:42:10.291708+0000","last_unstale":"2024-03-30T01:42:12.877000+0000","last_undegraded":"2024-03-30T01:42:12.877000+0000","last_fullsized":"2024-03-30T01:42:12.877000+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809931+0000","last_change":"2024-03-30T01:42:22.809931+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809931+0000","last_undegraded":"2024-03-30T01:42:22.809931+0000","last_fullsized":"2024-03-30T01:42:22.809931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877310+0000","last_change":"2024-03-30T01:42:09.829349+0000","last_active":"2024-03-30T01:42:12.877310+0000","last_peered":"2024-03-30T01:42:12.877310+0000","last_clean":"2024-03-30T01:42:12.877310+0000","last_became_active":"2024-03-30T01:42:09.828931+0000","last_became_peered":"2024-03-30T01:42:09.828931+0000","last_unstale":"2024-03-30T01:42:12.877310+0000","last_undegraded":"2024-03-30T01:42:12.877310+0000","last_fullsized":"2024-03-30T01:42:12.877310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874233+0000","last_change":"2024-03-30T01:42:10.300292+0000","last_active":"2024-03-30T01:42:12.874233+0000","last_peered":"2024-03-30T01:42:12.874233+0000","last_clean":"2024-03-30T01:42:12.874233+0000","last_became_active":"2024-03-30T01:42:10.299867+0000","last_became_peered":"2024-03-30T01:42:10.299867+0000","last_unstale":"2024-03-30T01:42:12.874233+0000","last_undegraded":"2024-03-30T01:42:12.874233+0000","last_fullsized":"2024-03-30T01:42:12.874233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812740+0000","last_change":"2024-03-30T01:42:22.812740+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812740+0000","last_undegraded":"2024-03-30T01:42:22.812740+0000","last_fullsized":"2024-03-30T01:42:22.812740+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:13.222722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877289+0000","last_change":"2024-03-30T01:42:09.829899+0000","last_active":"2024-03-30T01:42:12.877289+0000","last_peered":"2024-03-30T01:42:12.877289+0000","last_clean":"2024-03-30T01:42:12.877289+0000","last_became_active":"2024-03-30T01:42:09.829601+0000","last_became_peered":"2024-03-30T01:42:09.829601+0000","last_unstale":"2024-03-30T01:42:12.877289+0000","last_undegraded":"2024-03-30T01:42:12.877289+0000","last_fullsized":"2024-03-30T01:42:12.877289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:30.887459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877284+0000","last_change":"2024-03-30T01:42:10.298381+0000","last_active":"2024-03-30T01:42:12.877284+0000","last_peered":"2024-03-30T01:42:12.877284+0000","last_clean":"2024-03-30T01:42:12.877284+0000","last_became_active":"2024-03-30T01:42:10.298109+0000","last_became_peered":"2024-03-30T01:42:10.298109+0000","last_unstale":"2024-03-30T01:42:12.877284+0000","last_undegraded":"2024-03-30T01:42:12.877284+0000","last_fullsized":"2024-03-30T01:42:12.877284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.804764+0000","last_change":"2024-03-30T01:42:22.804764+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.804764+0000","last_undegraded":"2024-03-30T01:42:22.804764+0000","last_fullsized":"2024-03-30T01:42:22.804764+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881147+0000","last_change":"2024-03-30T01:42:09.824850+0000","last_active":"2024-03-30T01:42:12.881147+0000","last_peered":"2024-03-30T01:42:12.881147+0000","last_clean":"2024-03-30T01:42:12.881147+0000","last_became_active":"2024-03-30T01:42:09.824537+0000","last_became_peered":"2024-03-30T01:42:09.824537+0000","last_unstale":"2024-03-30T01:42:12.881147+0000","last_undegraded":"2024-03-30T01:42:12.881147+0000","last_fullsized":"2024-03-30T01:42:12.881147+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:17.759503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874391+0000","last_change":"2024-03-30T01:42:10.285919+0000","last_active":"2024-03-30T01:42:12.874391+0000","last_peered":"2024-03-30T01:42:12.874391+0000","last_clean":"2024-03-30T01:42:12.874391+0000","last_became_active":"2024-03-30T01:42:10.285629+0000","last_became_peered":"2024-03-30T01:42:10.285629+0000","last_unstale":"2024-03-30T01:42:12.874391+0000","last_undegraded":"2024-03-30T01:42:12.874391+0000","last_fullsized":"2024-03-30T01:42:12.874391+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809889+0000","last_change":"2024-03-30T01:42:22.809889+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809889+0000","last_undegraded":"2024-03-30T01:42:22.809889+0000","last_fullsized":"2024-03-30T01:42:22.809889+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:35.672193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877110+0000","last_change":"2024-03-30T01:42:10.297189+0000","last_active":"2024-03-30T01:42:12.877110+0000","last_peered":"2024-03-30T01:42:12.877110+0000","last_clean":"2024-03-30T01:42:12.877110+0000","last_became_active":"2024-03-30T01:42:10.296821+0000","last_became_peered":"2024-03-30T01:42:10.296821+0000","last_unstale":"2024-03-30T01:42:12.877110+0000","last_undegraded":"2024-03-30T01:42:12.877110+0000","last_fullsized":"2024-03-30T01:42:12.877110+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874246+0000","last_change":"2024-03-30T01:42:10.274200+0000","last_active":"2024-03-30T01:42:12.874246+0000","last_peered":"2024-03-30T01:42:12.874246+0000","last_clean":"2024-03-30T01:42:12.874246+0000","last_became_active":"2024-03-30T01:42:10.273923+0000","last_became_peered":"2024-03-30T01:42:10.273923+0000","last_unstale":"2024-03-30T01:42:12.874246+0000","last_undegraded":"2024-03-30T01:42:12.874246+0000","last_fullsized":"2024-03-30T01:42:12.874246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:44.766475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.880992+0000","last_change":"2024-03-30T01:42:10.290078+0000","last_active":"2024-03-30T01:42:12.880992+0000","last_peered":"2024-03-30T01:42:12.880992+0000","last_clean":"2024-03-30T01:42:12.880992+0000","last_became_active":"2024-03-30T01:42:10.289127+0000","last_became_peered":"2024-03-30T01:42:10.289127+0000","last_unstale":"2024-03-30T01:42:12.880992+0000","last_undegraded":"2024-03-30T01:42:12.880992+0000","last_fullsized":"2024-03-30T01:42:12.880992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:36.303285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881034+0000","last_change":"2024-03-30T01:42:10.291099+0000","last_active":"2024-03-30T01:42:12.881034+0000","last_peered":"2024-03-30T01:42:12.881034+0000","last_clean":"2024-03-30T01:42:12.881034+0000","last_became_active":"2024-03-30T01:42:10.290802+0000","last_became_peered":"2024-03-30T01:42:10.290802+0000","last_unstale":"2024-03-30T01:42:12.881034+0000","last_undegraded":"2024-03-30T01:42:12.881034+0000","last_fullsized":"2024-03-30T01:42:12.881034+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:13.070677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877459+0000","last_change":"2024-03-30T01:42:10.288633+0000","last_active":"2024-03-30T01:42:12.877459+0000","last_peered":"2024-03-30T01:42:12.877459+0000","last_clean":"2024-03-30T01:42:12.877459+0000","last_became_active":"2024-03-30T01:42:10.288326+0000","last_became_peered":"2024-03-30T01:42:10.288326+0000","last_unstale":"2024-03-30T01:42:12.877459+0000","last_undegraded":"2024-03-30T01:42:12.877459+0000","last_fullsized":"2024-03-30T01:42:12.877459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881079+0000","last_change":"2024-03-30T01:42:10.274845+0000","last_active":"2024-03-30T01:42:12.881079+0000","last_peered":"2024-03-30T01:42:12.881079+0000","last_clean":"2024-03-30T01:42:12.881079+0000","last_became_active":"2024-03-30T01:42:10.274566+0000","last_became_peered":"2024-03-30T01:42:10.274566+0000","last_unstale":"2024-03-30T01:42:12.881079+0000","last_undegraded":"2024-03-30T01:42:12.881079+0000","last_fullsized":"2024-03-30T01:42:12.881079+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:32.801396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877322+0000","last_change":"2024-03-30T01:42:10.290378+0000","last_active":"2024-03-30T01:42:12.877322+0000","last_peered":"2024-03-30T01:42:12.877322+0000","last_clean":"2024-03-30T01:42:12.877322+0000","last_became_active":"2024-03-30T01:42:10.290022+0000","last_became_peered":"2024-03-30T01:42:10.290022+0000","last_unstale":"2024-03-30T01:42:12.877322+0000","last_undegraded":"2024-03-30T01:42:12.877322+0000","last_fullsized":"2024-03-30T01:42:12.877322+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881124+0000","last_change":"2024-03-30T01:42:10.285378+0000","last_active":"2024-03-30T01:42:12.881124+0000","last_peered":"2024-03-30T01:42:12.881124+0000","last_clean":"2024-03-30T01:42:12.881124+0000","last_became_active":"2024-03-30T01:42:10.285105+0000","last_became_peered":"2024-03-30T01:42:10.285105+0000","last_unstale":"2024-03-30T01:42:12.881124+0000","last_undegraded":"2024-03-30T01:42:12.881124+0000","last_fullsized":"2024-03-30T01:42:12.881124+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:52.824468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875599+0000","last_change":"2024-03-30T01:42:10.286887+0000","last_active":"2024-03-30T01:42:12.875599+0000","last_peered":"2024-03-30T01:42:12.875599+0000","last_clean":"2024-03-30T01:42:12.875599+0000","last_became_active":"2024-03-30T01:42:10.278107+0000","last_became_peered":"2024-03-30T01:42:10.278107+0000","last_unstale":"2024-03-30T01:42:12.875599+0000","last_undegraded":"2024-03-30T01:42:12.875599+0000","last_fullsized":"2024-03-30T01:42:12.875599+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874268+0000","last_change":"2024-03-30T01:42:10.281331+0000","last_active":"2024-03-30T01:42:12.874268+0000","last_peered":"2024-03-30T01:42:12.874268+0000","last_clean":"2024-03-30T01:42:12.874268+0000","last_became_active":"2024-03-30T01:42:10.281011+0000","last_became_peered":"2024-03-30T01:42:10.281011+0000","last_unstale":"2024-03-30T01:42:12.874268+0000","last_undegraded":"2024-03-30T01:42:12.874268+0000","last_fullsized":"2024-03-30T01:42:12.874268+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874152+0000","last_change":"2024-03-30T01:42:10.299194+0000","last_active":"2024-03-30T01:42:12.874152+0000","last_peered":"2024-03-30T01:42:12.874152+0000","last_clean":"2024-03-30T01:42:12.874152+0000","last_became_active":"2024-03-30T01:42:10.298900+0000","last_became_peered":"2024-03-30T01:42:10.298900+0000","last_unstale":"2024-03-30T01:42:12.874152+0000","last_undegraded":"2024-03-30T01:42:12.874152+0000","last_fullsized":"2024-03-30T01:42:12.874152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877431+0000","last_change":"2024-03-30T01:42:10.304038+0000","last_active":"2024-03-30T01:42:12.877431+0000","last_peered":"2024-03-30T01:42:12.877431+0000","last_clean":"2024-03-30T01:42:12.877431+0000","last_became_active":"2024-03-30T01:42:10.303747+0000","last_became_peered":"2024-03-30T01:42:10.303747+0000","last_unstale":"2024-03-30T01:42:12.877431+0000","last_undegraded":"2024-03-30T01:42:12.877431+0000","last_fullsized":"2024-03-30T01:42:12.877431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881058+0000","last_change":"2024-03-30T01:42:10.300160+0000","last_active":"2024-03-30T01:42:12.881058+0000","last_peered":"2024-03-30T01:42:12.881058+0000","last_clean":"2024-03-30T01:42:12.881058+0000","last_became_active":"2024-03-30T01:42:10.299728+0000","last_became_peered":"2024-03-30T01:42:10.299728+0000","last_unstale":"2024-03-30T01:42:12.881058+0000","last_undegraded":"2024-03-30T01:42:12.881058+0000","last_fullsized":"2024-03-30T01:42:12.881058+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875715+0000","last_change":"2024-03-30T01:42:10.292351+0000","last_active":"2024-03-30T01:42:12.875715+0000","last_peered":"2024-03-30T01:42:12.875715+0000","last_clean":"2024-03-30T01:42:12.875715+0000","last_became_active":"2024-03-30T01:42:10.292040+0000","last_became_peered":"2024-03-30T01:42:10.292040+0000","last_unstale":"2024-03-30T01:42:12.875715+0000","last_undegraded":"2024-03-30T01:42:12.875715+0000","last_fullsized":"2024-03-30T01:42:12.875715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:09.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881101+0000","last_change":"2024-03-30T01:42:10.300393+0000","last_active":"2024-03-30T01:42:12.881101+0000","last_peered":"2024-03-30T01:42:12.881101+0000","last_clean":"2024-03-30T01:42:12.881101+0000","last_became_active":"2024-03-30T01:42:10.299795+0000","last_became_peered":"2024-03-30T01:42:10.299795+0000","last_unstale":"2024-03-30T01:42:12.881101+0000","last_undegraded":"2024-03-30T01:42:12.881101+0000","last_fullsized":"2024-03-30T01:42:12.881101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:21.256043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874478+0000","last_change":"2024-03-30T01:42:10.285733+0000","last_active":"2024-03-30T01:42:12.874478+0000","last_peered":"2024-03-30T01:42:12.874478+0000","last_clean":"2024-03-30T01:42:12.874478+0000","last_became_active":"2024-03-30T01:42:10.285428+0000","last_became_peered":"2024-03-30T01:42:10.285428+0000","last_unstale":"2024-03-30T01:42:12.874478+0000","last_undegraded":"2024-03-30T01:42:12.874478+0000","last_fullsized":"2024-03-30T01:42:12.874478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874143+0000","last_change":"2024-03-30T01:42:10.284419+0000","last_active":"2024-03-30T01:42:12.874143+0000","last_peered":"2024-03-30T01:42:12.874143+0000","last_clean":"2024-03-30T01:42:12.874143+0000","last_became_active":"2024-03-30T01:42:10.284098+0000","last_became_peered":"2024-03-30T01:42:10.284098+0000","last_unstale":"2024-03-30T01:42:12.874143+0000","last_undegraded":"2024-03-30T01:42:12.874143+0000","last_fullsized":"2024-03-30T01:42:12.874143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877370+0000","last_change":"2024-03-30T01:42:10.293953+0000","last_active":"2024-03-30T01:42:12.877370+0000","last_peered":"2024-03-30T01:42:12.877370+0000","last_clean":"2024-03-30T01:42:12.877370+0000","last_became_active":"2024-03-30T01:42:10.293641+0000","last_became_peered":"2024-03-30T01:42:10.293641+0000","last_unstale":"2024-03-30T01:42:12.877370+0000","last_undegraded":"2024-03-30T01:42:12.877370+0000","last_fullsized":"2024-03-30T01:42:12.877370+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874206+0000","last_change":"2024-03-30T01:42:10.294667+0000","last_active":"2024-03-30T01:42:12.874206+0000","last_peered":"2024-03-30T01:42:12.874206+0000","last_clean":"2024-03-30T01:42:12.874206+0000","last_became_active":"2024-03-30T01:42:10.294330+0000","last_became_peered":"2024-03-30T01:42:10.294330+0000","last_unstale":"2024-03-30T01:42:12.874206+0000","last_undegraded":"2024-03-30T01:42:12.874206+0000","last_fullsized":"2024-03-30T01:42:12.874206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877421+0000","last_change":"2024-03-30T01:42:10.295808+0000","last_active":"2024-03-30T01:42:12.877421+0000","last_peered":"2024-03-30T01:42:12.877421+0000","last_clean":"2024-03-30T01:42:12.877421+0000","last_became_active":"2024-03-30T01:42:10.295460+0000","last_became_peered":"2024-03-30T01:42:10.295460+0000","last_unstale":"2024-03-30T01:42:12.877421+0000","last_undegraded":"2024-03-30T01:42:12.877421+0000","last_fullsized":"2024-03-30T01:42:12.877421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877196+0000","last_change":"2024-03-30T01:42:10.294103+0000","last_active":"2024-03-30T01:42:12.877196+0000","last_peered":"2024-03-30T01:42:12.877196+0000","last_clean":"2024-03-30T01:42:12.877196+0000","last_became_active":"2024-03-30T01:42:10.293833+0000","last_became_peered":"2024-03-30T01:42:10.293833+0000","last_unstale":"2024-03-30T01:42:12.877196+0000","last_undegraded":"2024-03-30T01:42:12.877196+0000","last_fullsized":"2024-03-30T01:42:12.877196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877485+0000","last_change":"2024-03-30T01:42:10.296160+0000","last_active":"2024-03-30T01:42:12.877485+0000","last_peered":"2024-03-30T01:42:12.877485+0000","last_clean":"2024-03-30T01:42:12.877485+0000","last_became_active":"2024-03-30T01:42:10.295867+0000","last_became_peered":"2024-03-30T01:42:10.295867+0000","last_unstale":"2024-03-30T01:42:12.877485+0000","last_undegraded":"2024-03-30T01:42:12.877485+0000","last_fullsized":"2024-03-30T01:42:12.877485+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877510+0000","last_change":"2024-03-30T01:42:10.295963+0000","last_active":"2024-03-30T01:42:12.877510+0000","last_peered":"2024-03-30T01:42:12.877510+0000","last_clean":"2024-03-30T01:42:12.877510+0000","last_became_active":"2024-03-30T01:42:10.295545+0000","last_became_peered":"2024-03-30T01:42:10.295545+0000","last_unstale":"2024-03-30T01:42:12.877510+0000","last_undegraded":"2024-03-30T01:42:12.877510+0000","last_fullsized":"2024-03-30T01:42:12.877510+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877312+0000","last_change":"2024-03-30T01:42:10.285717+0000","last_active":"2024-03-30T01:42:12.877312+0000","last_peered":"2024-03-30T01:42:12.877312+0000","last_clean":"2024-03-30T01:42:12.877312+0000","last_became_active":"2024-03-30T01:42:10.285437+0000","last_became_peered":"2024-03-30T01:42:10.285437+0000","last_unstale":"2024-03-30T01:42:12.877312+0000","last_undegraded":"2024-03-30T01:42:12.877312+0000","last_fullsized":"2024-03-30T01:42:12.877312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876706+0000","last_change":"2024-03-30T01:42:10.298731+0000","last_active":"2024-03-30T01:42:12.876706+0000","last_peered":"2024-03-30T01:42:12.876706+0000","last_clean":"2024-03-30T01:42:12.876706+0000","last_became_active":"2024-03-30T01:42:10.298525+0000","last_became_peered":"2024-03-30T01:42:10.298525+0000","last_unstale":"2024-03-30T01:42:12.876706+0000","last_undegraded":"2024-03-30T01:42:12.876706+0000","last_fullsized":"2024-03-30T01:42:12.876706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877398+0000","last_change":"2024-03-30T01:42:10.289068+0000","last_active":"2024-03-30T01:42:12.877398+0000","last_peered":"2024-03-30T01:42:12.877398+0000","last_clean":"2024-03-30T01:42:12.877398+0000","last_became_active":"2024-03-30T01:42:10.288282+0000","last_became_peered":"2024-03-30T01:42:10.288282+0000","last_unstale":"2024-03-30T01:42:12.877398+0000","last_undegraded":"2024-03-30T01:42:12.877398+0000","last_fullsized":"2024-03-30T01:42:12.877398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877189+0000","last_change":"2024-03-30T01:42:10.287001+0000","last_active":"2024-03-30T01:42:12.877189+0000","last_peered":"2024-03-30T01:42:12.877189+0000","last_clean":"2024-03-30T01:42:12.877189+0000","last_became_active":"2024-03-30T01:42:10.286700+0000","last_became_peered":"2024-03-30T01:42:10.286700+0000","last_unstale":"2024-03-30T01:42:12.877189+0000","last_undegraded":"2024-03-30T01:42:12.877189+0000","last_fullsized":"2024-03-30T01:42:12.877189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875664+0000","last_change":"2024-03-30T01:42:10.299868+0000","last_active":"2024-03-30T01:42:12.875664+0000","last_peered":"2024-03-30T01:42:12.875664+0000","last_clean":"2024-03-30T01:42:12.875664+0000","last_became_active":"2024-03-30T01:42:10.299612+0000","last_became_peered":"2024-03-30T01:42:10.299612+0000","last_unstale":"2024-03-30T01:42:12.875664+0000","last_undegraded":"2024-03-30T01:42:12.875664+0000","last_fullsized":"2024-03-30T01:42:12.875664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:49.471045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875690+0000","last_change":"2024-03-30T01:42:10.294277+0000","last_active":"2024-03-30T01:42:12.875690+0000","last_peered":"2024-03-30T01:42:12.875690+0000","last_clean":"2024-03-30T01:42:12.875690+0000","last_became_active":"2024-03-30T01:42:10.293995+0000","last_became_peered":"2024-03-30T01:42:10.293995+0000","last_unstale":"2024-03-30T01:42:12.875690+0000","last_undegraded":"2024-03-30T01:42:12.875690+0000","last_fullsized":"2024-03-30T01:42:12.875690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:03.888060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877217+0000","last_change":"2024-03-30T01:42:10.285050+0000","last_active":"2024-03-30T01:42:12.877217+0000","last_peered":"2024-03-30T01:42:12.877217+0000","last_clean":"2024-03-30T01:42:12.877217+0000","last_became_active":"2024-03-30T01:42:10.284771+0000","last_became_peered":"2024-03-30T01:42:10.284771+0000","last_unstale":"2024-03-30T01:42:12.877217+0000","last_undegraded":"2024-03-30T01:42:12.877217+0000","last_fullsized":"2024-03-30T01:42:12.877217+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876935+0000","last_change":"2024-03-30T01:42:10.294965+0000","last_active":"2024-03-30T01:42:12.876935+0000","last_peered":"2024-03-30T01:42:12.876935+0000","last_clean":"2024-03-30T01:42:12.876935+0000","last_became_active":"2024-03-30T01:42:10.294685+0000","last_became_peered":"2024-03-30T01:42:10.294685+0000","last_unstale":"2024-03-30T01:42:12.876935+0000","last_undegraded":"2024-03-30T01:42:12.876935+0000","last_fullsized":"2024-03-30T01:42:12.876935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876977+0000","last_change":"2024-03-30T01:42:10.288258+0000","last_active":"2024-03-30T01:42:12.876977+0000","last_peered":"2024-03-30T01:42:12.876977+0000","last_clean":"2024-03-30T01:42:12.876977+0000","last_became_active":"2024-03-30T01:42:10.287993+0000","last_became_peered":"2024-03-30T01:42:10.287993+0000","last_unstale":"2024-03-30T01:42:12.876977+0000","last_undegraded":"2024-03-30T01:42:12.876977+0000","last_fullsized":"2024-03-30T01:42:12.876977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1060864,"data_stored":787628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.505},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.61199999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.56999999999999995}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.2009999999999996,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.41399999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.58299999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.47299999999999998}]},{"osd":1,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.3250000000000002,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.68300000000000005}]},{"osd":2,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.70399999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.79200000000000004}]},{"osd":7,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":15.038,"5min":15.038,"15min":15.038},"last":0.47499999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":11.858000000000001,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1289999999999996,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":11.99,"5min":11.99,"15min":11.99},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.57999999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":15.006,"5min":15.006,"15min":15.006},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":11.926,"5min":11.926,"15min":11.926},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":14.974,"5min":14.974,"15min":14.974},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.960000000000001,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":14.923999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.755}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":783548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.946999999999999,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.443}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":12.983000000000001,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.2279999999999998,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.3869999999999996,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":5.468,"5min":5.468,"15min":5.468},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":12.478999999999999,"5min":12.478999999999999,"15min":12.478999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.31},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":12.425000000000001,"5min":12.425000000000001,"15min":12.425000000000001},"last":0.33500000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":13.217000000000001,"5min":13.217000000000001,"15min":13.217000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":8.7400000000000002,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":8.5640000000000001,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":0.59099999999999997}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":8.7759999999999998,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":8.5190000000000001,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":8.5939999999999994,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":466944,"data_stored":197224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.378},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.45600000000000002}]},{"osd":1,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.441}]},{"osd":4,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.504}]},{"osd":5,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.53600000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.56899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.87}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":9.3970000000000002,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.747}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.84899999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":5.5289999999999999,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":9.452,"5min":9.452,"15min":9.452},"last":0.68200000000000005}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.62}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.79500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":6.6760000000000002,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.63200000000000001}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7480000000000002,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.7709999999999999,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.8410000000000002,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.37}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41199999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.442},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:42:28.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:42:28.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.687+0000 7f652d3af700 1 Processor -- start 2024-03-30T01:42:28.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.687+0000 7f652d3af700 1 -- start start 2024-03-30T01:42:28.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.687+0000 7f652d3af700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65281584b0 0x7f65281519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.687+0000 7f652d3af700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652806b300 con 0x7f65280c5710 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.687+0000 7f652d3af700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65280c6670 con 0x7f65280c72b0 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.687+0000 7f652d3af700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652806b550 con 0x7f65281584b0 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f65267fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f65280c5710 0x7f6528157da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47390/0 (socket says 172.21.15.31:47390) 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f65267fc700 1 -- 172.21.15.31:0/3959473999 learned_addr learned my addr 172.21.15.31:0/3959473999 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f65277fe700 1 --2- 172.21.15.31:0/3959473999 >> v2:172.21.15.59:3300/0 conn(0x7f65281584b0 0x7f65281519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336402521 0 0) 0x7f652806b300 con 0x7f65280c5710 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65100035b0 con 0x7f65280c5710 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892365684 0 0) 0x7f65280c6670 con 0x7f65280c72b0 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f652806b300 con 0x7f65280c72b0 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3840889047 0 0) 0x7f65100035b0 con 0x7f65280c5710 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 >> v2:172.21.15.59:3300/0 conn(0x7f65281584b0 msgr2=0x7f65281519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:28.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 --2- 172.21.15.31:0/3959473999 >> v2:172.21.15.59:3300/0 conn(0x7f65281584b0 0x7f65281519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 >> v1:172.21.15.59:6789/0 conn(0x7f65280c72b0 legacy=0x7f65280c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652806b830 con 0x7f65280c5710 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f651c001670 con 0x7f65280c5710 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 >> v1:172.21.15.31:6789/0 conn(0x7f65280c5710 legacy=0x7f6528157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6510003b90 0x7f6510003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6510004850 0x7f6510004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6510005210 0x7f651000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652806b550 con 0x7f6510003b90 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651000c140 con 0x7f6510005210 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.688+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651000c300 con 0x7f6510004850 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6526ffd700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6510003b90 0x7f6510003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f65277fe700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6510005210 0x7f651000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f65267fc700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6510004850 0x7f6510004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6526ffd700 1 -- 172.21.15.31:0/3959473999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6510004850 msgr2=0x7f6510004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6526ffd700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6510004850 0x7f6510004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6526ffd700 1 -- 172.21.15.31:0/3959473999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6510005210 msgr2=0x7f651000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6526ffd700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6510005210 0x7f651000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f65277fe700 1 -- 172.21.15.31:0/3959473999 reap_dead start 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6526ffd700 1 -- 172.21.15.31:0/3959473999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65100035b0 con 0x7f6510003b90 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f65267fc700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6510004850 0x7f6510004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6526ffd700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6510003b90 0x7f6510003f90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6518001940 tx=0x7f6510005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80503494a2197f96 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518001430 con 0x7f6510003b90 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6518001d70 con 0x7f6510003b90 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.689+0000 7f6525ffb700 1 -- 172.21.15.31:0/3959473999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518005530 con 0x7f6510003b90 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.690+0000 7f652d3af700 1 -- 172.21.15.31:0/3959473999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6510003b90 msgr2=0x7f6510003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.690+0000 7f652d3af700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6510003b90 0x7f6510003f90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6518001940 tx=0x7f6510005880 comp rx=0 tx=0).stop 2024-03-30T01:42:28.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.690+0000 7f652d3af700 1 -- 172.21.15.31:0/3959473999 shutdown_connections 2024-03-30T01:42:28.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.690+0000 7f652d3af700 1 --2- 172.21.15.31:0/3959473999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6510003b90 0x7f6510003f90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.690+0000 7f652d3af700 1 -- 172.21.15.31:0/3959473999 >> 172.21.15.31:0/3959473999 conn(0x7f65280abf60 msgr2=0x7f65280ac370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:28.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.690+0000 7f652d3af700 1 -- 172.21.15.31:0/3959473999 shutdown_connections 2024-03-30T01:42:28.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.690+0000 7f652d3af700 1 -- 172.21.15.31:0/3959473999 wait complete. 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.691+0000 7f652d3af700 1 Processor -- start 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.691+0000 7f652d3af700 1 -- start start 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f652d3af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6510003b90 0x7f652814fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f652d3af700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65281dd650 0x7f652814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f652d3af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652814c9b0 0x7f65281eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f652d3af700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65281e0550 con 0x7f652814c9b0 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f652d3af700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65280c3fa0 con 0x7f6510003b90 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f652d3af700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652814ce40 con 0x7f65281dd650 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f65277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652814c9b0 0x7f65281eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f65277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652814c9b0 0x7f65281eadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44672/0 (socket says 172.21.15.31:44672) 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f65277fe700 1 -- 172.21.15.31:0/2560024852 learned_addr learned my addr 172.21.15.31:0/2560024852 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f6526ffd700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6510003b90 0x7f652814fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.692+0000 7f65267fc700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65281dd650 0x7f652814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f65277fe700 1 -- 172.21.15.31:0/2560024852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65281dd650 msgr2=0x7f652814c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f65277fe700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65281dd650 0x7f652814c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f65277fe700 1 -- 172.21.15.31:0/2560024852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6510003b90 msgr2=0x7f652814fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f65277fe700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6510003b90 0x7f652814fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f65277fe700 1 -- 172.21.15.31:0/2560024852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652806c1b0 con 0x7f652814c9b0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f65277fe700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652814c9b0 0x7f65281eadb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f65280c4ab0 tx=0x7f6520011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f650f7fe700 1 -- 172.21.15.31:0/2560024852 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000e040 con 0x7f652814c9b0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f650f7fe700 1 -- 172.21.15.31:0/2560024852 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652000f040 con 0x7f652814c9b0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f650f7fe700 1 -- 172.21.15.31:0/2560024852 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000fc10 con 0x7f652814c9b0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652814d050 con 0x7f652814c9b0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.693+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6528150360 con 0x7f652814c9b0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.694+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65281ebac0 con 0x7f652814c9b0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.694+0000 7f650f7fe700 1 -- 172.21.15.31:0/2560024852 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f652000fd70 con 0x7f652814c9b0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.695+0000 7f650f7fe700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651406ac90 0x7f651406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.695+0000 7f650f7fe700 1 -- 172.21.15.31:0/2560024852 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f6520089fe0 con 0x7f652814c9b0 2024-03-30T01:42:28.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.695+0000 7f6526ffd700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651406ac90 0x7f651406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:28.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.696+0000 7f6526ffd700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651406ac90 0x7f651406d150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6518000c00 tx=0x7f651800a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:28.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.700+0000 7f650f7fe700 1 -- 172.21.15.31:0/2560024852 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f65281ebac0 con 0x7f652814c9b0 2024-03-30T01:42:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.910+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f652806c030 con 0x7f651406ac90 2024-03-30T01:42:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.924+0000 7f650f7fe700 1 -- 172.21.15.31:0/2560024852 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+516228 (secure 0 0 0) 0x7f6518022070 con 0x7f651406ac90 2024-03-30T01:42:28.937 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:28.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651406ac90 msgr2=0x7f651406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651406ac90 0x7f651406d150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6518000c00 tx=0x7f651800a000 comp rx=0 tx=0).stop 2024-03-30T01:42:28.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652814c9b0 msgr2=0x7f65281eadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:28.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652814c9b0 0x7f65281eadb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f65280c4ab0 tx=0x7f6520011ed0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 shutdown_connections 2024-03-30T01:42:28.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f652814c9b0 0x7f65281eadb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f651406ac90 0x7f651406d150 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6510003b90 0x7f652814fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 --2- 172.21.15.31:0/2560024852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f65281dd650 0x7f652814c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 >> 172.21.15.31:0/2560024852 conn(0x7f65280abf60 msgr2=0x7f65280c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 shutdown_connections 2024-03-30T01:42:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:28.929+0000 7f652d3af700 1 -- 172.21.15.31:0/2560024852 wait complete. 2024-03-30T01:42:28.943 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:42:28.949 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-03-30T01:42:27.827998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594241,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220932,"kb_used_data":4740,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753788,"statfs":{"total":773094113280,"available":772867878912,"internally_reserved":0,"allocated":4853760,"data_stored":2740433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.023745"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877685+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:42:11.877685+0000","last_peered":"2024-03-30T01:42:11.877685+0000","last_clean":"2024-03-30T01:42:11.877685+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:42:11.877685+0000","last_undegraded":"2024-03-30T01:42:11.877685+0000","last_fullsized":"2024-03-30T01:42:11.877685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874412+0000","last_change":"2024-03-30T01:42:09.836532+0000","last_active":"2024-03-30T01:42:12.874412+0000","last_peered":"2024-03-30T01:42:12.874412+0000","last_clean":"2024-03-30T01:42:12.874412+0000","last_became_active":"2024-03-30T01:42:09.836262+0000","last_became_peered":"2024-03-30T01:42:09.836262+0000","last_unstale":"2024-03-30T01:42:12.874412+0000","last_undegraded":"2024-03-30T01:42:12.874412+0000","last_fullsized":"2024-03-30T01:42:12.874412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:08.249431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877538+0000","last_change":"2024-03-30T01:42:10.289721+0000","last_active":"2024-03-30T01:42:12.877538+0000","last_peered":"2024-03-30T01:42:12.877538+0000","last_clean":"2024-03-30T01:42:12.877538+0000","last_became_active":"2024-03-30T01:42:10.289413+0000","last_became_peered":"2024-03-30T01:42:10.289413+0000","last_unstale":"2024-03-30T01:42:12.877538+0000","last_undegraded":"2024-03-30T01:42:12.877538+0000","last_fullsized":"2024-03-30T01:42:12.877538+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814741+0000","last_change":"2024-03-30T01:42:22.814741+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814741+0000","last_undegraded":"2024-03-30T01:42:22.814741+0000","last_fullsized":"2024-03-30T01:42:22.814741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:34.573756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876734+0000","last_change":"2024-03-30T01:42:09.829174+0000","last_active":"2024-03-30T01:42:12.876734+0000","last_peered":"2024-03-30T01:42:12.876734+0000","last_clean":"2024-03-30T01:42:12.876734+0000","last_became_active":"2024-03-30T01:42:09.828730+0000","last_became_peered":"2024-03-30T01:42:09.828730+0000","last_unstale":"2024-03-30T01:42:12.876734+0000","last_undegraded":"2024-03-30T01:42:12.876734+0000","last_fullsized":"2024-03-30T01:42:12.876734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874371+0000","last_change":"2024-03-30T01:42:10.284566+0000","last_active":"2024-03-30T01:42:12.874371+0000","last_peered":"2024-03-30T01:42:12.874371+0000","last_clean":"2024-03-30T01:42:12.874371+0000","last_became_active":"2024-03-30T01:42:10.284192+0000","last_became_peered":"2024-03-30T01:42:10.284192+0000","last_unstale":"2024-03-30T01:42:12.874371+0000","last_undegraded":"2024-03-30T01:42:12.874371+0000","last_fullsized":"2024-03-30T01:42:12.874371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:05.460383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809509+0000","last_change":"2024-03-30T01:42:22.809509+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809509+0000","last_undegraded":"2024-03-30T01:42:22.809509+0000","last_fullsized":"2024-03-30T01:42:22.809509+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:56.846961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874138+0000","last_change":"2024-03-30T01:42:09.832991+0000","last_active":"2024-03-30T01:42:12.874138+0000","last_peered":"2024-03-30T01:42:12.874138+0000","last_clean":"2024-03-30T01:42:12.874138+0000","last_became_active":"2024-03-30T01:42:09.832682+0000","last_became_peered":"2024-03-30T01:42:09.832682+0000","last_unstale":"2024-03-30T01:42:12.874138+0000","last_undegraded":"2024-03-30T01:42:12.874138+0000","last_fullsized":"2024-03-30T01:42:12.874138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:37.427434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874189+0000","last_change":"2024-03-30T01:42:10.299170+0000","last_active":"2024-03-30T01:42:12.874189+0000","last_peered":"2024-03-30T01:42:12.874189+0000","last_clean":"2024-03-30T01:42:12.874189+0000","last_became_active":"2024-03-30T01:42:10.298846+0000","last_became_peered":"2024-03-30T01:42:10.298846+0000","last_unstale":"2024-03-30T01:42:12.874189+0000","last_undegraded":"2024-03-30T01:42:12.874189+0000","last_fullsized":"2024-03-30T01:42:12.874189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809736+0000","last_change":"2024-03-30T01:42:22.809736+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809736+0000","last_undegraded":"2024-03-30T01:42:22.809736+0000","last_fullsized":"2024-03-30T01:42:22.809736+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:43.792947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881014+0000","last_change":"2024-03-30T01:42:09.822074+0000","last_active":"2024-03-30T01:42:12.881014+0000","last_peered":"2024-03-30T01:42:12.881014+0000","last_clean":"2024-03-30T01:42:12.881014+0000","last_became_active":"2024-03-30T01:42:09.821786+0000","last_became_peered":"2024-03-30T01:42:09.821786+0000","last_unstale":"2024-03-30T01:42:12.881014+0000","last_undegraded":"2024-03-30T01:42:12.881014+0000","last_fullsized":"2024-03-30T01:42:12.881014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:18.840885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877139+0000","last_change":"2024-03-30T01:42:10.291029+0000","last_active":"2024-03-30T01:42:12.877139+0000","last_peered":"2024-03-30T01:42:12.877139+0000","last_clean":"2024-03-30T01:42:12.877139+0000","last_became_active":"2024-03-30T01:42:10.290728+0000","last_became_peered":"2024-03-30T01:42:10.290728+0000","last_unstale":"2024-03-30T01:42:12.877139+0000","last_undegraded":"2024-03-30T01:42:12.877139+0000","last_fullsized":"2024-03-30T01:42:12.877139+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.804768+0000","last_change":"2024-03-30T01:42:22.804768+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.804768+0000","last_undegraded":"2024-03-30T01:42:22.804768+0000","last_fullsized":"2024-03-30T01:42:22.804768+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877173+0000","last_change":"2024-03-30T01:42:09.827116+0000","last_active":"2024-03-30T01:42:12.877173+0000","last_peered":"2024-03-30T01:42:12.877173+0000","last_clean":"2024-03-30T01:42:12.877173+0000","last_became_active":"2024-03-30T01:42:09.826876+0000","last_became_peered":"2024-03-30T01:42:09.826876+0000","last_unstale":"2024-03-30T01:42:12.877173+0000","last_undegraded":"2024-03-30T01:42:12.877173+0000","last_fullsized":"2024-03-30T01:42:12.877173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:38.792762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877100+0000","last_change":"2024-03-30T01:42:10.288334+0000","last_active":"2024-03-30T01:42:12.877100+0000","last_peered":"2024-03-30T01:42:12.877100+0000","last_clean":"2024-03-30T01:42:12.877100+0000","last_became_active":"2024-03-30T01:42:10.288093+0000","last_became_peered":"2024-03-30T01:42:10.288093+0000","last_unstale":"2024-03-30T01:42:12.877100+0000","last_undegraded":"2024-03-30T01:42:12.877100+0000","last_fullsized":"2024-03-30T01:42:12.877100+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815309+0000","last_change":"2024-03-30T01:42:22.815309+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815309+0000","last_undegraded":"2024-03-30T01:42:22.815309+0000","last_fullsized":"2024-03-30T01:42:22.815309+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:44:11.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877240+0000","last_change":"2024-03-30T01:42:09.830490+0000","last_active":"2024-03-30T01:42:12.877240+0000","last_peered":"2024-03-30T01:42:12.877240+0000","last_clean":"2024-03-30T01:42:12.877240+0000","last_became_active":"2024-03-30T01:42:09.830197+0000","last_became_peered":"2024-03-30T01:42:09.830197+0000","last_unstale":"2024-03-30T01:42:12.877240+0000","last_undegraded":"2024-03-30T01:42:12.877240+0000","last_fullsized":"2024-03-30T01:42:12.877240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:46.433516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876955+0000","last_change":"2024-03-30T01:42:10.299986+0000","last_active":"2024-03-30T01:42:12.876955+0000","last_peered":"2024-03-30T01:42:12.876955+0000","last_clean":"2024-03-30T01:42:12.876955+0000","last_became_active":"2024-03-30T01:42:10.299776+0000","last_became_peered":"2024-03-30T01:42:10.299776+0000","last_unstale":"2024-03-30T01:42:12.876955+0000","last_undegraded":"2024-03-30T01:42:12.876955+0000","last_fullsized":"2024-03-30T01:42:12.876955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812125+0000","last_change":"2024-03-30T01:42:22.812125+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812125+0000","last_undegraded":"2024-03-30T01:42:22.812125+0000","last_fullsized":"2024-03-30T01:42:22.812125+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:58.845872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875741+0000","last_change":"2024-03-30T01:42:09.824373+0000","last_active":"2024-03-30T01:42:12.875741+0000","last_peered":"2024-03-30T01:42:12.875741+0000","last_clean":"2024-03-30T01:42:12.875741+0000","last_became_active":"2024-03-30T01:42:09.824089+0000","last_became_peered":"2024-03-30T01:42:09.824089+0000","last_unstale":"2024-03-30T01:42:12.875741+0000","last_undegraded":"2024-03-30T01:42:12.875741+0000","last_fullsized":"2024-03-30T01:42:12.875741+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:37.434399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874455+0000","last_change":"2024-03-30T01:42:10.284232+0000","last_active":"2024-03-30T01:42:12.874455+0000","last_peered":"2024-03-30T01:42:12.874455+0000","last_clean":"2024-03-30T01:42:12.874455+0000","last_became_active":"2024-03-30T01:42:10.283909+0000","last_became_peered":"2024-03-30T01:42:10.283909+0000","last_unstale":"2024-03-30T01:42:12.874455+0000","last_undegraded":"2024-03-30T01:42:12.874455+0000","last_fullsized":"2024-03-30T01:42:12.874455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:33.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815396+0000","last_change":"2024-03-30T01:42:22.815396+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815396+0000","last_undegraded":"2024-03-30T01:42:22.815396+0000","last_fullsized":"2024-03-30T01:42:22.815396+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:42:51.968187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877562+0000","last_change":"2024-03-30T01:42:09.833189+0000","last_active":"2024-03-30T01:42:12.877562+0000","last_peered":"2024-03-30T01:42:12.877562+0000","last_clean":"2024-03-30T01:42:12.877562+0000","last_became_active":"2024-03-30T01:42:09.832891+0000","last_became_peered":"2024-03-30T01:42:09.832891+0000","last_unstale":"2024-03-30T01:42:12.877562+0000","last_undegraded":"2024-03-30T01:42:12.877562+0000","last_fullsized":"2024-03-30T01:42:12.877562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877221+0000","last_change":"2024-03-30T01:42:10.298216+0000","last_active":"2024-03-30T01:42:12.877221+0000","last_peered":"2024-03-30T01:42:12.877221+0000","last_clean":"2024-03-30T01:42:12.877221+0000","last_became_active":"2024-03-30T01:42:10.297976+0000","last_became_peered":"2024-03-30T01:42:10.297976+0000","last_unstale":"2024-03-30T01:42:12.877221+0000","last_undegraded":"2024-03-30T01:42:12.877221+0000","last_fullsized":"2024-03-30T01:42:12.877221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.807923+0000","last_change":"2024-03-30T01:42:22.807923+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.807923+0000","last_undegraded":"2024-03-30T01:42:22.807923+0000","last_fullsized":"2024-03-30T01:42:22.807923+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875636+0000","last_change":"2024-03-30T01:42:09.829638+0000","last_active":"2024-03-30T01:42:12.875636+0000","last_peered":"2024-03-30T01:42:12.875636+0000","last_clean":"2024-03-30T01:42:12.875636+0000","last_became_active":"2024-03-30T01:42:09.829317+0000","last_became_peered":"2024-03-30T01:42:09.829317+0000","last_unstale":"2024-03-30T01:42:12.875636+0000","last_undegraded":"2024-03-30T01:42:12.875636+0000","last_fullsized":"2024-03-30T01:42:12.875636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:33.639097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876807+0000","last_change":"2024-03-30T01:42:10.292043+0000","last_active":"2024-03-30T01:42:12.876807+0000","last_peered":"2024-03-30T01:42:12.876807+0000","last_clean":"2024-03-30T01:42:12.876807+0000","last_became_active":"2024-03-30T01:42:10.291598+0000","last_became_peered":"2024-03-30T01:42:10.291598+0000","last_unstale":"2024-03-30T01:42:12.876807+0000","last_undegraded":"2024-03-30T01:42:12.876807+0000","last_fullsized":"2024-03-30T01:42:12.876807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.810206+0000","last_change":"2024-03-30T01:42:22.810206+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.810206+0000","last_undegraded":"2024-03-30T01:42:22.810206+0000","last_fullsized":"2024-03-30T01:42:22.810206+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:57.966402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874291+0000","last_change":"2024-03-30T01:42:09.829777+0000","last_active":"2024-03-30T01:42:12.874291+0000","last_peered":"2024-03-30T01:42:12.874291+0000","last_clean":"2024-03-30T01:42:12.874291+0000","last_became_active":"2024-03-30T01:42:09.829483+0000","last_became_peered":"2024-03-30T01:42:09.829483+0000","last_unstale":"2024-03-30T01:42:12.874291+0000","last_undegraded":"2024-03-30T01:42:12.874291+0000","last_fullsized":"2024-03-30T01:42:12.874291+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:45.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877410+0000","last_change":"2024-03-30T01:42:10.285649+0000","last_active":"2024-03-30T01:42:12.877410+0000","last_peered":"2024-03-30T01:42:12.877410+0000","last_clean":"2024-03-30T01:42:12.877410+0000","last_became_active":"2024-03-30T01:42:10.285297+0000","last_became_peered":"2024-03-30T01:42:10.285297+0000","last_unstale":"2024-03-30T01:42:12.877410+0000","last_undegraded":"2024-03-30T01:42:12.877410+0000","last_fullsized":"2024-03-30T01:42:12.877410+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809986+0000","last_change":"2024-03-30T01:42:22.809986+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809986+0000","last_undegraded":"2024-03-30T01:42:22.809986+0000","last_fullsized":"2024-03-30T01:42:22.809986+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:19:37.521361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875652+0000","last_change":"2024-03-30T01:42:09.826256+0000","last_active":"2024-03-30T01:42:12.875652+0000","last_peered":"2024-03-30T01:42:12.875652+0000","last_clean":"2024-03-30T01:42:12.875652+0000","last_became_active":"2024-03-30T01:42:09.825956+0000","last_became_peered":"2024-03-30T01:42:09.825956+0000","last_unstale":"2024-03-30T01:42:12.875652+0000","last_undegraded":"2024-03-30T01:42:12.875652+0000","last_fullsized":"2024-03-30T01:42:12.875652+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:59:41.446927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877149+0000","last_change":"2024-03-30T01:42:10.290469+0000","last_active":"2024-03-30T01:42:12.877149+0000","last_peered":"2024-03-30T01:42:12.877149+0000","last_clean":"2024-03-30T01:42:12.877149+0000","last_became_active":"2024-03-30T01:42:10.290148+0000","last_became_peered":"2024-03-30T01:42:10.290148+0000","last_unstale":"2024-03-30T01:42:12.877149+0000","last_undegraded":"2024-03-30T01:42:12.877149+0000","last_fullsized":"2024-03-30T01:42:12.877149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.805285+0000","last_change":"2024-03-30T01:42:22.805285+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.805285+0000","last_undegraded":"2024-03-30T01:42:22.805285+0000","last_fullsized":"2024-03-30T01:42:22.805285+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:54.923310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875675+0000","last_change":"2024-03-30T01:42:09.830452+0000","last_active":"2024-03-30T01:42:12.875675+0000","last_peered":"2024-03-30T01:42:12.875675+0000","last_clean":"2024-03-30T01:42:12.875675+0000","last_became_active":"2024-03-30T01:42:09.830173+0000","last_became_peered":"2024-03-30T01:42:09.830173+0000","last_unstale":"2024-03-30T01:42:12.875675+0000","last_undegraded":"2024-03-30T01:42:12.875675+0000","last_fullsized":"2024-03-30T01:42:12.875675+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:12:42.807604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874172+0000","last_change":"2024-03-30T01:42:10.279913+0000","last_active":"2024-03-30T01:42:12.874172+0000","last_peered":"2024-03-30T01:42:12.874172+0000","last_clean":"2024-03-30T01:42:12.874172+0000","last_became_active":"2024-03-30T01:42:10.279450+0000","last_became_peered":"2024-03-30T01:42:10.279450+0000","last_unstale":"2024-03-30T01:42:12.874172+0000","last_undegraded":"2024-03-30T01:42:12.874172+0000","last_fullsized":"2024-03-30T01:42:12.874172+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814722+0000","last_change":"2024-03-30T01:42:22.814722+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814722+0000","last_undegraded":"2024-03-30T01:42:22.814722+0000","last_fullsized":"2024-03-30T01:42:22.814722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:44.454854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877240+0000","last_change":"2024-03-30T01:42:09.828572+0000","last_active":"2024-03-30T01:42:12.877240+0000","last_peered":"2024-03-30T01:42:12.877240+0000","last_clean":"2024-03-30T01:42:12.877240+0000","last_became_active":"2024-03-30T01:42:09.828225+0000","last_became_peered":"2024-03-30T01:42:09.828225+0000","last_unstale":"2024-03-30T01:42:12.877240+0000","last_undegraded":"2024-03-30T01:42:12.877240+0000","last_fullsized":"2024-03-30T01:42:12.877240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875755+0000","last_change":"2024-03-30T01:42:10.286900+0000","last_active":"2024-03-30T01:42:12.875755+0000","last_peered":"2024-03-30T01:42:12.875755+0000","last_clean":"2024-03-30T01:42:12.875755+0000","last_became_active":"2024-03-30T01:42:10.286598+0000","last_became_peered":"2024-03-30T01:42:10.286598+0000","last_unstale":"2024-03-30T01:42:12.875755+0000","last_undegraded":"2024-03-30T01:42:12.875755+0000","last_fullsized":"2024-03-30T01:42:12.875755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:46.592007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.805879+0000","last_change":"2024-03-30T01:42:22.805879+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.805879+0000","last_undegraded":"2024-03-30T01:42:22.805879+0000","last_fullsized":"2024-03-30T01:42:22.805879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:57.756956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875729+0000","last_change":"2024-03-30T01:42:09.823221+0000","last_active":"2024-03-30T01:42:12.875729+0000","last_peered":"2024-03-30T01:42:12.875729+0000","last_clean":"2024-03-30T01:42:12.875729+0000","last_became_active":"2024-03-30T01:42:09.822882+0000","last_became_peered":"2024-03-30T01:42:09.822882+0000","last_unstale":"2024-03-30T01:42:12.875729+0000","last_undegraded":"2024-03-30T01:42:12.875729+0000","last_fullsized":"2024-03-30T01:42:12.875729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:00:31.185685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874279+0000","last_change":"2024-03-30T01:42:10.304085+0000","last_active":"2024-03-30T01:42:12.874279+0000","last_peered":"2024-03-30T01:42:12.874279+0000","last_clean":"2024-03-30T01:42:12.874279+0000","last_became_active":"2024-03-30T01:42:10.303817+0000","last_became_peered":"2024-03-30T01:42:10.303817+0000","last_unstale":"2024-03-30T01:42:12.874279+0000","last_undegraded":"2024-03-30T01:42:12.874279+0000","last_fullsized":"2024-03-30T01:42:12.874279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814798+0000","last_change":"2024-03-30T01:42:22.814798+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814798+0000","last_undegraded":"2024-03-30T01:42:22.814798+0000","last_fullsized":"2024-03-30T01:42:22.814798+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:06.958915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874246+0000","last_change":"2024-03-30T01:42:09.829713+0000","last_active":"2024-03-30T01:42:12.874246+0000","last_peered":"2024-03-30T01:42:12.874246+0000","last_clean":"2024-03-30T01:42:12.874246+0000","last_became_active":"2024-03-30T01:42:09.829375+0000","last_became_peered":"2024-03-30T01:42:09.829375+0000","last_unstale":"2024-03-30T01:42:12.874246+0000","last_undegraded":"2024-03-30T01:42:12.874246+0000","last_fullsized":"2024-03-30T01:42:12.874246+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:49:51.601663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877472+0000","last_change":"2024-03-30T01:42:10.295516+0000","last_active":"2024-03-30T01:42:12.877472+0000","last_peered":"2024-03-30T01:42:12.877472+0000","last_clean":"2024-03-30T01:42:12.877472+0000","last_became_active":"2024-03-30T01:42:10.295133+0000","last_became_peered":"2024-03-30T01:42:10.295133+0000","last_unstale":"2024-03-30T01:42:12.877472+0000","last_undegraded":"2024-03-30T01:42:12.877472+0000","last_fullsized":"2024-03-30T01:42:12.877472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812100+0000","last_change":"2024-03-30T01:42:22.812100+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812100+0000","last_undegraded":"2024-03-30T01:42:22.812100+0000","last_fullsized":"2024-03-30T01:42:22.812100+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:46.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877549+0000","last_change":"2024-03-30T01:42:09.826895+0000","last_active":"2024-03-30T01:42:12.877549+0000","last_peered":"2024-03-30T01:42:12.877549+0000","last_clean":"2024-03-30T01:42:12.877549+0000","last_became_active":"2024-03-30T01:42:09.826609+0000","last_became_peered":"2024-03-30T01:42:09.826609+0000","last_unstale":"2024-03-30T01:42:12.877549+0000","last_undegraded":"2024-03-30T01:42:12.877549+0000","last_fullsized":"2024-03-30T01:42:12.877549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:42:59.036073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876887+0000","last_change":"2024-03-30T01:42:10.291564+0000","last_active":"2024-03-30T01:42:12.876887+0000","last_peered":"2024-03-30T01:42:12.876887+0000","last_clean":"2024-03-30T01:42:12.876887+0000","last_became_active":"2024-03-30T01:42:10.291285+0000","last_became_peered":"2024-03-30T01:42:10.291285+0000","last_unstale":"2024-03-30T01:42:12.876887+0000","last_undegraded":"2024-03-30T01:42:12.876887+0000","last_fullsized":"2024-03-30T01:42:12.876887+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812173+0000","last_change":"2024-03-30T01:42:22.812173+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812173+0000","last_undegraded":"2024-03-30T01:42:22.812173+0000","last_fullsized":"2024-03-30T01:42:22.812173+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:49.116227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874199+0000","last_change":"2024-03-30T01:42:09.831869+0000","last_active":"2024-03-30T01:42:12.874199+0000","last_peered":"2024-03-30T01:42:12.874199+0000","last_clean":"2024-03-30T01:42:12.874199+0000","last_became_active":"2024-03-30T01:42:09.831562+0000","last_became_peered":"2024-03-30T01:42:09.831562+0000","last_unstale":"2024-03-30T01:42:12.874199+0000","last_undegraded":"2024-03-30T01:42:12.874199+0000","last_fullsized":"2024-03-30T01:42:12.874199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:35.938001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876785+0000","last_change":"2024-03-30T01:42:10.288399+0000","last_active":"2024-03-30T01:42:12.876785+0000","last_peered":"2024-03-30T01:42:12.876785+0000","last_clean":"2024-03-30T01:42:12.876785+0000","last_became_active":"2024-03-30T01:42:10.287890+0000","last_became_peered":"2024-03-30T01:42:10.287890+0000","last_unstale":"2024-03-30T01:42:12.876785+0000","last_undegraded":"2024-03-30T01:42:12.876785+0000","last_fullsized":"2024-03-30T01:42:12.876785+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812225+0000","last_change":"2024-03-30T01:42:22.812225+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812225+0000","last_undegraded":"2024-03-30T01:42:22.812225+0000","last_fullsized":"2024-03-30T01:42:22.812225+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:59.368657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.880961+0000","last_change":"2024-03-30T01:42:09.820451+0000","last_active":"2024-03-30T01:42:12.880961+0000","last_peered":"2024-03-30T01:42:12.880961+0000","last_clean":"2024-03-30T01:42:12.880961+0000","last_became_active":"2024-03-30T01:42:09.820152+0000","last_became_peered":"2024-03-30T01:42:09.820152+0000","last_unstale":"2024-03-30T01:42:12.880961+0000","last_undegraded":"2024-03-30T01:42:12.880961+0000","last_fullsized":"2024-03-30T01:42:12.880961+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:04.153303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874321+0000","last_change":"2024-03-30T01:42:10.279681+0000","last_active":"2024-03-30T01:42:12.874321+0000","last_peered":"2024-03-30T01:42:12.874321+0000","last_clean":"2024-03-30T01:42:12.874321+0000","last_became_active":"2024-03-30T01:42:10.279333+0000","last_became_peered":"2024-03-30T01:42:10.279333+0000","last_unstale":"2024-03-30T01:42:12.874321+0000","last_undegraded":"2024-03-30T01:42:12.874321+0000","last_fullsized":"2024-03-30T01:42:12.874321+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814765+0000","last_change":"2024-03-30T01:42:22.814765+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814765+0000","last_undegraded":"2024-03-30T01:42:22.814765+0000","last_fullsized":"2024-03-30T01:42:22.814765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.379389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877207+0000","last_change":"2024-03-30T01:42:09.829194+0000","last_active":"2024-03-30T01:42:12.877207+0000","last_peered":"2024-03-30T01:42:12.877207+0000","last_clean":"2024-03-30T01:42:12.877207+0000","last_became_active":"2024-03-30T01:42:09.828257+0000","last_became_peered":"2024-03-30T01:42:09.828257+0000","last_unstale":"2024-03-30T01:42:12.877207+0000","last_undegraded":"2024-03-30T01:42:12.877207+0000","last_fullsized":"2024-03-30T01:42:12.877207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876760+0000","last_change":"2024-03-30T01:42:10.288326+0000","last_active":"2024-03-30T01:42:12.876760+0000","last_peered":"2024-03-30T01:42:12.876760+0000","last_clean":"2024-03-30T01:42:12.876760+0000","last_became_active":"2024-03-30T01:42:10.288050+0000","last_became_peered":"2024-03-30T01:42:10.288050+0000","last_unstale":"2024-03-30T01:42:12.876760+0000","last_undegraded":"2024-03-30T01:42:12.876760+0000","last_fullsized":"2024-03-30T01:42:12.876760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809500+0000","last_change":"2024-03-30T01:42:22.809500+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809500+0000","last_undegraded":"2024-03-30T01:42:22.809500+0000","last_fullsized":"2024-03-30T01:42:22.809500+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:20.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876861+0000","last_change":"2024-03-30T01:42:09.832318+0000","last_active":"2024-03-30T01:42:12.876861+0000","last_peered":"2024-03-30T01:42:12.876861+0000","last_clean":"2024-03-30T01:42:12.876861+0000","last_became_active":"2024-03-30T01:42:09.831964+0000","last_became_peered":"2024-03-30T01:42:09.831964+0000","last_unstale":"2024-03-30T01:42:12.876861+0000","last_undegraded":"2024-03-30T01:42:12.876861+0000","last_fullsized":"2024-03-30T01:42:12.876861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:05.700970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877072+0000","last_change":"2024-03-30T01:42:10.285053+0000","last_active":"2024-03-30T01:42:12.877072+0000","last_peered":"2024-03-30T01:42:12.877072+0000","last_clean":"2024-03-30T01:42:12.877072+0000","last_became_active":"2024-03-30T01:42:10.284814+0000","last_became_peered":"2024-03-30T01:42:10.284814+0000","last_unstale":"2024-03-30T01:42:12.877072+0000","last_undegraded":"2024-03-30T01:42:12.877072+0000","last_fullsized":"2024-03-30T01:42:12.877072+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:28.875197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.807918+0000","last_change":"2024-03-30T01:42:22.807918+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.807918+0000","last_undegraded":"2024-03-30T01:42:22.807918+0000","last_fullsized":"2024-03-30T01:42:22.807918+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:29:59.201428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:18.899337+0000","last_change":"2024-03-30T01:42:09.823152+0000","last_active":"2024-03-30T01:42:18.899337+0000","last_peered":"2024-03-30T01:42:18.899337+0000","last_clean":"2024-03-30T01:42:18.899337+0000","last_became_active":"2024-03-30T01:42:09.822826+0000","last_became_peered":"2024-03-30T01:42:09.822826+0000","last_unstale":"2024-03-30T01:42:18.899337+0000","last_undegraded":"2024-03-30T01:42:18.899337+0000","last_fullsized":"2024-03-30T01:42:18.899337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:17.799215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874291+0000","last_change":"2024-03-30T01:42:10.296944+0000","last_active":"2024-03-30T01:42:12.874291+0000","last_peered":"2024-03-30T01:42:12.874291+0000","last_clean":"2024-03-30T01:42:12.874291+0000","last_became_active":"2024-03-30T01:42:10.296714+0000","last_became_peered":"2024-03-30T01:42:10.296714+0000","last_unstale":"2024-03-30T01:42:12.874291+0000","last_undegraded":"2024-03-30T01:42:12.874291+0000","last_fullsized":"2024-03-30T01:42:12.874291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815307+0000","last_change":"2024-03-30T01:42:22.815307+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815307+0000","last_undegraded":"2024-03-30T01:42:22.815307+0000","last_fullsized":"2024-03-30T01:42:22.815307+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877496+0000","last_change":"2024-03-30T01:42:09.833439+0000","last_active":"2024-03-30T01:42:12.877496+0000","last_peered":"2024-03-30T01:42:12.877496+0000","last_clean":"2024-03-30T01:42:12.877496+0000","last_became_active":"2024-03-30T01:42:09.833137+0000","last_became_peered":"2024-03-30T01:42:09.833137+0000","last_unstale":"2024-03-30T01:42:12.877496+0000","last_undegraded":"2024-03-30T01:42:12.877496+0000","last_fullsized":"2024-03-30T01:42:12.877496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877264+0000","last_change":"2024-03-30T01:42:10.292182+0000","last_active":"2024-03-30T01:42:12.877264+0000","last_peered":"2024-03-30T01:42:12.877264+0000","last_clean":"2024-03-30T01:42:12.877264+0000","last_became_active":"2024-03-30T01:42:10.291868+0000","last_became_peered":"2024-03-30T01:42:10.291868+0000","last_unstale":"2024-03-30T01:42:12.877264+0000","last_undegraded":"2024-03-30T01:42:12.877264+0000","last_fullsized":"2024-03-30T01:42:12.877264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.807910+0000","last_change":"2024-03-30T01:42:22.807910+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.807910+0000","last_undegraded":"2024-03-30T01:42:22.807910+0000","last_fullsized":"2024-03-30T01:42:22.807910+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:12.384136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877526+0000","last_change":"2024-03-30T01:42:09.826561+0000","last_active":"2024-03-30T01:42:12.877526+0000","last_peered":"2024-03-30T01:42:12.877526+0000","last_clean":"2024-03-30T01:42:12.877526+0000","last_became_active":"2024-03-30T01:42:09.826059+0000","last_became_peered":"2024-03-30T01:42:09.826059+0000","last_unstale":"2024-03-30T01:42:12.877526+0000","last_undegraded":"2024-03-30T01:42:12.877526+0000","last_fullsized":"2024-03-30T01:42:12.877526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874434+0000","last_change":"2024-03-30T01:42:10.284755+0000","last_active":"2024-03-30T01:42:12.874434+0000","last_peered":"2024-03-30T01:42:12.874434+0000","last_clean":"2024-03-30T01:42:12.874434+0000","last_became_active":"2024-03-30T01:42:10.284421+0000","last_became_peered":"2024-03-30T01:42:10.284421+0000","last_unstale":"2024-03-30T01:42:12.874434+0000","last_undegraded":"2024-03-30T01:42:12.874434+0000","last_fullsized":"2024-03-30T01:42:12.874434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.810039+0000","last_change":"2024-03-30T01:42:22.810039+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.810039+0000","last_undegraded":"2024-03-30T01:42:22.810039+0000","last_fullsized":"2024-03-30T01:42:22.810039+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:44.146196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877020+0000","last_change":"2024-03-30T01:42:09.828624+0000","last_active":"2024-03-30T01:42:12.877020+0000","last_peered":"2024-03-30T01:42:12.877020+0000","last_clean":"2024-03-30T01:42:12.877020+0000","last_became_active":"2024-03-30T01:42:09.828341+0000","last_became_peered":"2024-03-30T01:42:09.828341+0000","last_unstale":"2024-03-30T01:42:12.877020+0000","last_undegraded":"2024-03-30T01:42:12.877020+0000","last_fullsized":"2024-03-30T01:42:12.877020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:32.138427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875705+0000","last_change":"2024-03-30T01:42:10.295259+0000","last_active":"2024-03-30T01:42:12.875705+0000","last_peered":"2024-03-30T01:42:12.875705+0000","last_clean":"2024-03-30T01:42:12.875705+0000","last_became_active":"2024-03-30T01:42:10.295054+0000","last_became_peered":"2024-03-30T01:42:10.295054+0000","last_unstale":"2024-03-30T01:42:12.875705+0000","last_undegraded":"2024-03-30T01:42:12.875705+0000","last_fullsized":"2024-03-30T01:42:12.875705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:36.942583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.808461+0000","last_change":"2024-03-30T01:42:22.808461+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.808461+0000","last_undegraded":"2024-03-30T01:42:22.808461+0000","last_fullsized":"2024-03-30T01:42:22.808461+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:54:59.010522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874117+0000","last_change":"2024-03-30T01:42:09.820512+0000","last_active":"2024-03-30T01:42:12.874117+0000","last_peered":"2024-03-30T01:42:12.874117+0000","last_clean":"2024-03-30T01:42:12.874117+0000","last_became_active":"2024-03-30T01:42:09.820253+0000","last_became_peered":"2024-03-30T01:42:09.820253+0000","last_unstale":"2024-03-30T01:42:12.874117+0000","last_undegraded":"2024-03-30T01:42:12.874117+0000","last_fullsized":"2024-03-30T01:42:12.874117+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.455801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876834+0000","last_change":"2024-03-30T01:42:10.291872+0000","last_active":"2024-03-30T01:42:12.876834+0000","last_peered":"2024-03-30T01:42:12.876834+0000","last_clean":"2024-03-30T01:42:12.876834+0000","last_became_active":"2024-03-30T01:42:10.291199+0000","last_became_peered":"2024-03-30T01:42:10.291199+0000","last_unstale":"2024-03-30T01:42:12.876834+0000","last_undegraded":"2024-03-30T01:42:12.876834+0000","last_fullsized":"2024-03-30T01:42:12.876834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815299+0000","last_change":"2024-03-30T01:42:22.815299+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815299+0000","last_undegraded":"2024-03-30T01:42:22.815299+0000","last_fullsized":"2024-03-30T01:42:22.815299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877166+0000","last_change":"2024-03-30T01:42:09.826326+0000","last_active":"2024-03-30T01:42:12.877166+0000","last_peered":"2024-03-30T01:42:12.877166+0000","last_clean":"2024-03-30T01:42:12.877166+0000","last_became_active":"2024-03-30T01:42:09.826050+0000","last_became_peered":"2024-03-30T01:42:09.826050+0000","last_unstale":"2024-03-30T01:42:12.877166+0000","last_undegraded":"2024-03-30T01:42:12.877166+0000","last_fullsized":"2024-03-30T01:42:12.877166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874163+0000","last_change":"2024-03-30T01:42:10.303359+0000","last_active":"2024-03-30T01:42:12.874163+0000","last_peered":"2024-03-30T01:42:12.874163+0000","last_clean":"2024-03-30T01:42:12.874163+0000","last_became_active":"2024-03-30T01:42:10.303039+0000","last_became_peered":"2024-03-30T01:42:10.303039+0000","last_unstale":"2024-03-30T01:42:12.874163+0000","last_undegraded":"2024-03-30T01:42:12.874163+0000","last_fullsized":"2024-03-30T01:42:12.874163+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815535+0000","last_change":"2024-03-30T01:42:22.815535+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815535+0000","last_undegraded":"2024-03-30T01:42:22.815535+0000","last_fullsized":"2024-03-30T01:42:22.815535+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:36.452658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874345+0000","last_change":"2024-03-30T01:42:09.831981+0000","last_active":"2024-03-30T01:42:12.874345+0000","last_peered":"2024-03-30T01:42:12.874345+0000","last_clean":"2024-03-30T01:42:12.874345+0000","last_became_active":"2024-03-30T01:42:09.831647+0000","last_became_peered":"2024-03-30T01:42:09.831647+0000","last_unstale":"2024-03-30T01:42:12.874345+0000","last_undegraded":"2024-03-30T01:42:12.874345+0000","last_fullsized":"2024-03-30T01:42:12.874345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:02.883477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877386+0000","last_change":"2024-03-30T01:42:10.289213+0000","last_active":"2024-03-30T01:42:12.877386+0000","last_peered":"2024-03-30T01:42:12.877386+0000","last_clean":"2024-03-30T01:42:12.877386+0000","last_became_active":"2024-03-30T01:42:10.288677+0000","last_became_peered":"2024-03-30T01:42:10.288677+0000","last_unstale":"2024-03-30T01:42:12.877386+0000","last_undegraded":"2024-03-30T01:42:12.877386+0000","last_fullsized":"2024-03-30T01:42:12.877386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814739+0000","last_change":"2024-03-30T01:42:22.814739+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814739+0000","last_undegraded":"2024-03-30T01:42:22.814739+0000","last_fullsized":"2024-03-30T01:42:22.814739+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:32.897671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877446+0000","last_change":"2024-03-30T01:42:09.831496+0000","last_active":"2024-03-30T01:42:12.877446+0000","last_peered":"2024-03-30T01:42:12.877446+0000","last_clean":"2024-03-30T01:42:12.877446+0000","last_became_active":"2024-03-30T01:42:09.831197+0000","last_became_peered":"2024-03-30T01:42:09.831197+0000","last_unstale":"2024-03-30T01:42:12.877446+0000","last_undegraded":"2024-03-30T01:42:12.877446+0000","last_fullsized":"2024-03-30T01:42:12.877446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877126+0000","last_change":"2024-03-30T01:42:10.287526+0000","last_active":"2024-03-30T01:42:12.877126+0000","last_peered":"2024-03-30T01:42:12.877126+0000","last_clean":"2024-03-30T01:42:12.877126+0000","last_became_active":"2024-03-30T01:42:10.287255+0000","last_became_peered":"2024-03-30T01:42:10.287255+0000","last_unstale":"2024-03-30T01:42:12.877126+0000","last_undegraded":"2024-03-30T01:42:12.877126+0000","last_fullsized":"2024-03-30T01:42:12.877126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812195+0000","last_change":"2024-03-30T01:42:22.812195+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812195+0000","last_undegraded":"2024-03-30T01:42:22.812195+0000","last_fullsized":"2024-03-30T01:42:22.812195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:35.849177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874220+0000","last_change":"2024-03-30T01:42:09.824809+0000","last_active":"2024-03-30T01:42:12.874220+0000","last_peered":"2024-03-30T01:42:12.874220+0000","last_clean":"2024-03-30T01:42:12.874220+0000","last_became_active":"2024-03-30T01:42:09.824493+0000","last_became_peered":"2024-03-30T01:42:09.824493+0000","last_unstale":"2024-03-30T01:42:12.874220+0000","last_undegraded":"2024-03-30T01:42:12.874220+0000","last_fullsized":"2024-03-30T01:42:12.874220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877000+0000","last_change":"2024-03-30T01:42:10.291979+0000","last_active":"2024-03-30T01:42:12.877000+0000","last_peered":"2024-03-30T01:42:12.877000+0000","last_clean":"2024-03-30T01:42:12.877000+0000","last_became_active":"2024-03-30T01:42:10.291708+0000","last_became_peered":"2024-03-30T01:42:10.291708+0000","last_unstale":"2024-03-30T01:42:12.877000+0000","last_undegraded":"2024-03-30T01:42:12.877000+0000","last_fullsized":"2024-03-30T01:42:12.877000+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809931+0000","last_change":"2024-03-30T01:42:22.809931+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809931+0000","last_undegraded":"2024-03-30T01:42:22.809931+0000","last_fullsized":"2024-03-30T01:42:22.809931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877310+0000","last_change":"2024-03-30T01:42:09.829349+0000","last_active":"2024-03-30T01:42:12.877310+0000","last_peered":"2024-03-30T01:42:12.877310+0000","last_clean":"2024-03-30T01:42:12.877310+0000","last_became_active":"2024-03-30T01:42:09.828931+0000","last_became_peered":"2024-03-30T01:42:09.828931+0000","last_unstale":"2024-03-30T01:42:12.877310+0000","last_undegraded":"2024-03-30T01:42:12.877310+0000","last_fullsized":"2024-03-30T01:42:12.877310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874233+0000","last_change":"2024-03-30T01:42:10.300292+0000","last_active":"2024-03-30T01:42:12.874233+0000","last_peered":"2024-03-30T01:42:12.874233+0000","last_clean":"2024-03-30T01:42:12.874233+0000","last_became_active":"2024-03-30T01:42:10.299867+0000","last_became_peered":"2024-03-30T01:42:10.299867+0000","last_unstale":"2024-03-30T01:42:12.874233+0000","last_undegraded":"2024-03-30T01:42:12.874233+0000","last_fullsized":"2024-03-30T01:42:12.874233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812740+0000","last_change":"2024-03-30T01:42:22.812740+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812740+0000","last_undegraded":"2024-03-30T01:42:22.812740+0000","last_fullsized":"2024-03-30T01:42:22.812740+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:13.222722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877289+0000","last_change":"2024-03-30T01:42:09.829899+0000","last_active":"2024-03-30T01:42:12.877289+0000","last_peered":"2024-03-30T01:42:12.877289+0000","last_clean":"2024-03-30T01:42:12.877289+0000","last_became_active":"2024-03-30T01:42:09.829601+0000","last_became_peered":"2024-03-30T01:42:09.829601+0000","last_unstale":"2024-03-30T01:42:12.877289+0000","last_undegraded":"2024-03-30T01:42:12.877289+0000","last_fullsized":"2024-03-30T01:42:12.877289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:30.887459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877284+0000","last_change":"2024-03-30T01:42:10.298381+0000","last_active":"2024-03-30T01:42:12.877284+0000","last_peered":"2024-03-30T01:42:12.877284+0000","last_clean":"2024-03-30T01:42:12.877284+0000","last_became_active":"2024-03-30T01:42:10.298109+0000","last_became_peered":"2024-03-30T01:42:10.298109+0000","last_unstale":"2024-03-30T01:42:12.877284+0000","last_undegraded":"2024-03-30T01:42:12.877284+0000","last_fullsized":"2024-03-30T01:42:12.877284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.804764+0000","last_change":"2024-03-30T01:42:22.804764+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.804764+0000","last_undegraded":"2024-03-30T01:42:22.804764+0000","last_fullsized":"2024-03-30T01:42:22.804764+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881147+0000","last_change":"2024-03-30T01:42:09.824850+0000","last_active":"2024-03-30T01:42:12.881147+0000","last_peered":"2024-03-30T01:42:12.881147+0000","last_clean":"2024-03-30T01:42:12.881147+0000","last_became_active":"2024-03-30T01:42:09.824537+0000","last_became_peered":"2024-03-30T01:42:09.824537+0000","last_unstale":"2024-03-30T01:42:12.881147+0000","last_undegraded":"2024-03-30T01:42:12.881147+0000","last_fullsized":"2024-03-30T01:42:12.881147+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:17.759503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874391+0000","last_change":"2024-03-30T01:42:10.285919+0000","last_active":"2024-03-30T01:42:12.874391+0000","last_peered":"2024-03-30T01:42:12.874391+0000","last_clean":"2024-03-30T01:42:12.874391+0000","last_became_active":"2024-03-30T01:42:10.285629+0000","last_became_peered":"2024-03-30T01:42:10.285629+0000","last_unstale":"2024-03-30T01:42:12.874391+0000","last_undegraded":"2024-03-30T01:42:12.874391+0000","last_fullsized":"2024-03-30T01:42:12.874391+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809889+0000","last_change":"2024-03-30T01:42:22.809889+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809889+0000","last_undegraded":"2024-03-30T01:42:22.809889+0000","last_fullsized":"2024-03-30T01:42:22.809889+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:35.672193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877110+0000","last_change":"2024-03-30T01:42:10.297189+0000","last_active":"2024-03-30T01:42:12.877110+0000","last_peered":"2024-03-30T01:42:12.877110+0000","last_clean":"2024-03-30T01:42:12.877110+0000","last_became_active":"2024-03-30T01:42:10.296821+0000","last_became_peered":"2024-03-30T01:42:10.296821+0000","last_unstale":"2024-03-30T01:42:12.877110+0000","last_undegraded":"2024-03-30T01:42:12.877110+0000","last_fullsized":"2024-03-30T01:42:12.877110+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874246+0000","last_change":"2024-03-30T01:42:10.274200+0000","last_active":"2024-03-30T01:42:12.874246+0000","last_peered":"2024-03-30T01:42:12.874246+0000","last_clean":"2024-03-30T01:42:12.874246+0000","last_became_active":"2024-03-30T01:42:10.273923+0000","last_became_peered":"2024-03-30T01:42:10.273923+0000","last_unstale":"2024-03-30T01:42:12.874246+0000","last_undegraded":"2024-03-30T01:42:12.874246+0000","last_fullsized":"2024-03-30T01:42:12.874246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:44.766475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.880992+0000","last_change":"2024-03-30T01:42:10.290078+0000","last_active":"2024-03-30T01:42:12.880992+0000","last_peered":"2024-03-30T01:42:12.880992+0000","last_clean":"2024-03-30T01:42:12.880992+0000","last_became_active":"2024-03-30T01:42:10.289127+0000","last_became_peered":"2024-03-30T01:42:10.289127+0000","last_unstale":"2024-03-30T01:42:12.880992+0000","last_undegraded":"2024-03-30T01:42:12.880992+0000","last_fullsized":"2024-03-30T01:42:12.880992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:36.303285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881034+0000","last_change":"2024-03-30T01:42:10.291099+0000","last_active":"2024-03-30T01:42:12.881034+0000","last_peered":"2024-03-30T01:42:12.881034+0000","last_clean":"2024-03-30T01:42:12.881034+0000","last_became_active":"2024-03-30T01:42:10.290802+0000","last_became_peered":"2024-03-30T01:42:10.290802+0000","last_unstale":"2024-03-30T01:42:12.881034+0000","last_undegraded":"2024-03-30T01:42:12.881034+0000","last_fullsized":"2024-03-30T01:42:12.881034+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:13.070677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877459+0000","last_change":"2024-03-30T01:42:10.288633+0000","last_active":"2024-03-30T01:42:12.877459+0000","last_peered":"2024-03-30T01:42:12.877459+0000","last_clean":"2024-03-30T01:42:12.877459+0000","last_became_active":"2024-03-30T01:42:10.288326+0000","last_became_peered":"2024-03-30T01:42:10.288326+0000","last_unstale":"2024-03-30T01:42:12.877459+0000","last_undegraded":"2024-03-30T01:42:12.877459+0000","last_fullsized":"2024-03-30T01:42:12.877459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881079+0000","last_change":"2024-03-30T01:42:10.274845+0000","last_active":"2024-03-30T01:42:12.881079+0000","last_peered":"2024-03-30T01:42:12.881079+0000","last_clean":"2024-03-30T01:42:12.881079+0000","last_became_active":"2024-03-30T01:42:10.274566+0000","last_became_peered":"2024-03-30T01:42:10.274566+0000","last_unstale":"2024-03-30T01:42:12.881079+0000","last_undegraded":"2024-03-30T01:42:12.881079+0000","last_fullsized":"2024-03-30T01:42:12.881079+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:32.801396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877322+0000","last_change":"2024-03-30T01:42:10.290378+0000","last_active":"2024-03-30T01:42:12.877322+0000","last_peered":"2024-03-30T01:42:12.877322+0000","last_clean":"2024-03-30T01:42:12.877322+0000","last_became_active":"2024-03-30T01:42:10.290022+0000","last_became_peered":"2024-03-30T01:42:10.290022+0000","last_unstale":"2024-03-30T01:42:12.877322+0000","last_undegraded":"2024-03-30T01:42:12.877322+0000","last_fullsized":"2024-03-30T01:42:12.877322+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881124+0000","last_change":"2024-03-30T01:42:10.285378+0000","last_active":"2024-03-30T01:42:12.881124+0000","last_peered":"2024-03-30T01:42:12.881124+0000","last_clean":"2024-03-30T01:42:12.881124+0000","last_became_active":"2024-03-30T01:42:10.285105+0000","last_became_peered":"2024-03-30T01:42:10.285105+0000","last_unstale":"2024-03-30T01:42:12.881124+0000","last_undegraded":"2024-03-30T01:42:12.881124+0000","last_fullsized":"2024-03-30T01:42:12.881124+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:52.824468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875599+0000","last_change":"2024-03-30T01:42:10.286887+0000","last_active":"2024-03-30T01:42:12.875599+0000","last_peered":"2024-03-30T01:42:12.875599+0000","last_clean":"2024-03-30T01:42:12.875599+0000","last_became_active":"2024-03-30T01:42:10.278107+0000","last_became_peered":"2024-03-30T01:42:10.278107+0000","last_unstale":"2024-03-30T01:42:12.875599+0000","last_undegraded":"2024-03-30T01:42:12.875599+0000","last_fullsized":"2024-03-30T01:42:12.875599+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874268+0000","last_change":"2024-03-30T01:42:10.281331+0000","last_active":"2024-03-30T01:42:12.874268+0000","last_peered":"2024-03-30T01:42:12.874268+0000","last_clean":"2024-03-30T01:42:12.874268+0000","last_became_active":"2024-03-30T01:42:10.281011+0000","last_became_peered":"2024-03-30T01:42:10.281011+0000","last_unstale":"2024-03-30T01:42:12.874268+0000","last_undegraded":"2024-03-30T01:42:12.874268+0000","last_fullsized":"2024-03-30T01:42:12.874268+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874152+0000","last_change":"2024-03-30T01:42:10.299194+0000","last_active":"2024-03-30T01:42:12.874152+0000","last_peered":"2024-03-30T01:42:12.874152+0000","last_clean":"2024-03-30T01:42:12.874152+0000","last_became_active":"2024-03-30T01:42:10.298900+0000","last_became_peered":"2024-03-30T01:42:10.298900+0000","last_unstale":"2024-03-30T01:42:12.874152+0000","last_undegraded":"2024-03-30T01:42:12.874152+0000","last_fullsized":"2024-03-30T01:42:12.874152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877431+0000","last_change":"2024-03-30T01:42:10.304038+0000","last_active":"2024-03-30T01:42:12.877431+0000","last_peered":"2024-03-30T01:42:12.877431+0000","last_clean":"2024-03-30T01:42:12.877431+0000","last_became_active":"2024-03-30T01:42:10.303747+0000","last_became_peered":"2024-03-30T01:42:10.303747+0000","last_unstale":"2024-03-30T01:42:12.877431+0000","last_undegraded":"2024-03-30T01:42:12.877431+0000","last_fullsized":"2024-03-30T01:42:12.877431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881058+0000","last_change":"2024-03-30T01:42:10.300160+0000","last_active":"2024-03-30T01:42:12.881058+0000","last_peered":"2024-03-30T01:42:12.881058+0000","last_clean":"2024-03-30T01:42:12.881058+0000","last_became_active":"2024-03-30T01:42:10.299728+0000","last_became_peered":"2024-03-30T01:42:10.299728+0000","last_unstale":"2024-03-30T01:42:12.881058+0000","last_undegraded":"2024-03-30T01:42:12.881058+0000","last_fullsized":"2024-03-30T01:42:12.881058+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875715+0000","last_change":"2024-03-30T01:42:10.292351+0000","last_active":"2024-03-30T01:42:12.875715+0000","last_peered":"2024-03-30T01:42:12.875715+0000","last_clean":"2024-03-30T01:42:12.875715+0000","last_became_active":"2024-03-30T01:42:10.292040+0000","last_became_peered":"2024-03-30T01:42:10.292040+0000","last_unstale":"2024-03-30T01:42:12.875715+0000","last_undegraded":"2024-03-30T01:42:12.875715+0000","last_fullsized":"2024-03-30T01:42:12.875715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:09.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881101+0000","last_change":"2024-03-30T01:42:10.300393+0000","last_active":"2024-03-30T01:42:12.881101+0000","last_peered":"2024-03-30T01:42:12.881101+0000","last_clean":"2024-03-30T01:42:12.881101+0000","last_became_active":"2024-03-30T01:42:10.299795+0000","last_became_peered":"2024-03-30T01:42:10.299795+0000","last_unstale":"2024-03-30T01:42:12.881101+0000","last_undegraded":"2024-03-30T01:42:12.881101+0000","last_fullsized":"2024-03-30T01:42:12.881101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:21.256043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874478+0000","last_change":"2024-03-30T01:42:10.285733+0000","last_active":"2024-03-30T01:42:12.874478+0000","last_peered":"2024-03-30T01:42:12.874478+0000","last_clean":"2024-03-30T01:42:12.874478+0000","last_became_active":"2024-03-30T01:42:10.285428+0000","last_became_peered":"2024-03-30T01:42:10.285428+0000","last_unstale":"2024-03-30T01:42:12.874478+0000","last_undegraded":"2024-03-30T01:42:12.874478+0000","last_fullsized":"2024-03-30T01:42:12.874478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874143+0000","last_change":"2024-03-30T01:42:10.284419+0000","last_active":"2024-03-30T01:42:12.874143+0000","last_peered":"2024-03-30T01:42:12.874143+0000","last_clean":"2024-03-30T01:42:12.874143+0000","last_became_active":"2024-03-30T01:42:10.284098+0000","last_became_peered":"2024-03-30T01:42:10.284098+0000","last_unstale":"2024-03-30T01:42:12.874143+0000","last_undegraded":"2024-03-30T01:42:12.874143+0000","last_fullsized":"2024-03-30T01:42:12.874143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877370+0000","last_change":"2024-03-30T01:42:10.293953+0000","last_active":"2024-03-30T01:42:12.877370+0000","last_peered":"2024-03-30T01:42:12.877370+0000","last_clean":"2024-03-30T01:42:12.877370+0000","last_became_active":"2024-03-30T01:42:10.293641+0000","last_became_peered":"2024-03-30T01:42:10.293641+0000","last_unstale":"2024-03-30T01:42:12.877370+0000","last_undegraded":"2024-03-30T01:42:12.877370+0000","last_fullsized":"2024-03-30T01:42:12.877370+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874206+0000","last_change":"2024-03-30T01:42:10.294667+0000","last_active":"2024-03-30T01:42:12.874206+0000","last_peered":"2024-03-30T01:42:12.874206+0000","last_clean":"2024-03-30T01:42:12.874206+0000","last_became_active":"2024-03-30T01:42:10.294330+0000","last_became_peered":"2024-03-30T01:42:10.294330+0000","last_unstale":"2024-03-30T01:42:12.874206+0000","last_undegraded":"2024-03-30T01:42:12.874206+0000","last_fullsized":"2024-03-30T01:42:12.874206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877421+0000","last_change":"2024-03-30T01:42:10.295808+0000","last_active":"2024-03-30T01:42:12.877421+0000","last_peered":"2024-03-30T01:42:12.877421+0000","last_clean":"2024-03-30T01:42:12.877421+0000","last_became_active":"2024-03-30T01:42:10.295460+0000","last_became_peered":"2024-03-30T01:42:10.295460+0000","last_unstale":"2024-03-30T01:42:12.877421+0000","last_undegraded":"2024-03-30T01:42:12.877421+0000","last_fullsized":"2024-03-30T01:42:12.877421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877196+0000","last_change":"2024-03-30T01:42:10.294103+0000","last_active":"2024-03-30T01:42:12.877196+0000","last_peered":"2024-03-30T01:42:12.877196+0000","last_clean":"2024-03-30T01:42:12.877196+0000","last_became_active":"2024-03-30T01:42:10.293833+0000","last_became_peered":"2024-03-30T01:42:10.293833+0000","last_unstale":"2024-03-30T01:42:12.877196+0000","last_undegraded":"2024-03-30T01:42:12.877196+0000","last_fullsized":"2024-03-30T01:42:12.877196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877485+0000","last_change":"2024-03-30T01:42:10.296160+0000","last_active":"2024-03-30T01:42:12.877485+0000","last_peered":"2024-03-30T01:42:12.877485+0000","last_clean":"2024-03-30T01:42:12.877485+0000","last_became_active":"2024-03-30T01:42:10.295867+0000","last_became_peered":"2024-03-30T01:42:10.295867+0000","last_unstale":"2024-03-30T01:42:12.877485+0000","last_undegraded":"2024-03-30T01:42:12.877485+0000","last_fullsized":"2024-03-30T01:42:12.877485+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877510+0000","last_change":"2024-03-30T01:42:10.295963+0000","last_active":"2024-03-30T01:42:12.877510+0000","last_peered":"2024-03-30T01:42:12.877510+0000","last_clean":"2024-03-30T01:42:12.877510+0000","last_became_active":"2024-03-30T01:42:10.295545+0000","last_became_peered":"2024-03-30T01:42:10.295545+0000","last_unstale":"2024-03-30T01:42:12.877510+0000","last_undegraded":"2024-03-30T01:42:12.877510+0000","last_fullsized":"2024-03-30T01:42:12.877510+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877312+0000","last_change":"2024-03-30T01:42:10.285717+0000","last_active":"2024-03-30T01:42:12.877312+0000","last_peered":"2024-03-30T01:42:12.877312+0000","last_clean":"2024-03-30T01:42:12.877312+0000","last_became_active":"2024-03-30T01:42:10.285437+0000","last_became_peered":"2024-03-30T01:42:10.285437+0000","last_unstale":"2024-03-30T01:42:12.877312+0000","last_undegraded":"2024-03-30T01:42:12.877312+0000","last_fullsized":"2024-03-30T01:42:12.877312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876706+0000","last_change":"2024-03-30T01:42:10.298731+0000","last_active":"2024-03-30T01:42:12.876706+0000","last_peered":"2024-03-30T01:42:12.876706+0000","last_clean":"2024-03-30T01:42:12.876706+0000","last_became_active":"2024-03-30T01:42:10.298525+0000","last_became_peered":"2024-03-30T01:42:10.298525+0000","last_unstale":"2024-03-30T01:42:12.876706+0000","last_undegraded":"2024-03-30T01:42:12.876706+0000","last_fullsized":"2024-03-30T01:42:12.876706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877398+0000","last_change":"2024-03-30T01:42:10.289068+0000","last_active":"2024-03-30T01:42:12.877398+0000","last_peered":"2024-03-30T01:42:12.877398+0000","last_clean":"2024-03-30T01:42:12.877398+0000","last_became_active":"2024-03-30T01:42:10.288282+0000","last_became_peered":"2024-03-30T01:42:10.288282+0000","last_unstale":"2024-03-30T01:42:12.877398+0000","last_undegraded":"2024-03-30T01:42:12.877398+0000","last_fullsized":"2024-03-30T01:42:12.877398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877189+0000","last_change":"2024-03-30T01:42:10.287001+0000","last_active":"2024-03-30T01:42:12.877189+0000","last_peered":"2024-03-30T01:42:12.877189+0000","last_clean":"2024-03-30T01:42:12.877189+0000","last_became_active":"2024-03-30T01:42:10.286700+0000","last_became_peered":"2024-03-30T01:42:10.286700+0000","last_unstale":"2024-03-30T01:42:12.877189+0000","last_undegraded":"2024-03-30T01:42:12.877189+0000","last_fullsized":"2024-03-30T01:42:12.877189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875664+0000","last_change":"2024-03-30T01:42:10.299868+0000","last_active":"2024-03-30T01:42:12.875664+0000","last_peered":"2024-03-30T01:42:12.875664+0000","last_clean":"2024-03-30T01:42:12.875664+0000","last_became_active":"2024-03-30T01:42:10.299612+0000","last_became_peered":"2024-03-30T01:42:10.299612+0000","last_unstale":"2024-03-30T01:42:12.875664+0000","last_undegraded":"2024-03-30T01:42:12.875664+0000","last_fullsized":"2024-03-30T01:42:12.875664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:49.471045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875690+0000","last_change":"2024-03-30T01:42:10.294277+0000","last_active":"2024-03-30T01:42:12.875690+0000","last_peered":"2024-03-30T01:42:12.875690+0000","last_clean":"2024-03-30T01:42:12.875690+0000","last_became_active":"2024-03-30T01:42:10.293995+0000","last_became_peered":"2024-03-30T01:42:10.293995+0000","last_unstale":"2024-03-30T01:42:12.875690+0000","last_undegraded":"2024-03-30T01:42:12.875690+0000","last_fullsized":"2024-03-30T01:42:12.875690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:03.888060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877217+0000","last_change":"2024-03-30T01:42:10.285050+0000","last_active":"2024-03-30T01:42:12.877217+0000","last_peered":"2024-03-30T01:42:12.877217+0000","last_clean":"2024-03-30T01:42:12.877217+0000","last_became_active":"2024-03-30T01:42:10.284771+0000","last_became_peered":"2024-03-30T01:42:10.284771+0000","last_unstale":"2024-03-30T01:42:12.877217+0000","last_undegraded":"2024-03-30T01:42:12.877217+0000","last_fullsized":"2024-03-30T01:42:12.877217+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876935+0000","last_change":"2024-03-30T01:42:10.294965+0000","last_active":"2024-03-30T01:42:12.876935+0000","last_peered":"2024-03-30T01:42:12.876935+0000","last_clean":"2024-03-30T01:42:12.876935+0000","last_became_active":"2024-03-30T01:42:10.294685+0000","last_became_peered":"2024-03-30T01:42:10.294685+0000","last_unstale":"2024-03-30T01:42:12.876935+0000","last_undegraded":"2024-03-30T01:42:12.876935+0000","last_fullsized":"2024-03-30T01:42:12.876935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876977+0000","last_change":"2024-03-30T01:42:10.288258+0000","last_active":"2024-03-30T01:42:12.876977+0000","last_peered":"2024-03-30T01:42:12.876977+0000","last_clean":"2024-03-30T01:42:12.876977+0000","last_became_active":"2024-03-30T01:42:10.287993+0000","last_became_peered":"2024-03-30T01:42:10.287993+0000","last_unstale":"2024-03-30T01:42:12.876977+0000","last_undegraded":"2024-03-30T01:42:12.876977+0000","last_fullsized":"2024-03-30T01:42:12.876977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1060864,"data_stored":787628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.505},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.61199999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.56999999999999995}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.2009999999999996,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.41399999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.58299999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.47299999999999998}]},{"osd":1,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.3250000000000002,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.68300000000000005}]},{"osd":2,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.70399999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.79200000000000004}]},{"osd":7,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":15.038,"5min":15.038,"15min":15.038},"last":0.47499999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":11.858000000000001,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1289999999999996,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":11.99,"5min":11.99,"15min":11.99},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.57999999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":15.006,"5min":15.006,"15min":15.006},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":11.926,"5min":11.926,"15min":11.926},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":14.974,"5min":14.974,"15min":14.974},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.960000000000001,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":14.923999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.755}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":783548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.946999999999999,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.443}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":12.983000000000001,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.2279999999999998,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.3869999999999996,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":5.468,"5min":5.468,"15min":5.468},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":12.478999999999999,"5min":12.478999999999999,"15min":12.478999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.31},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":12.425000000000001,"5min":12.425000000000001,"15min":12.425000000000001},"last":0.33500000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":13.217000000000001,"5min":13.217000000000001,"15min":13.217000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":8.7400000000000002,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":8.5640000000000001,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":0.59099999999999997}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":8.7759999999999998,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":8.5190000000000001,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":8.5939999999999994,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":466944,"data_stored":197224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.378},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.45600000000000002}]},{"osd":1,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.441}]},{"osd":4,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.504}]},{"osd":5,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.53600000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.56899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.87}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":9.3970000000000002,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.747}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.84899999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":5.5289999999999999,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":9.452,"5min":9.452,"15min":9.452},"last":0.68200000000000005}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.62}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.79500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":6.6760000000000002,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.63200000000000001}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7480000000000002,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.7709999999999999,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.8410000000000002,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.37}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41199999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.442},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:42:28.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:42:29.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.122+0000 7fd228290700 1 Processor -- start 2024-03-30T01:42:29.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.122+0000 7fd228290700 1 -- start start 2024-03-30T01:42:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.122+0000 7fd228290700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd220154760 0x7fd220155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.122+0000 7fd228290700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2200c8d10 con 0x7fd22006b170 2024-03-30T01:42:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.122+0000 7fd228290700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2200c98b0 con 0x7fd2200bd7f0 2024-03-30T01:42:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd228290700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22006bef0 con 0x7fd220154760 2024-03-30T01:42:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22602c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd22006b170 0x7fd2200c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47406/0 (socket says 172.21.15.31:47406) 2024-03-30T01:42:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22682d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd220154760 0x7fd220155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22602c700 1 -- 172.21.15.31:0/1831495247 learned_addr learned my addr 172.21.15.31:0/1831495247 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22682d700 1 -- 172.21.15.31:0/1831495247 >> v1:172.21.15.59:6789/0 conn(0x7fd2200bd7f0 legacy=0x7fd22015d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22682d700 1 -- 172.21.15.31:0/1831495247 >> v1:172.21.15.31:6789/0 conn(0x7fd22006b170 legacy=0x7fd2200c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22682d700 1 -- 172.21.15.31:0/1831495247 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2200c8100 con 0x7fd220154760 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22682d700 1 --2- 172.21.15.31:0/1831495247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 0x7fd220155c50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd218009d90 tx=0x7fd21800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e63f1ca4da0e1a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22502a700 1 -- 172.21.15.31:0/1831495247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800e070 con 0x7fd220154760 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22502a700 1 -- 172.21.15.31:0/1831495247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21800bbd0 con 0x7fd220154760 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.123+0000 7fd22502a700 1 -- 172.21.15.31:0/1831495247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218012430 con 0x7fd220154760 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.124+0000 7fd228290700 1 -- 172.21.15.31:0/1831495247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 msgr2=0x7fd220155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.124+0000 7fd228290700 1 --2- 172.21.15.31:0/1831495247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 0x7fd220155c50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd218009d90 tx=0x7fd21800b330 comp rx=0 tx=0).stop 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.124+0000 7fd228290700 1 -- 172.21.15.31:0/1831495247 shutdown_connections 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.124+0000 7fd228290700 1 --2- 172.21.15.31:0/1831495247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 0x7fd220155c50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.124+0000 7fd228290700 1 -- 172.21.15.31:0/1831495247 >> 172.21.15.31:0/1831495247 conn(0x7fd2200ca0d0 msgr2=0x7fd2200ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.124+0000 7fd228290700 1 -- 172.21.15.31:0/1831495247 shutdown_connections 2024-03-30T01:42:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.124+0000 7fd228290700 1 -- 172.21.15.31:0/1831495247 wait complete. 2024-03-30T01:42:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.125+0000 7fd228290700 1 Processor -- start 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.125+0000 7fd228290700 1 -- start start 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd228290700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd22006b170 0x7fd2201ed920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd228290700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2200bd7f0 0x7fd2200c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd228290700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 0x7fd2200c3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd228290700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22014f2a0 con 0x7fd220154760 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd228290700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201ede60 con 0x7fd22006b170 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd228290700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2200ae9f0 con 0x7fd2200bd7f0 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd22682d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 0x7fd2200c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd22682d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 0x7fd2200c3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44698/0 (socket says 172.21.15.31:44698) 2024-03-30T01:42:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd22682d700 1 -- 172.21.15.31:0/4120741320 learned_addr learned my addr 172.21.15.31:0/4120741320 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd22602c700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd22006b170 0x7fd2201ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.126+0000 7fd22582b700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2200bd7f0 0x7fd2200c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd22682d700 1 -- 172.21.15.31:0/4120741320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2200bd7f0 msgr2=0x7fd2200c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd22682d700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2200bd7f0 0x7fd2200c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd22682d700 1 -- 172.21.15.31:0/4120741320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd22006b170 msgr2=0x7fd2201ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd22682d700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd22006b170 0x7fd2201ed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd22682d700 1 -- 172.21.15.31:0/4120741320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd218009a70 con 0x7fd220154760 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd22582b700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2200bd7f0 0x7fd2200c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd22682d700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 0x7fd2200c3330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd21800b370 tx=0x7fd218004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd216ffd700 1 -- 172.21.15.31:0/4120741320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218012e50 con 0x7fd220154760 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd216ffd700 1 -- 172.21.15.31:0/4120741320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd218020070 con 0x7fd220154760 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2200aecd0 con 0x7fd220154760 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd216ffd700 1 -- 172.21.15.31:0/4120741320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218004280 con 0x7fd220154760 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.127+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2200add60 con 0x7fd220154760 2024-03-30T01:42:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.128+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22004ea80 con 0x7fd220154760 2024-03-30T01:42:29.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.129+0000 7fd216ffd700 1 -- 172.21.15.31:0/4120741320 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd2180043e0 con 0x7fd220154760 2024-03-30T01:42:29.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.129+0000 7fd216ffd700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd20406acc0 0x7fd20406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.129+0000 7fd216ffd700 1 -- 172.21.15.31:0/4120741320 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fd218022070 con 0x7fd220154760 2024-03-30T01:42:29.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.129+0000 7fd22602c700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd20406acc0 0x7fd20406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.130+0000 7fd22602c700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd20406acc0 0x7fd20406d180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd21c005910 tx=0x7fd21c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:29.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.134+0000 7fd216ffd700 1 -- 172.21.15.31:0/4120741320 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd218056720 con 0x7fd220154760 2024-03-30T01:42:29.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.346+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2200c89a0 con 0x7fd20406acc0 2024-03-30T01:42:29.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.360+0000 7fd216ffd700 1 -- 172.21.15.31:0/4120741320 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+516228 (secure 0 0 0) 0x7fd21c001580 con 0x7fd20406acc0 2024-03-30T01:42:29.375 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd20406acc0 msgr2=0x7fd20406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd20406acc0 0x7fd20406d180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd21c005910 tx=0x7fd21c009040 comp rx=0 tx=0).stop 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 msgr2=0x7fd2200c3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 0x7fd2200c3330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd21800b370 tx=0x7fd218004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 shutdown_connections 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd220154760 0x7fd2200c3330 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd20406acc0 0x7fd20406d180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd22006b170 0x7fd2201ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 --2- 172.21.15.31:0/4120741320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2200bd7f0 0x7fd2200c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 >> 172.21.15.31:0/4120741320 conn(0x7fd2200ca0d0 msgr2=0x7fd220153600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 shutdown_connections 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.365+0000 7fd228290700 1 -- 172.21.15.31:0/4120741320 wait complete. 2024-03-30T01:42:29.383 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:42:29.384 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-03-30T01:42:27.827998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594241,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220932,"kb_used_data":4740,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753788,"statfs":{"total":773094113280,"available":772867878912,"internally_reserved":0,"allocated":4853760,"data_stored":2740433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.023745"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:11.877685+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:42:11.877685+0000","last_peered":"2024-03-30T01:42:11.877685+0000","last_clean":"2024-03-30T01:42:11.877685+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:42:11.877685+0000","last_undegraded":"2024-03-30T01:42:11.877685+0000","last_fullsized":"2024-03-30T01:42:11.877685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874412+0000","last_change":"2024-03-30T01:42:09.836532+0000","last_active":"2024-03-30T01:42:12.874412+0000","last_peered":"2024-03-30T01:42:12.874412+0000","last_clean":"2024-03-30T01:42:12.874412+0000","last_became_active":"2024-03-30T01:42:09.836262+0000","last_became_peered":"2024-03-30T01:42:09.836262+0000","last_unstale":"2024-03-30T01:42:12.874412+0000","last_undegraded":"2024-03-30T01:42:12.874412+0000","last_fullsized":"2024-03-30T01:42:12.874412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:08.249431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877538+0000","last_change":"2024-03-30T01:42:10.289721+0000","last_active":"2024-03-30T01:42:12.877538+0000","last_peered":"2024-03-30T01:42:12.877538+0000","last_clean":"2024-03-30T01:42:12.877538+0000","last_became_active":"2024-03-30T01:42:10.289413+0000","last_became_peered":"2024-03-30T01:42:10.289413+0000","last_unstale":"2024-03-30T01:42:12.877538+0000","last_undegraded":"2024-03-30T01:42:12.877538+0000","last_fullsized":"2024-03-30T01:42:12.877538+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814741+0000","last_change":"2024-03-30T01:42:22.814741+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814741+0000","last_undegraded":"2024-03-30T01:42:22.814741+0000","last_fullsized":"2024-03-30T01:42:22.814741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:34.573756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876734+0000","last_change":"2024-03-30T01:42:09.829174+0000","last_active":"2024-03-30T01:42:12.876734+0000","last_peered":"2024-03-30T01:42:12.876734+0000","last_clean":"2024-03-30T01:42:12.876734+0000","last_became_active":"2024-03-30T01:42:09.828730+0000","last_became_peered":"2024-03-30T01:42:09.828730+0000","last_unstale":"2024-03-30T01:42:12.876734+0000","last_undegraded":"2024-03-30T01:42:12.876734+0000","last_fullsized":"2024-03-30T01:42:12.876734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874371+0000","last_change":"2024-03-30T01:42:10.284566+0000","last_active":"2024-03-30T01:42:12.874371+0000","last_peered":"2024-03-30T01:42:12.874371+0000","last_clean":"2024-03-30T01:42:12.874371+0000","last_became_active":"2024-03-30T01:42:10.284192+0000","last_became_peered":"2024-03-30T01:42:10.284192+0000","last_unstale":"2024-03-30T01:42:12.874371+0000","last_undegraded":"2024-03-30T01:42:12.874371+0000","last_fullsized":"2024-03-30T01:42:12.874371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:05.460383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809509+0000","last_change":"2024-03-30T01:42:22.809509+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809509+0000","last_undegraded":"2024-03-30T01:42:22.809509+0000","last_fullsized":"2024-03-30T01:42:22.809509+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:56.846961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874138+0000","last_change":"2024-03-30T01:42:09.832991+0000","last_active":"2024-03-30T01:42:12.874138+0000","last_peered":"2024-03-30T01:42:12.874138+0000","last_clean":"2024-03-30T01:42:12.874138+0000","last_became_active":"2024-03-30T01:42:09.832682+0000","last_became_peered":"2024-03-30T01:42:09.832682+0000","last_unstale":"2024-03-30T01:42:12.874138+0000","last_undegraded":"2024-03-30T01:42:12.874138+0000","last_fullsized":"2024-03-30T01:42:12.874138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:37.427434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874189+0000","last_change":"2024-03-30T01:42:10.299170+0000","last_active":"2024-03-30T01:42:12.874189+0000","last_peered":"2024-03-30T01:42:12.874189+0000","last_clean":"2024-03-30T01:42:12.874189+0000","last_became_active":"2024-03-30T01:42:10.298846+0000","last_became_peered":"2024-03-30T01:42:10.298846+0000","last_unstale":"2024-03-30T01:42:12.874189+0000","last_undegraded":"2024-03-30T01:42:12.874189+0000","last_fullsized":"2024-03-30T01:42:12.874189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809736+0000","last_change":"2024-03-30T01:42:22.809736+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809736+0000","last_undegraded":"2024-03-30T01:42:22.809736+0000","last_fullsized":"2024-03-30T01:42:22.809736+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:43.792947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881014+0000","last_change":"2024-03-30T01:42:09.822074+0000","last_active":"2024-03-30T01:42:12.881014+0000","last_peered":"2024-03-30T01:42:12.881014+0000","last_clean":"2024-03-30T01:42:12.881014+0000","last_became_active":"2024-03-30T01:42:09.821786+0000","last_became_peered":"2024-03-30T01:42:09.821786+0000","last_unstale":"2024-03-30T01:42:12.881014+0000","last_undegraded":"2024-03-30T01:42:12.881014+0000","last_fullsized":"2024-03-30T01:42:12.881014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:18.840885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877139+0000","last_change":"2024-03-30T01:42:10.291029+0000","last_active":"2024-03-30T01:42:12.877139+0000","last_peered":"2024-03-30T01:42:12.877139+0000","last_clean":"2024-03-30T01:42:12.877139+0000","last_became_active":"2024-03-30T01:42:10.290728+0000","last_became_peered":"2024-03-30T01:42:10.290728+0000","last_unstale":"2024-03-30T01:42:12.877139+0000","last_undegraded":"2024-03-30T01:42:12.877139+0000","last_fullsized":"2024-03-30T01:42:12.877139+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.804768+0000","last_change":"2024-03-30T01:42:22.804768+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.804768+0000","last_undegraded":"2024-03-30T01:42:22.804768+0000","last_fullsized":"2024-03-30T01:42:22.804768+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877173+0000","last_change":"2024-03-30T01:42:09.827116+0000","last_active":"2024-03-30T01:42:12.877173+0000","last_peered":"2024-03-30T01:42:12.877173+0000","last_clean":"2024-03-30T01:42:12.877173+0000","last_became_active":"2024-03-30T01:42:09.826876+0000","last_became_peered":"2024-03-30T01:42:09.826876+0000","last_unstale":"2024-03-30T01:42:12.877173+0000","last_undegraded":"2024-03-30T01:42:12.877173+0000","last_fullsized":"2024-03-30T01:42:12.877173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:38.792762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877100+0000","last_change":"2024-03-30T01:42:10.288334+0000","last_active":"2024-03-30T01:42:12.877100+0000","last_peered":"2024-03-30T01:42:12.877100+0000","last_clean":"2024-03-30T01:42:12.877100+0000","last_became_active":"2024-03-30T01:42:10.288093+0000","last_became_peered":"2024-03-30T01:42:10.288093+0000","last_unstale":"2024-03-30T01:42:12.877100+0000","last_undegraded":"2024-03-30T01:42:12.877100+0000","last_fullsized":"2024-03-30T01:42:12.877100+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815309+0000","last_change":"2024-03-30T01:42:22.815309+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815309+0000","last_undegraded":"2024-03-30T01:42:22.815309+0000","last_fullsized":"2024-03-30T01:42:22.815309+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:44:11.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877240+0000","last_change":"2024-03-30T01:42:09.830490+0000","last_active":"2024-03-30T01:42:12.877240+0000","last_peered":"2024-03-30T01:42:12.877240+0000","last_clean":"2024-03-30T01:42:12.877240+0000","last_became_active":"2024-03-30T01:42:09.830197+0000","last_became_peered":"2024-03-30T01:42:09.830197+0000","last_unstale":"2024-03-30T01:42:12.877240+0000","last_undegraded":"2024-03-30T01:42:12.877240+0000","last_fullsized":"2024-03-30T01:42:12.877240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:46.433516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876955+0000","last_change":"2024-03-30T01:42:10.299986+0000","last_active":"2024-03-30T01:42:12.876955+0000","last_peered":"2024-03-30T01:42:12.876955+0000","last_clean":"2024-03-30T01:42:12.876955+0000","last_became_active":"2024-03-30T01:42:10.299776+0000","last_became_peered":"2024-03-30T01:42:10.299776+0000","last_unstale":"2024-03-30T01:42:12.876955+0000","last_undegraded":"2024-03-30T01:42:12.876955+0000","last_fullsized":"2024-03-30T01:42:12.876955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812125+0000","last_change":"2024-03-30T01:42:22.812125+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812125+0000","last_undegraded":"2024-03-30T01:42:22.812125+0000","last_fullsized":"2024-03-30T01:42:22.812125+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:58.845872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875741+0000","last_change":"2024-03-30T01:42:09.824373+0000","last_active":"2024-03-30T01:42:12.875741+0000","last_peered":"2024-03-30T01:42:12.875741+0000","last_clean":"2024-03-30T01:42:12.875741+0000","last_became_active":"2024-03-30T01:42:09.824089+0000","last_became_peered":"2024-03-30T01:42:09.824089+0000","last_unstale":"2024-03-30T01:42:12.875741+0000","last_undegraded":"2024-03-30T01:42:12.875741+0000","last_fullsized":"2024-03-30T01:42:12.875741+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:37.434399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874455+0000","last_change":"2024-03-30T01:42:10.284232+0000","last_active":"2024-03-30T01:42:12.874455+0000","last_peered":"2024-03-30T01:42:12.874455+0000","last_clean":"2024-03-30T01:42:12.874455+0000","last_became_active":"2024-03-30T01:42:10.283909+0000","last_became_peered":"2024-03-30T01:42:10.283909+0000","last_unstale":"2024-03-30T01:42:12.874455+0000","last_undegraded":"2024-03-30T01:42:12.874455+0000","last_fullsized":"2024-03-30T01:42:12.874455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:33.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815396+0000","last_change":"2024-03-30T01:42:22.815396+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815396+0000","last_undegraded":"2024-03-30T01:42:22.815396+0000","last_fullsized":"2024-03-30T01:42:22.815396+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:42:51.968187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877562+0000","last_change":"2024-03-30T01:42:09.833189+0000","last_active":"2024-03-30T01:42:12.877562+0000","last_peered":"2024-03-30T01:42:12.877562+0000","last_clean":"2024-03-30T01:42:12.877562+0000","last_became_active":"2024-03-30T01:42:09.832891+0000","last_became_peered":"2024-03-30T01:42:09.832891+0000","last_unstale":"2024-03-30T01:42:12.877562+0000","last_undegraded":"2024-03-30T01:42:12.877562+0000","last_fullsized":"2024-03-30T01:42:12.877562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877221+0000","last_change":"2024-03-30T01:42:10.298216+0000","last_active":"2024-03-30T01:42:12.877221+0000","last_peered":"2024-03-30T01:42:12.877221+0000","last_clean":"2024-03-30T01:42:12.877221+0000","last_became_active":"2024-03-30T01:42:10.297976+0000","last_became_peered":"2024-03-30T01:42:10.297976+0000","last_unstale":"2024-03-30T01:42:12.877221+0000","last_undegraded":"2024-03-30T01:42:12.877221+0000","last_fullsized":"2024-03-30T01:42:12.877221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.807923+0000","last_change":"2024-03-30T01:42:22.807923+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.807923+0000","last_undegraded":"2024-03-30T01:42:22.807923+0000","last_fullsized":"2024-03-30T01:42:22.807923+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875636+0000","last_change":"2024-03-30T01:42:09.829638+0000","last_active":"2024-03-30T01:42:12.875636+0000","last_peered":"2024-03-30T01:42:12.875636+0000","last_clean":"2024-03-30T01:42:12.875636+0000","last_became_active":"2024-03-30T01:42:09.829317+0000","last_became_peered":"2024-03-30T01:42:09.829317+0000","last_unstale":"2024-03-30T01:42:12.875636+0000","last_undegraded":"2024-03-30T01:42:12.875636+0000","last_fullsized":"2024-03-30T01:42:12.875636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:33.639097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876807+0000","last_change":"2024-03-30T01:42:10.292043+0000","last_active":"2024-03-30T01:42:12.876807+0000","last_peered":"2024-03-30T01:42:12.876807+0000","last_clean":"2024-03-30T01:42:12.876807+0000","last_became_active":"2024-03-30T01:42:10.291598+0000","last_became_peered":"2024-03-30T01:42:10.291598+0000","last_unstale":"2024-03-30T01:42:12.876807+0000","last_undegraded":"2024-03-30T01:42:12.876807+0000","last_fullsized":"2024-03-30T01:42:12.876807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.810206+0000","last_change":"2024-03-30T01:42:22.810206+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.810206+0000","last_undegraded":"2024-03-30T01:42:22.810206+0000","last_fullsized":"2024-03-30T01:42:22.810206+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:57.966402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874291+0000","last_change":"2024-03-30T01:42:09.829777+0000","last_active":"2024-03-30T01:42:12.874291+0000","last_peered":"2024-03-30T01:42:12.874291+0000","last_clean":"2024-03-30T01:42:12.874291+0000","last_became_active":"2024-03-30T01:42:09.829483+0000","last_became_peered":"2024-03-30T01:42:09.829483+0000","last_unstale":"2024-03-30T01:42:12.874291+0000","last_undegraded":"2024-03-30T01:42:12.874291+0000","last_fullsized":"2024-03-30T01:42:12.874291+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:45.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877410+0000","last_change":"2024-03-30T01:42:10.285649+0000","last_active":"2024-03-30T01:42:12.877410+0000","last_peered":"2024-03-30T01:42:12.877410+0000","last_clean":"2024-03-30T01:42:12.877410+0000","last_became_active":"2024-03-30T01:42:10.285297+0000","last_became_peered":"2024-03-30T01:42:10.285297+0000","last_unstale":"2024-03-30T01:42:12.877410+0000","last_undegraded":"2024-03-30T01:42:12.877410+0000","last_fullsized":"2024-03-30T01:42:12.877410+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809986+0000","last_change":"2024-03-30T01:42:22.809986+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809986+0000","last_undegraded":"2024-03-30T01:42:22.809986+0000","last_fullsized":"2024-03-30T01:42:22.809986+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:19:37.521361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875652+0000","last_change":"2024-03-30T01:42:09.826256+0000","last_active":"2024-03-30T01:42:12.875652+0000","last_peered":"2024-03-30T01:42:12.875652+0000","last_clean":"2024-03-30T01:42:12.875652+0000","last_became_active":"2024-03-30T01:42:09.825956+0000","last_became_peered":"2024-03-30T01:42:09.825956+0000","last_unstale":"2024-03-30T01:42:12.875652+0000","last_undegraded":"2024-03-30T01:42:12.875652+0000","last_fullsized":"2024-03-30T01:42:12.875652+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:59:41.446927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877149+0000","last_change":"2024-03-30T01:42:10.290469+0000","last_active":"2024-03-30T01:42:12.877149+0000","last_peered":"2024-03-30T01:42:12.877149+0000","last_clean":"2024-03-30T01:42:12.877149+0000","last_became_active":"2024-03-30T01:42:10.290148+0000","last_became_peered":"2024-03-30T01:42:10.290148+0000","last_unstale":"2024-03-30T01:42:12.877149+0000","last_undegraded":"2024-03-30T01:42:12.877149+0000","last_fullsized":"2024-03-30T01:42:12.877149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.805285+0000","last_change":"2024-03-30T01:42:22.805285+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.805285+0000","last_undegraded":"2024-03-30T01:42:22.805285+0000","last_fullsized":"2024-03-30T01:42:22.805285+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:54.923310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875675+0000","last_change":"2024-03-30T01:42:09.830452+0000","last_active":"2024-03-30T01:42:12.875675+0000","last_peered":"2024-03-30T01:42:12.875675+0000","last_clean":"2024-03-30T01:42:12.875675+0000","last_became_active":"2024-03-30T01:42:09.830173+0000","last_became_peered":"2024-03-30T01:42:09.830173+0000","last_unstale":"2024-03-30T01:42:12.875675+0000","last_undegraded":"2024-03-30T01:42:12.875675+0000","last_fullsized":"2024-03-30T01:42:12.875675+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:12:42.807604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874172+0000","last_change":"2024-03-30T01:42:10.279913+0000","last_active":"2024-03-30T01:42:12.874172+0000","last_peered":"2024-03-30T01:42:12.874172+0000","last_clean":"2024-03-30T01:42:12.874172+0000","last_became_active":"2024-03-30T01:42:10.279450+0000","last_became_peered":"2024-03-30T01:42:10.279450+0000","last_unstale":"2024-03-30T01:42:12.874172+0000","last_undegraded":"2024-03-30T01:42:12.874172+0000","last_fullsized":"2024-03-30T01:42:12.874172+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814722+0000","last_change":"2024-03-30T01:42:22.814722+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814722+0000","last_undegraded":"2024-03-30T01:42:22.814722+0000","last_fullsized":"2024-03-30T01:42:22.814722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:44.454854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877240+0000","last_change":"2024-03-30T01:42:09.828572+0000","last_active":"2024-03-30T01:42:12.877240+0000","last_peered":"2024-03-30T01:42:12.877240+0000","last_clean":"2024-03-30T01:42:12.877240+0000","last_became_active":"2024-03-30T01:42:09.828225+0000","last_became_peered":"2024-03-30T01:42:09.828225+0000","last_unstale":"2024-03-30T01:42:12.877240+0000","last_undegraded":"2024-03-30T01:42:12.877240+0000","last_fullsized":"2024-03-30T01:42:12.877240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875755+0000","last_change":"2024-03-30T01:42:10.286900+0000","last_active":"2024-03-30T01:42:12.875755+0000","last_peered":"2024-03-30T01:42:12.875755+0000","last_clean":"2024-03-30T01:42:12.875755+0000","last_became_active":"2024-03-30T01:42:10.286598+0000","last_became_peered":"2024-03-30T01:42:10.286598+0000","last_unstale":"2024-03-30T01:42:12.875755+0000","last_undegraded":"2024-03-30T01:42:12.875755+0000","last_fullsized":"2024-03-30T01:42:12.875755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:46.592007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.805879+0000","last_change":"2024-03-30T01:42:22.805879+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.805879+0000","last_undegraded":"2024-03-30T01:42:22.805879+0000","last_fullsized":"2024-03-30T01:42:22.805879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:57.756956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875729+0000","last_change":"2024-03-30T01:42:09.823221+0000","last_active":"2024-03-30T01:42:12.875729+0000","last_peered":"2024-03-30T01:42:12.875729+0000","last_clean":"2024-03-30T01:42:12.875729+0000","last_became_active":"2024-03-30T01:42:09.822882+0000","last_became_peered":"2024-03-30T01:42:09.822882+0000","last_unstale":"2024-03-30T01:42:12.875729+0000","last_undegraded":"2024-03-30T01:42:12.875729+0000","last_fullsized":"2024-03-30T01:42:12.875729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:00:31.185685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874279+0000","last_change":"2024-03-30T01:42:10.304085+0000","last_active":"2024-03-30T01:42:12.874279+0000","last_peered":"2024-03-30T01:42:12.874279+0000","last_clean":"2024-03-30T01:42:12.874279+0000","last_became_active":"2024-03-30T01:42:10.303817+0000","last_became_peered":"2024-03-30T01:42:10.303817+0000","last_unstale":"2024-03-30T01:42:12.874279+0000","last_undegraded":"2024-03-30T01:42:12.874279+0000","last_fullsized":"2024-03-30T01:42:12.874279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814798+0000","last_change":"2024-03-30T01:42:22.814798+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814798+0000","last_undegraded":"2024-03-30T01:42:22.814798+0000","last_fullsized":"2024-03-30T01:42:22.814798+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:06.958915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874246+0000","last_change":"2024-03-30T01:42:09.829713+0000","last_active":"2024-03-30T01:42:12.874246+0000","last_peered":"2024-03-30T01:42:12.874246+0000","last_clean":"2024-03-30T01:42:12.874246+0000","last_became_active":"2024-03-30T01:42:09.829375+0000","last_became_peered":"2024-03-30T01:42:09.829375+0000","last_unstale":"2024-03-30T01:42:12.874246+0000","last_undegraded":"2024-03-30T01:42:12.874246+0000","last_fullsized":"2024-03-30T01:42:12.874246+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:49:51.601663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877472+0000","last_change":"2024-03-30T01:42:10.295516+0000","last_active":"2024-03-30T01:42:12.877472+0000","last_peered":"2024-03-30T01:42:12.877472+0000","last_clean":"2024-03-30T01:42:12.877472+0000","last_became_active":"2024-03-30T01:42:10.295133+0000","last_became_peered":"2024-03-30T01:42:10.295133+0000","last_unstale":"2024-03-30T01:42:12.877472+0000","last_undegraded":"2024-03-30T01:42:12.877472+0000","last_fullsized":"2024-03-30T01:42:12.877472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812100+0000","last_change":"2024-03-30T01:42:22.812100+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812100+0000","last_undegraded":"2024-03-30T01:42:22.812100+0000","last_fullsized":"2024-03-30T01:42:22.812100+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:46.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877549+0000","last_change":"2024-03-30T01:42:09.826895+0000","last_active":"2024-03-30T01:42:12.877549+0000","last_peered":"2024-03-30T01:42:12.877549+0000","last_clean":"2024-03-30T01:42:12.877549+0000","last_became_active":"2024-03-30T01:42:09.826609+0000","last_became_peered":"2024-03-30T01:42:09.826609+0000","last_unstale":"2024-03-30T01:42:12.877549+0000","last_undegraded":"2024-03-30T01:42:12.877549+0000","last_fullsized":"2024-03-30T01:42:12.877549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:42:59.036073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876887+0000","last_change":"2024-03-30T01:42:10.291564+0000","last_active":"2024-03-30T01:42:12.876887+0000","last_peered":"2024-03-30T01:42:12.876887+0000","last_clean":"2024-03-30T01:42:12.876887+0000","last_became_active":"2024-03-30T01:42:10.291285+0000","last_became_peered":"2024-03-30T01:42:10.291285+0000","last_unstale":"2024-03-30T01:42:12.876887+0000","last_undegraded":"2024-03-30T01:42:12.876887+0000","last_fullsized":"2024-03-30T01:42:12.876887+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812173+0000","last_change":"2024-03-30T01:42:22.812173+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812173+0000","last_undegraded":"2024-03-30T01:42:22.812173+0000","last_fullsized":"2024-03-30T01:42:22.812173+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:49.116227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874199+0000","last_change":"2024-03-30T01:42:09.831869+0000","last_active":"2024-03-30T01:42:12.874199+0000","last_peered":"2024-03-30T01:42:12.874199+0000","last_clean":"2024-03-30T01:42:12.874199+0000","last_became_active":"2024-03-30T01:42:09.831562+0000","last_became_peered":"2024-03-30T01:42:09.831562+0000","last_unstale":"2024-03-30T01:42:12.874199+0000","last_undegraded":"2024-03-30T01:42:12.874199+0000","last_fullsized":"2024-03-30T01:42:12.874199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:35.938001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876785+0000","last_change":"2024-03-30T01:42:10.288399+0000","last_active":"2024-03-30T01:42:12.876785+0000","last_peered":"2024-03-30T01:42:12.876785+0000","last_clean":"2024-03-30T01:42:12.876785+0000","last_became_active":"2024-03-30T01:42:10.287890+0000","last_became_peered":"2024-03-30T01:42:10.287890+0000","last_unstale":"2024-03-30T01:42:12.876785+0000","last_undegraded":"2024-03-30T01:42:12.876785+0000","last_fullsized":"2024-03-30T01:42:12.876785+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812225+0000","last_change":"2024-03-30T01:42:22.812225+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812225+0000","last_undegraded":"2024-03-30T01:42:22.812225+0000","last_fullsized":"2024-03-30T01:42:22.812225+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:59.368657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.880961+0000","last_change":"2024-03-30T01:42:09.820451+0000","last_active":"2024-03-30T01:42:12.880961+0000","last_peered":"2024-03-30T01:42:12.880961+0000","last_clean":"2024-03-30T01:42:12.880961+0000","last_became_active":"2024-03-30T01:42:09.820152+0000","last_became_peered":"2024-03-30T01:42:09.820152+0000","last_unstale":"2024-03-30T01:42:12.880961+0000","last_undegraded":"2024-03-30T01:42:12.880961+0000","last_fullsized":"2024-03-30T01:42:12.880961+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:04.153303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874321+0000","last_change":"2024-03-30T01:42:10.279681+0000","last_active":"2024-03-30T01:42:12.874321+0000","last_peered":"2024-03-30T01:42:12.874321+0000","last_clean":"2024-03-30T01:42:12.874321+0000","last_became_active":"2024-03-30T01:42:10.279333+0000","last_became_peered":"2024-03-30T01:42:10.279333+0000","last_unstale":"2024-03-30T01:42:12.874321+0000","last_undegraded":"2024-03-30T01:42:12.874321+0000","last_fullsized":"2024-03-30T01:42:12.874321+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814765+0000","last_change":"2024-03-30T01:42:22.814765+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814765+0000","last_undegraded":"2024-03-30T01:42:22.814765+0000","last_fullsized":"2024-03-30T01:42:22.814765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.379389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877207+0000","last_change":"2024-03-30T01:42:09.829194+0000","last_active":"2024-03-30T01:42:12.877207+0000","last_peered":"2024-03-30T01:42:12.877207+0000","last_clean":"2024-03-30T01:42:12.877207+0000","last_became_active":"2024-03-30T01:42:09.828257+0000","last_became_peered":"2024-03-30T01:42:09.828257+0000","last_unstale":"2024-03-30T01:42:12.877207+0000","last_undegraded":"2024-03-30T01:42:12.877207+0000","last_fullsized":"2024-03-30T01:42:12.877207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876760+0000","last_change":"2024-03-30T01:42:10.288326+0000","last_active":"2024-03-30T01:42:12.876760+0000","last_peered":"2024-03-30T01:42:12.876760+0000","last_clean":"2024-03-30T01:42:12.876760+0000","last_became_active":"2024-03-30T01:42:10.288050+0000","last_became_peered":"2024-03-30T01:42:10.288050+0000","last_unstale":"2024-03-30T01:42:12.876760+0000","last_undegraded":"2024-03-30T01:42:12.876760+0000","last_fullsized":"2024-03-30T01:42:12.876760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809500+0000","last_change":"2024-03-30T01:42:22.809500+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809500+0000","last_undegraded":"2024-03-30T01:42:22.809500+0000","last_fullsized":"2024-03-30T01:42:22.809500+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:20.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876861+0000","last_change":"2024-03-30T01:42:09.832318+0000","last_active":"2024-03-30T01:42:12.876861+0000","last_peered":"2024-03-30T01:42:12.876861+0000","last_clean":"2024-03-30T01:42:12.876861+0000","last_became_active":"2024-03-30T01:42:09.831964+0000","last_became_peered":"2024-03-30T01:42:09.831964+0000","last_unstale":"2024-03-30T01:42:12.876861+0000","last_undegraded":"2024-03-30T01:42:12.876861+0000","last_fullsized":"2024-03-30T01:42:12.876861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:05.700970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877072+0000","last_change":"2024-03-30T01:42:10.285053+0000","last_active":"2024-03-30T01:42:12.877072+0000","last_peered":"2024-03-30T01:42:12.877072+0000","last_clean":"2024-03-30T01:42:12.877072+0000","last_became_active":"2024-03-30T01:42:10.284814+0000","last_became_peered":"2024-03-30T01:42:10.284814+0000","last_unstale":"2024-03-30T01:42:12.877072+0000","last_undegraded":"2024-03-30T01:42:12.877072+0000","last_fullsized":"2024-03-30T01:42:12.877072+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:28.875197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.807918+0000","last_change":"2024-03-30T01:42:22.807918+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.807918+0000","last_undegraded":"2024-03-30T01:42:22.807918+0000","last_fullsized":"2024-03-30T01:42:22.807918+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:29:59.201428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:18.899337+0000","last_change":"2024-03-30T01:42:09.823152+0000","last_active":"2024-03-30T01:42:18.899337+0000","last_peered":"2024-03-30T01:42:18.899337+0000","last_clean":"2024-03-30T01:42:18.899337+0000","last_became_active":"2024-03-30T01:42:09.822826+0000","last_became_peered":"2024-03-30T01:42:09.822826+0000","last_unstale":"2024-03-30T01:42:18.899337+0000","last_undegraded":"2024-03-30T01:42:18.899337+0000","last_fullsized":"2024-03-30T01:42:18.899337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:17.799215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874291+0000","last_change":"2024-03-30T01:42:10.296944+0000","last_active":"2024-03-30T01:42:12.874291+0000","last_peered":"2024-03-30T01:42:12.874291+0000","last_clean":"2024-03-30T01:42:12.874291+0000","last_became_active":"2024-03-30T01:42:10.296714+0000","last_became_peered":"2024-03-30T01:42:10.296714+0000","last_unstale":"2024-03-30T01:42:12.874291+0000","last_undegraded":"2024-03-30T01:42:12.874291+0000","last_fullsized":"2024-03-30T01:42:12.874291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815307+0000","last_change":"2024-03-30T01:42:22.815307+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815307+0000","last_undegraded":"2024-03-30T01:42:22.815307+0000","last_fullsized":"2024-03-30T01:42:22.815307+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877496+0000","last_change":"2024-03-30T01:42:09.833439+0000","last_active":"2024-03-30T01:42:12.877496+0000","last_peered":"2024-03-30T01:42:12.877496+0000","last_clean":"2024-03-30T01:42:12.877496+0000","last_became_active":"2024-03-30T01:42:09.833137+0000","last_became_peered":"2024-03-30T01:42:09.833137+0000","last_unstale":"2024-03-30T01:42:12.877496+0000","last_undegraded":"2024-03-30T01:42:12.877496+0000","last_fullsized":"2024-03-30T01:42:12.877496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877264+0000","last_change":"2024-03-30T01:42:10.292182+0000","last_active":"2024-03-30T01:42:12.877264+0000","last_peered":"2024-03-30T01:42:12.877264+0000","last_clean":"2024-03-30T01:42:12.877264+0000","last_became_active":"2024-03-30T01:42:10.291868+0000","last_became_peered":"2024-03-30T01:42:10.291868+0000","last_unstale":"2024-03-30T01:42:12.877264+0000","last_undegraded":"2024-03-30T01:42:12.877264+0000","last_fullsized":"2024-03-30T01:42:12.877264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.807910+0000","last_change":"2024-03-30T01:42:22.807910+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.807910+0000","last_undegraded":"2024-03-30T01:42:22.807910+0000","last_fullsized":"2024-03-30T01:42:22.807910+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:12.384136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877526+0000","last_change":"2024-03-30T01:42:09.826561+0000","last_active":"2024-03-30T01:42:12.877526+0000","last_peered":"2024-03-30T01:42:12.877526+0000","last_clean":"2024-03-30T01:42:12.877526+0000","last_became_active":"2024-03-30T01:42:09.826059+0000","last_became_peered":"2024-03-30T01:42:09.826059+0000","last_unstale":"2024-03-30T01:42:12.877526+0000","last_undegraded":"2024-03-30T01:42:12.877526+0000","last_fullsized":"2024-03-30T01:42:12.877526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874434+0000","last_change":"2024-03-30T01:42:10.284755+0000","last_active":"2024-03-30T01:42:12.874434+0000","last_peered":"2024-03-30T01:42:12.874434+0000","last_clean":"2024-03-30T01:42:12.874434+0000","last_became_active":"2024-03-30T01:42:10.284421+0000","last_became_peered":"2024-03-30T01:42:10.284421+0000","last_unstale":"2024-03-30T01:42:12.874434+0000","last_undegraded":"2024-03-30T01:42:12.874434+0000","last_fullsized":"2024-03-30T01:42:12.874434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.810039+0000","last_change":"2024-03-30T01:42:22.810039+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.810039+0000","last_undegraded":"2024-03-30T01:42:22.810039+0000","last_fullsized":"2024-03-30T01:42:22.810039+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:44.146196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877020+0000","last_change":"2024-03-30T01:42:09.828624+0000","last_active":"2024-03-30T01:42:12.877020+0000","last_peered":"2024-03-30T01:42:12.877020+0000","last_clean":"2024-03-30T01:42:12.877020+0000","last_became_active":"2024-03-30T01:42:09.828341+0000","last_became_peered":"2024-03-30T01:42:09.828341+0000","last_unstale":"2024-03-30T01:42:12.877020+0000","last_undegraded":"2024-03-30T01:42:12.877020+0000","last_fullsized":"2024-03-30T01:42:12.877020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:32.138427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875705+0000","last_change":"2024-03-30T01:42:10.295259+0000","last_active":"2024-03-30T01:42:12.875705+0000","last_peered":"2024-03-30T01:42:12.875705+0000","last_clean":"2024-03-30T01:42:12.875705+0000","last_became_active":"2024-03-30T01:42:10.295054+0000","last_became_peered":"2024-03-30T01:42:10.295054+0000","last_unstale":"2024-03-30T01:42:12.875705+0000","last_undegraded":"2024-03-30T01:42:12.875705+0000","last_fullsized":"2024-03-30T01:42:12.875705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:36.942583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.808461+0000","last_change":"2024-03-30T01:42:22.808461+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.808461+0000","last_undegraded":"2024-03-30T01:42:22.808461+0000","last_fullsized":"2024-03-30T01:42:22.808461+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:54:59.010522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874117+0000","last_change":"2024-03-30T01:42:09.820512+0000","last_active":"2024-03-30T01:42:12.874117+0000","last_peered":"2024-03-30T01:42:12.874117+0000","last_clean":"2024-03-30T01:42:12.874117+0000","last_became_active":"2024-03-30T01:42:09.820253+0000","last_became_peered":"2024-03-30T01:42:09.820253+0000","last_unstale":"2024-03-30T01:42:12.874117+0000","last_undegraded":"2024-03-30T01:42:12.874117+0000","last_fullsized":"2024-03-30T01:42:12.874117+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.455801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876834+0000","last_change":"2024-03-30T01:42:10.291872+0000","last_active":"2024-03-30T01:42:12.876834+0000","last_peered":"2024-03-30T01:42:12.876834+0000","last_clean":"2024-03-30T01:42:12.876834+0000","last_became_active":"2024-03-30T01:42:10.291199+0000","last_became_peered":"2024-03-30T01:42:10.291199+0000","last_unstale":"2024-03-30T01:42:12.876834+0000","last_undegraded":"2024-03-30T01:42:12.876834+0000","last_fullsized":"2024-03-30T01:42:12.876834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815299+0000","last_change":"2024-03-30T01:42:22.815299+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815299+0000","last_undegraded":"2024-03-30T01:42:22.815299+0000","last_fullsized":"2024-03-30T01:42:22.815299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877166+0000","last_change":"2024-03-30T01:42:09.826326+0000","last_active":"2024-03-30T01:42:12.877166+0000","last_peered":"2024-03-30T01:42:12.877166+0000","last_clean":"2024-03-30T01:42:12.877166+0000","last_became_active":"2024-03-30T01:42:09.826050+0000","last_became_peered":"2024-03-30T01:42:09.826050+0000","last_unstale":"2024-03-30T01:42:12.877166+0000","last_undegraded":"2024-03-30T01:42:12.877166+0000","last_fullsized":"2024-03-30T01:42:12.877166+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874163+0000","last_change":"2024-03-30T01:42:10.303359+0000","last_active":"2024-03-30T01:42:12.874163+0000","last_peered":"2024-03-30T01:42:12.874163+0000","last_clean":"2024-03-30T01:42:12.874163+0000","last_became_active":"2024-03-30T01:42:10.303039+0000","last_became_peered":"2024-03-30T01:42:10.303039+0000","last_unstale":"2024-03-30T01:42:12.874163+0000","last_undegraded":"2024-03-30T01:42:12.874163+0000","last_fullsized":"2024-03-30T01:42:12.874163+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.815535+0000","last_change":"2024-03-30T01:42:22.815535+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.815535+0000","last_undegraded":"2024-03-30T01:42:22.815535+0000","last_fullsized":"2024-03-30T01:42:22.815535+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:36.452658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874345+0000","last_change":"2024-03-30T01:42:09.831981+0000","last_active":"2024-03-30T01:42:12.874345+0000","last_peered":"2024-03-30T01:42:12.874345+0000","last_clean":"2024-03-30T01:42:12.874345+0000","last_became_active":"2024-03-30T01:42:09.831647+0000","last_became_peered":"2024-03-30T01:42:09.831647+0000","last_unstale":"2024-03-30T01:42:12.874345+0000","last_undegraded":"2024-03-30T01:42:12.874345+0000","last_fullsized":"2024-03-30T01:42:12.874345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:02.883477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877386+0000","last_change":"2024-03-30T01:42:10.289213+0000","last_active":"2024-03-30T01:42:12.877386+0000","last_peered":"2024-03-30T01:42:12.877386+0000","last_clean":"2024-03-30T01:42:12.877386+0000","last_became_active":"2024-03-30T01:42:10.288677+0000","last_became_peered":"2024-03-30T01:42:10.288677+0000","last_unstale":"2024-03-30T01:42:12.877386+0000","last_undegraded":"2024-03-30T01:42:12.877386+0000","last_fullsized":"2024-03-30T01:42:12.877386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.814739+0000","last_change":"2024-03-30T01:42:22.814739+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.814739+0000","last_undegraded":"2024-03-30T01:42:22.814739+0000","last_fullsized":"2024-03-30T01:42:22.814739+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:32.897671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877446+0000","last_change":"2024-03-30T01:42:09.831496+0000","last_active":"2024-03-30T01:42:12.877446+0000","last_peered":"2024-03-30T01:42:12.877446+0000","last_clean":"2024-03-30T01:42:12.877446+0000","last_became_active":"2024-03-30T01:42:09.831197+0000","last_became_peered":"2024-03-30T01:42:09.831197+0000","last_unstale":"2024-03-30T01:42:12.877446+0000","last_undegraded":"2024-03-30T01:42:12.877446+0000","last_fullsized":"2024-03-30T01:42:12.877446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877126+0000","last_change":"2024-03-30T01:42:10.287526+0000","last_active":"2024-03-30T01:42:12.877126+0000","last_peered":"2024-03-30T01:42:12.877126+0000","last_clean":"2024-03-30T01:42:12.877126+0000","last_became_active":"2024-03-30T01:42:10.287255+0000","last_became_peered":"2024-03-30T01:42:10.287255+0000","last_unstale":"2024-03-30T01:42:12.877126+0000","last_undegraded":"2024-03-30T01:42:12.877126+0000","last_fullsized":"2024-03-30T01:42:12.877126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812195+0000","last_change":"2024-03-30T01:42:22.812195+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812195+0000","last_undegraded":"2024-03-30T01:42:22.812195+0000","last_fullsized":"2024-03-30T01:42:22.812195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:35.849177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874220+0000","last_change":"2024-03-30T01:42:09.824809+0000","last_active":"2024-03-30T01:42:12.874220+0000","last_peered":"2024-03-30T01:42:12.874220+0000","last_clean":"2024-03-30T01:42:12.874220+0000","last_became_active":"2024-03-30T01:42:09.824493+0000","last_became_peered":"2024-03-30T01:42:09.824493+0000","last_unstale":"2024-03-30T01:42:12.874220+0000","last_undegraded":"2024-03-30T01:42:12.874220+0000","last_fullsized":"2024-03-30T01:42:12.874220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877000+0000","last_change":"2024-03-30T01:42:10.291979+0000","last_active":"2024-03-30T01:42:12.877000+0000","last_peered":"2024-03-30T01:42:12.877000+0000","last_clean":"2024-03-30T01:42:12.877000+0000","last_became_active":"2024-03-30T01:42:10.291708+0000","last_became_peered":"2024-03-30T01:42:10.291708+0000","last_unstale":"2024-03-30T01:42:12.877000+0000","last_undegraded":"2024-03-30T01:42:12.877000+0000","last_fullsized":"2024-03-30T01:42:12.877000+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809931+0000","last_change":"2024-03-30T01:42:22.809931+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809931+0000","last_undegraded":"2024-03-30T01:42:22.809931+0000","last_fullsized":"2024-03-30T01:42:22.809931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877310+0000","last_change":"2024-03-30T01:42:09.829349+0000","last_active":"2024-03-30T01:42:12.877310+0000","last_peered":"2024-03-30T01:42:12.877310+0000","last_clean":"2024-03-30T01:42:12.877310+0000","last_became_active":"2024-03-30T01:42:09.828931+0000","last_became_peered":"2024-03-30T01:42:09.828931+0000","last_unstale":"2024-03-30T01:42:12.877310+0000","last_undegraded":"2024-03-30T01:42:12.877310+0000","last_fullsized":"2024-03-30T01:42:12.877310+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874233+0000","last_change":"2024-03-30T01:42:10.300292+0000","last_active":"2024-03-30T01:42:12.874233+0000","last_peered":"2024-03-30T01:42:12.874233+0000","last_clean":"2024-03-30T01:42:12.874233+0000","last_became_active":"2024-03-30T01:42:10.299867+0000","last_became_peered":"2024-03-30T01:42:10.299867+0000","last_unstale":"2024-03-30T01:42:12.874233+0000","last_undegraded":"2024-03-30T01:42:12.874233+0000","last_fullsized":"2024-03-30T01:42:12.874233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.812740+0000","last_change":"2024-03-30T01:42:22.812740+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.812740+0000","last_undegraded":"2024-03-30T01:42:22.812740+0000","last_fullsized":"2024-03-30T01:42:22.812740+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:13.222722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877289+0000","last_change":"2024-03-30T01:42:09.829899+0000","last_active":"2024-03-30T01:42:12.877289+0000","last_peered":"2024-03-30T01:42:12.877289+0000","last_clean":"2024-03-30T01:42:12.877289+0000","last_became_active":"2024-03-30T01:42:09.829601+0000","last_became_peered":"2024-03-30T01:42:09.829601+0000","last_unstale":"2024-03-30T01:42:12.877289+0000","last_undegraded":"2024-03-30T01:42:12.877289+0000","last_fullsized":"2024-03-30T01:42:12.877289+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:30.887459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877284+0000","last_change":"2024-03-30T01:42:10.298381+0000","last_active":"2024-03-30T01:42:12.877284+0000","last_peered":"2024-03-30T01:42:12.877284+0000","last_clean":"2024-03-30T01:42:12.877284+0000","last_became_active":"2024-03-30T01:42:10.298109+0000","last_became_peered":"2024-03-30T01:42:10.298109+0000","last_unstale":"2024-03-30T01:42:12.877284+0000","last_undegraded":"2024-03-30T01:42:12.877284+0000","last_fullsized":"2024-03-30T01:42:12.877284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.804764+0000","last_change":"2024-03-30T01:42:22.804764+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.804764+0000","last_undegraded":"2024-03-30T01:42:22.804764+0000","last_fullsized":"2024-03-30T01:42:22.804764+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881147+0000","last_change":"2024-03-30T01:42:09.824850+0000","last_active":"2024-03-30T01:42:12.881147+0000","last_peered":"2024-03-30T01:42:12.881147+0000","last_clean":"2024-03-30T01:42:12.881147+0000","last_became_active":"2024-03-30T01:42:09.824537+0000","last_became_peered":"2024-03-30T01:42:09.824537+0000","last_unstale":"2024-03-30T01:42:12.881147+0000","last_undegraded":"2024-03-30T01:42:12.881147+0000","last_fullsized":"2024-03-30T01:42:12.881147+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:17.759503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874391+0000","last_change":"2024-03-30T01:42:10.285919+0000","last_active":"2024-03-30T01:42:12.874391+0000","last_peered":"2024-03-30T01:42:12.874391+0000","last_clean":"2024-03-30T01:42:12.874391+0000","last_became_active":"2024-03-30T01:42:10.285629+0000","last_became_peered":"2024-03-30T01:42:10.285629+0000","last_unstale":"2024-03-30T01:42:12.874391+0000","last_undegraded":"2024-03-30T01:42:12.874391+0000","last_fullsized":"2024-03-30T01:42:12.874391+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-03-30T01:42:22.809889+0000","last_change":"2024-03-30T01:42:22.809889+0000","last_active":"2024-03-30T01:42:22.795162+0000","last_peered":"2024-03-30T01:42:22.795162+0000","last_clean":"2024-03-30T01:42:22.795162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:22.809889+0000","last_undegraded":"2024-03-30T01:42:22.809889+0000","last_fullsized":"2024-03-30T01:42:22.809889+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:35.672193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877110+0000","last_change":"2024-03-30T01:42:10.297189+0000","last_active":"2024-03-30T01:42:12.877110+0000","last_peered":"2024-03-30T01:42:12.877110+0000","last_clean":"2024-03-30T01:42:12.877110+0000","last_became_active":"2024-03-30T01:42:10.296821+0000","last_became_peered":"2024-03-30T01:42:10.296821+0000","last_unstale":"2024-03-30T01:42:12.877110+0000","last_undegraded":"2024-03-30T01:42:12.877110+0000","last_fullsized":"2024-03-30T01:42:12.877110+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874246+0000","last_change":"2024-03-30T01:42:10.274200+0000","last_active":"2024-03-30T01:42:12.874246+0000","last_peered":"2024-03-30T01:42:12.874246+0000","last_clean":"2024-03-30T01:42:12.874246+0000","last_became_active":"2024-03-30T01:42:10.273923+0000","last_became_peered":"2024-03-30T01:42:10.273923+0000","last_unstale":"2024-03-30T01:42:12.874246+0000","last_undegraded":"2024-03-30T01:42:12.874246+0000","last_fullsized":"2024-03-30T01:42:12.874246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:44.766475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.880992+0000","last_change":"2024-03-30T01:42:10.290078+0000","last_active":"2024-03-30T01:42:12.880992+0000","last_peered":"2024-03-30T01:42:12.880992+0000","last_clean":"2024-03-30T01:42:12.880992+0000","last_became_active":"2024-03-30T01:42:10.289127+0000","last_became_peered":"2024-03-30T01:42:10.289127+0000","last_unstale":"2024-03-30T01:42:12.880992+0000","last_undegraded":"2024-03-30T01:42:12.880992+0000","last_fullsized":"2024-03-30T01:42:12.880992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:36.303285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881034+0000","last_change":"2024-03-30T01:42:10.291099+0000","last_active":"2024-03-30T01:42:12.881034+0000","last_peered":"2024-03-30T01:42:12.881034+0000","last_clean":"2024-03-30T01:42:12.881034+0000","last_became_active":"2024-03-30T01:42:10.290802+0000","last_became_peered":"2024-03-30T01:42:10.290802+0000","last_unstale":"2024-03-30T01:42:12.881034+0000","last_undegraded":"2024-03-30T01:42:12.881034+0000","last_fullsized":"2024-03-30T01:42:12.881034+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:13.070677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877459+0000","last_change":"2024-03-30T01:42:10.288633+0000","last_active":"2024-03-30T01:42:12.877459+0000","last_peered":"2024-03-30T01:42:12.877459+0000","last_clean":"2024-03-30T01:42:12.877459+0000","last_became_active":"2024-03-30T01:42:10.288326+0000","last_became_peered":"2024-03-30T01:42:10.288326+0000","last_unstale":"2024-03-30T01:42:12.877459+0000","last_undegraded":"2024-03-30T01:42:12.877459+0000","last_fullsized":"2024-03-30T01:42:12.877459+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881079+0000","last_change":"2024-03-30T01:42:10.274845+0000","last_active":"2024-03-30T01:42:12.881079+0000","last_peered":"2024-03-30T01:42:12.881079+0000","last_clean":"2024-03-30T01:42:12.881079+0000","last_became_active":"2024-03-30T01:42:10.274566+0000","last_became_peered":"2024-03-30T01:42:10.274566+0000","last_unstale":"2024-03-30T01:42:12.881079+0000","last_undegraded":"2024-03-30T01:42:12.881079+0000","last_fullsized":"2024-03-30T01:42:12.881079+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:32.801396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877322+0000","last_change":"2024-03-30T01:42:10.290378+0000","last_active":"2024-03-30T01:42:12.877322+0000","last_peered":"2024-03-30T01:42:12.877322+0000","last_clean":"2024-03-30T01:42:12.877322+0000","last_became_active":"2024-03-30T01:42:10.290022+0000","last_became_peered":"2024-03-30T01:42:10.290022+0000","last_unstale":"2024-03-30T01:42:12.877322+0000","last_undegraded":"2024-03-30T01:42:12.877322+0000","last_fullsized":"2024-03-30T01:42:12.877322+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881124+0000","last_change":"2024-03-30T01:42:10.285378+0000","last_active":"2024-03-30T01:42:12.881124+0000","last_peered":"2024-03-30T01:42:12.881124+0000","last_clean":"2024-03-30T01:42:12.881124+0000","last_became_active":"2024-03-30T01:42:10.285105+0000","last_became_peered":"2024-03-30T01:42:10.285105+0000","last_unstale":"2024-03-30T01:42:12.881124+0000","last_undegraded":"2024-03-30T01:42:12.881124+0000","last_fullsized":"2024-03-30T01:42:12.881124+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:52.824468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875599+0000","last_change":"2024-03-30T01:42:10.286887+0000","last_active":"2024-03-30T01:42:12.875599+0000","last_peered":"2024-03-30T01:42:12.875599+0000","last_clean":"2024-03-30T01:42:12.875599+0000","last_became_active":"2024-03-30T01:42:10.278107+0000","last_became_peered":"2024-03-30T01:42:10.278107+0000","last_unstale":"2024-03-30T01:42:12.875599+0000","last_undegraded":"2024-03-30T01:42:12.875599+0000","last_fullsized":"2024-03-30T01:42:12.875599+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874268+0000","last_change":"2024-03-30T01:42:10.281331+0000","last_active":"2024-03-30T01:42:12.874268+0000","last_peered":"2024-03-30T01:42:12.874268+0000","last_clean":"2024-03-30T01:42:12.874268+0000","last_became_active":"2024-03-30T01:42:10.281011+0000","last_became_peered":"2024-03-30T01:42:10.281011+0000","last_unstale":"2024-03-30T01:42:12.874268+0000","last_undegraded":"2024-03-30T01:42:12.874268+0000","last_fullsized":"2024-03-30T01:42:12.874268+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874152+0000","last_change":"2024-03-30T01:42:10.299194+0000","last_active":"2024-03-30T01:42:12.874152+0000","last_peered":"2024-03-30T01:42:12.874152+0000","last_clean":"2024-03-30T01:42:12.874152+0000","last_became_active":"2024-03-30T01:42:10.298900+0000","last_became_peered":"2024-03-30T01:42:10.298900+0000","last_unstale":"2024-03-30T01:42:12.874152+0000","last_undegraded":"2024-03-30T01:42:12.874152+0000","last_fullsized":"2024-03-30T01:42:12.874152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877431+0000","last_change":"2024-03-30T01:42:10.304038+0000","last_active":"2024-03-30T01:42:12.877431+0000","last_peered":"2024-03-30T01:42:12.877431+0000","last_clean":"2024-03-30T01:42:12.877431+0000","last_became_active":"2024-03-30T01:42:10.303747+0000","last_became_peered":"2024-03-30T01:42:10.303747+0000","last_unstale":"2024-03-30T01:42:12.877431+0000","last_undegraded":"2024-03-30T01:42:12.877431+0000","last_fullsized":"2024-03-30T01:42:12.877431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881058+0000","last_change":"2024-03-30T01:42:10.300160+0000","last_active":"2024-03-30T01:42:12.881058+0000","last_peered":"2024-03-30T01:42:12.881058+0000","last_clean":"2024-03-30T01:42:12.881058+0000","last_became_active":"2024-03-30T01:42:10.299728+0000","last_became_peered":"2024-03-30T01:42:10.299728+0000","last_unstale":"2024-03-30T01:42:12.881058+0000","last_undegraded":"2024-03-30T01:42:12.881058+0000","last_fullsized":"2024-03-30T01:42:12.881058+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875715+0000","last_change":"2024-03-30T01:42:10.292351+0000","last_active":"2024-03-30T01:42:12.875715+0000","last_peered":"2024-03-30T01:42:12.875715+0000","last_clean":"2024-03-30T01:42:12.875715+0000","last_became_active":"2024-03-30T01:42:10.292040+0000","last_became_peered":"2024-03-30T01:42:10.292040+0000","last_unstale":"2024-03-30T01:42:12.875715+0000","last_undegraded":"2024-03-30T01:42:12.875715+0000","last_fullsized":"2024-03-30T01:42:12.875715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:09.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.881101+0000","last_change":"2024-03-30T01:42:10.300393+0000","last_active":"2024-03-30T01:42:12.881101+0000","last_peered":"2024-03-30T01:42:12.881101+0000","last_clean":"2024-03-30T01:42:12.881101+0000","last_became_active":"2024-03-30T01:42:10.299795+0000","last_became_peered":"2024-03-30T01:42:10.299795+0000","last_unstale":"2024-03-30T01:42:12.881101+0000","last_undegraded":"2024-03-30T01:42:12.881101+0000","last_fullsized":"2024-03-30T01:42:12.881101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:21.256043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874478+0000","last_change":"2024-03-30T01:42:10.285733+0000","last_active":"2024-03-30T01:42:12.874478+0000","last_peered":"2024-03-30T01:42:12.874478+0000","last_clean":"2024-03-30T01:42:12.874478+0000","last_became_active":"2024-03-30T01:42:10.285428+0000","last_became_peered":"2024-03-30T01:42:10.285428+0000","last_unstale":"2024-03-30T01:42:12.874478+0000","last_undegraded":"2024-03-30T01:42:12.874478+0000","last_fullsized":"2024-03-30T01:42:12.874478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874143+0000","last_change":"2024-03-30T01:42:10.284419+0000","last_active":"2024-03-30T01:42:12.874143+0000","last_peered":"2024-03-30T01:42:12.874143+0000","last_clean":"2024-03-30T01:42:12.874143+0000","last_became_active":"2024-03-30T01:42:10.284098+0000","last_became_peered":"2024-03-30T01:42:10.284098+0000","last_unstale":"2024-03-30T01:42:12.874143+0000","last_undegraded":"2024-03-30T01:42:12.874143+0000","last_fullsized":"2024-03-30T01:42:12.874143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877370+0000","last_change":"2024-03-30T01:42:10.293953+0000","last_active":"2024-03-30T01:42:12.877370+0000","last_peered":"2024-03-30T01:42:12.877370+0000","last_clean":"2024-03-30T01:42:12.877370+0000","last_became_active":"2024-03-30T01:42:10.293641+0000","last_became_peered":"2024-03-30T01:42:10.293641+0000","last_unstale":"2024-03-30T01:42:12.877370+0000","last_undegraded":"2024-03-30T01:42:12.877370+0000","last_fullsized":"2024-03-30T01:42:12.877370+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.874206+0000","last_change":"2024-03-30T01:42:10.294667+0000","last_active":"2024-03-30T01:42:12.874206+0000","last_peered":"2024-03-30T01:42:12.874206+0000","last_clean":"2024-03-30T01:42:12.874206+0000","last_became_active":"2024-03-30T01:42:10.294330+0000","last_became_peered":"2024-03-30T01:42:10.294330+0000","last_unstale":"2024-03-30T01:42:12.874206+0000","last_undegraded":"2024-03-30T01:42:12.874206+0000","last_fullsized":"2024-03-30T01:42:12.874206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877421+0000","last_change":"2024-03-30T01:42:10.295808+0000","last_active":"2024-03-30T01:42:12.877421+0000","last_peered":"2024-03-30T01:42:12.877421+0000","last_clean":"2024-03-30T01:42:12.877421+0000","last_became_active":"2024-03-30T01:42:10.295460+0000","last_became_peered":"2024-03-30T01:42:10.295460+0000","last_unstale":"2024-03-30T01:42:12.877421+0000","last_undegraded":"2024-03-30T01:42:12.877421+0000","last_fullsized":"2024-03-30T01:42:12.877421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877196+0000","last_change":"2024-03-30T01:42:10.294103+0000","last_active":"2024-03-30T01:42:12.877196+0000","last_peered":"2024-03-30T01:42:12.877196+0000","last_clean":"2024-03-30T01:42:12.877196+0000","last_became_active":"2024-03-30T01:42:10.293833+0000","last_became_peered":"2024-03-30T01:42:10.293833+0000","last_unstale":"2024-03-30T01:42:12.877196+0000","last_undegraded":"2024-03-30T01:42:12.877196+0000","last_fullsized":"2024-03-30T01:42:12.877196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877485+0000","last_change":"2024-03-30T01:42:10.296160+0000","last_active":"2024-03-30T01:42:12.877485+0000","last_peered":"2024-03-30T01:42:12.877485+0000","last_clean":"2024-03-30T01:42:12.877485+0000","last_became_active":"2024-03-30T01:42:10.295867+0000","last_became_peered":"2024-03-30T01:42:10.295867+0000","last_unstale":"2024-03-30T01:42:12.877485+0000","last_undegraded":"2024-03-30T01:42:12.877485+0000","last_fullsized":"2024-03-30T01:42:12.877485+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877510+0000","last_change":"2024-03-30T01:42:10.295963+0000","last_active":"2024-03-30T01:42:12.877510+0000","last_peered":"2024-03-30T01:42:12.877510+0000","last_clean":"2024-03-30T01:42:12.877510+0000","last_became_active":"2024-03-30T01:42:10.295545+0000","last_became_peered":"2024-03-30T01:42:10.295545+0000","last_unstale":"2024-03-30T01:42:12.877510+0000","last_undegraded":"2024-03-30T01:42:12.877510+0000","last_fullsized":"2024-03-30T01:42:12.877510+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877312+0000","last_change":"2024-03-30T01:42:10.285717+0000","last_active":"2024-03-30T01:42:12.877312+0000","last_peered":"2024-03-30T01:42:12.877312+0000","last_clean":"2024-03-30T01:42:12.877312+0000","last_became_active":"2024-03-30T01:42:10.285437+0000","last_became_peered":"2024-03-30T01:42:10.285437+0000","last_unstale":"2024-03-30T01:42:12.877312+0000","last_undegraded":"2024-03-30T01:42:12.877312+0000","last_fullsized":"2024-03-30T01:42:12.877312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876706+0000","last_change":"2024-03-30T01:42:10.298731+0000","last_active":"2024-03-30T01:42:12.876706+0000","last_peered":"2024-03-30T01:42:12.876706+0000","last_clean":"2024-03-30T01:42:12.876706+0000","last_became_active":"2024-03-30T01:42:10.298525+0000","last_became_peered":"2024-03-30T01:42:10.298525+0000","last_unstale":"2024-03-30T01:42:12.876706+0000","last_undegraded":"2024-03-30T01:42:12.876706+0000","last_fullsized":"2024-03-30T01:42:12.876706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877398+0000","last_change":"2024-03-30T01:42:10.289068+0000","last_active":"2024-03-30T01:42:12.877398+0000","last_peered":"2024-03-30T01:42:12.877398+0000","last_clean":"2024-03-30T01:42:12.877398+0000","last_became_active":"2024-03-30T01:42:10.288282+0000","last_became_peered":"2024-03-30T01:42:10.288282+0000","last_unstale":"2024-03-30T01:42:12.877398+0000","last_undegraded":"2024-03-30T01:42:12.877398+0000","last_fullsized":"2024-03-30T01:42:12.877398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877189+0000","last_change":"2024-03-30T01:42:10.287001+0000","last_active":"2024-03-30T01:42:12.877189+0000","last_peered":"2024-03-30T01:42:12.877189+0000","last_clean":"2024-03-30T01:42:12.877189+0000","last_became_active":"2024-03-30T01:42:10.286700+0000","last_became_peered":"2024-03-30T01:42:10.286700+0000","last_unstale":"2024-03-30T01:42:12.877189+0000","last_undegraded":"2024-03-30T01:42:12.877189+0000","last_fullsized":"2024-03-30T01:42:12.877189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875664+0000","last_change":"2024-03-30T01:42:10.299868+0000","last_active":"2024-03-30T01:42:12.875664+0000","last_peered":"2024-03-30T01:42:12.875664+0000","last_clean":"2024-03-30T01:42:12.875664+0000","last_became_active":"2024-03-30T01:42:10.299612+0000","last_became_peered":"2024-03-30T01:42:10.299612+0000","last_unstale":"2024-03-30T01:42:12.875664+0000","last_undegraded":"2024-03-30T01:42:12.875664+0000","last_fullsized":"2024-03-30T01:42:12.875664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:49.471045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.875690+0000","last_change":"2024-03-30T01:42:10.294277+0000","last_active":"2024-03-30T01:42:12.875690+0000","last_peered":"2024-03-30T01:42:12.875690+0000","last_clean":"2024-03-30T01:42:12.875690+0000","last_became_active":"2024-03-30T01:42:10.293995+0000","last_became_peered":"2024-03-30T01:42:10.293995+0000","last_unstale":"2024-03-30T01:42:12.875690+0000","last_undegraded":"2024-03-30T01:42:12.875690+0000","last_fullsized":"2024-03-30T01:42:12.875690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:03.888060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.877217+0000","last_change":"2024-03-30T01:42:10.285050+0000","last_active":"2024-03-30T01:42:12.877217+0000","last_peered":"2024-03-30T01:42:12.877217+0000","last_clean":"2024-03-30T01:42:12.877217+0000","last_became_active":"2024-03-30T01:42:10.284771+0000","last_became_peered":"2024-03-30T01:42:10.284771+0000","last_unstale":"2024-03-30T01:42:12.877217+0000","last_undegraded":"2024-03-30T01:42:12.877217+0000","last_fullsized":"2024-03-30T01:42:12.877217+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876935+0000","last_change":"2024-03-30T01:42:10.294965+0000","last_active":"2024-03-30T01:42:12.876935+0000","last_peered":"2024-03-30T01:42:12.876935+0000","last_clean":"2024-03-30T01:42:12.876935+0000","last_became_active":"2024-03-30T01:42:10.294685+0000","last_became_peered":"2024-03-30T01:42:10.294685+0000","last_unstale":"2024-03-30T01:42:12.876935+0000","last_undegraded":"2024-03-30T01:42:12.876935+0000","last_fullsized":"2024-03-30T01:42:12.876935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:42:23.802198+0000","last_change":"2024-03-30T01:42:23.802198+0000","last_active":"2024-03-30T01:42:23.802198+0000","last_peered":"2024-03-30T01:42:23.802198+0000","last_clean":"2024-03-30T01:42:23.802198+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:42:23.802198+0000","last_undegraded":"2024-03-30T01:42:23.802198+0000","last_fullsized":"2024-03-30T01:42:23.802198+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T01:42:12.876977+0000","last_change":"2024-03-30T01:42:10.288258+0000","last_active":"2024-03-30T01:42:12.876977+0000","last_peered":"2024-03-30T01:42:12.876977+0000","last_clean":"2024-03-30T01:42:12.876977+0000","last_became_active":"2024-03-30T01:42:10.287993+0000","last_became_peered":"2024-03-30T01:42:10.287993+0000","last_unstale":"2024-03-30T01:42:12.876977+0000","last_undegraded":"2024-03-30T01:42:12.876977+0000","last_fullsized":"2024-03-30T01:42:12.876977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1060864,"data_stored":787628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.505},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.61199999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.56999999999999995}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.2009999999999996,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.41399999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.58299999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.47299999999999998}]},{"osd":1,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.3250000000000002,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.68300000000000005}]},{"osd":2,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.70399999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.79200000000000004}]},{"osd":7,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":15.038,"5min":15.038,"15min":15.038},"last":0.47499999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":11.858000000000001,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.50600000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1289999999999996,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":11.99,"5min":11.99,"15min":11.99},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.57999999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":15.006,"5min":15.006,"15min":15.006},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":11.926,"5min":11.926,"15min":11.926},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":14.974,"5min":14.974,"15min":14.974},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.960000000000001,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":14.923999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.755}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":783548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.946999999999999,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.443}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":12.983000000000001,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.2279999999999998,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.3869999999999996,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":5.468,"5min":5.468,"15min":5.468},"last":0.56699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":12.478999999999999,"5min":12.478999999999999,"15min":12.478999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.31},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":12.425000000000001,"5min":12.425000000000001,"15min":12.425000000000001},"last":0.33500000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":13.217000000000001,"5min":13.217000000000001,"15min":13.217000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":8.7400000000000002,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":8.5640000000000001,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":0.59099999999999997}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":8.7759999999999998,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":8.5190000000000001,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":8.5939999999999994,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":466944,"data_stored":197224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.378},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.45600000000000002}]},{"osd":1,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.441}]},{"osd":4,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.504}]},{"osd":5,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.53600000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.56899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.87}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":9.3970000000000002,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.747}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.84899999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":5.5289999999999999,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":9.452,"5min":9.452,"15min":9.452},"last":0.68200000000000005}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.62}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.79500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":6.6760000000000002,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.63200000000000001}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7480000000000002,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.7709999999999999,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.8410000000000002,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.37}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41199999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.442},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:42:29.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-03-30T01:42:29.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.814741+0000', 'last_change': '2024-03-30T01:42:22.814741+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.814741+0000', 'last_undegraded': '2024-03-30T01:42:22.814741+0000', 'last_fullsized': '2024-03-30T01:42:22.814741+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T12:11:34.573756+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T01:42:29.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-03-30T01:42:29.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-03-30T01:42:29.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.809509+0000', 'last_change': '2024-03-30T01:42:22.809509+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.809509+0000', 'last_undegraded': '2024-03-30T01:42:22.809509+0000', 'last_fullsized': '2024-03-30T01:42:22.809509+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:31:56.846961+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.809736+0000', 'last_change': '2024-03-30T01:42:22.809736+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.809736+0000', 'last_undegraded': '2024-03-30T01:42:22.809736+0000', 'last_fullsized': '2024-03-30T01:42:22.809736+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:21:43.792947+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.804768+0000', 'last_change': '2024-03-30T01:42:22.804768+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.804768+0000', 'last_undegraded': '2024-03-30T01:42:22.804768+0000', 'last_fullsized': '2024-03-30T01:42:22.804768+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.815309+0000', 'last_change': '2024-03-30T01:42:22.815309+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.815309+0000', 'last_undegraded': '2024-03-30T01:42:22.815309+0000', 'last_fullsized': '2024-03-30T01:42:22.815309+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:44:11.796599+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.812125+0000', 'last_change': '2024-03-30T01:42:22.812125+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.812125+0000', 'last_undegraded': '2024-03-30T01:42:22.812125+0000', 'last_fullsized': '2024-03-30T01:42:22.812125+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:18:58.845872+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.815396+0000', 'last_change': '2024-03-30T01:42:22.815396+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.815396+0000', 'last_undegraded': '2024-03-30T01:42:22.815396+0000', 'last_fullsized': '2024-03-30T01:42:22.815396+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T08:42:51.968187+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-03-30T01:42:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.807923+0000', 'last_change': '2024-03-30T01:42:22.807923+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.807923+0000', 'last_undegraded': '2024-03-30T01:42:22.807923+0000', 'last_fullsized': '2024-03-30T01:42:22.807923+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.810206+0000', 'last_change': '2024-03-30T01:42:22.810206+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.810206+0000', 'last_undegraded': '2024-03-30T01:42:22.810206+0000', 'last_fullsized': '2024-03-30T01:42:22.810206+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:10:57.966402+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.809986+0000', 'last_change': '2024-03-30T01:42:22.809986+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.809986+0000', 'last_undegraded': '2024-03-30T01:42:22.809986+0000', 'last_fullsized': '2024-03-30T01:42:22.809986+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:19:37.521361+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T01:42:29.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.805285+0000', 'last_change': '2024-03-30T01:42:22.805285+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.805285+0000', 'last_undegraded': '2024-03-30T01:42:22.805285+0000', 'last_fullsized': '2024-03-30T01:42:22.805285+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T11:55:54.923310+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.814722+0000', 'last_change': '2024-03-30T01:42:22.814722+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.814722+0000', 'last_undegraded': '2024-03-30T01:42:22.814722+0000', 'last_fullsized': '2024-03-30T01:42:22.814722+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:36:44.454854+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.805879+0000', 'last_change': '2024-03-30T01:42:22.805879+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.805879+0000', 'last_undegraded': '2024-03-30T01:42:22.805879+0000', 'last_fullsized': '2024-03-30T01:42:22.805879+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:21:57.756956+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T01:42:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.814798+0000', 'last_change': '2024-03-30T01:42:22.814798+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.814798+0000', 'last_undegraded': '2024-03-30T01:42:22.814798+0000', 'last_fullsized': '2024-03-30T01:42:22.814798+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T05:52:06.958915+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.812100+0000', 'last_change': '2024-03-30T01:42:22.812100+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.812100+0000', 'last_undegraded': '2024-03-30T01:42:22.812100+0000', 'last_fullsized': '2024-03-30T01:42:22.812100+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:25:46.724138+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-03-30T01:42:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.812173+0000', 'last_change': '2024-03-30T01:42:22.812173+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.812173+0000', 'last_undegraded': '2024-03-30T01:42:22.812173+0000', 'last_fullsized': '2024-03-30T01:42:22.812173+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:28:49.116227+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.812225+0000', 'last_change': '2024-03-30T01:42:22.812225+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.812225+0000', 'last_undegraded': '2024-03-30T01:42:22.812225+0000', 'last_fullsized': '2024-03-30T01:42:22.812225+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:53:59.368657+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.814765+0000', 'last_change': '2024-03-30T01:42:22.814765+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.814765+0000', 'last_undegraded': '2024-03-30T01:42:22.814765+0000', 'last_fullsized': '2024-03-30T01:42:22.814765+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:29:14.379389+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.809500+0000', 'last_change': '2024-03-30T01:42:22.809500+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.809500+0000', 'last_undegraded': '2024-03-30T01:42:22.809500+0000', 'last_fullsized': '2024-03-30T01:42:22.809500+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:11:20.217220+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.807918+0000', 'last_change': '2024-03-30T01:42:22.807918+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.807918+0000', 'last_undegraded': '2024-03-30T01:42:22.807918+0000', 'last_fullsized': '2024-03-30T01:42:22.807918+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T13:29:59.201428+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.815307+0000', 'last_change': '2024-03-30T01:42:22.815307+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.815307+0000', 'last_undegraded': '2024-03-30T01:42:22.815307+0000', 'last_fullsized': '2024-03-30T01:42:22.815307+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.807910+0000', 'last_change': '2024-03-30T01:42:22.807910+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.807910+0000', 'last_undegraded': '2024-03-30T01:42:22.807910+0000', 'last_fullsized': '2024-03-30T01:42:22.807910+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T03:18:12.384136+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.810039+0000', 'last_change': '2024-03-30T01:42:22.810039+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.810039+0000', 'last_undegraded': '2024-03-30T01:42:22.810039+0000', 'last_fullsized': '2024-03-30T01:42:22.810039+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T10:54:44.146196+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.808461+0000', 'last_change': '2024-03-30T01:42:22.808461+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.808461+0000', 'last_undegraded': '2024-03-30T01:42:22.808461+0000', 'last_fullsized': '2024-03-30T01:42:22.808461+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:54:59.010522+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-03-30T01:42:29.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.815299+0000', 'last_change': '2024-03-30T01:42:22.815299+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.815299+0000', 'last_undegraded': '2024-03-30T01:42:22.815299+0000', 'last_fullsized': '2024-03-30T01:42:22.815299+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.815535+0000', 'last_change': '2024-03-30T01:42:22.815535+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.815535+0000', 'last_undegraded': '2024-03-30T01:42:22.815535+0000', 'last_fullsized': '2024-03-30T01:42:22.815535+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T06:04:36.452658+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.814739+0000', 'last_change': '2024-03-30T01:42:22.814739+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.814739+0000', 'last_undegraded': '2024-03-30T01:42:22.814739+0000', 'last_fullsized': '2024-03-30T01:42:22.814739+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:08:32.897671+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-03-30T01:42:29.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.812195+0000', 'last_change': '2024-03-30T01:42:22.812195+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.812195+0000', 'last_undegraded': '2024-03-30T01:42:22.812195+0000', 'last_fullsized': '2024-03-30T01:42:22.812195+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:52:35.849177+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.809931+0000', 'last_change': '2024-03-30T01:42:22.809931+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.809931+0000', 'last_undegraded': '2024-03-30T01:42:22.809931+0000', 'last_fullsized': '2024-03-30T01:42:22.809931+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-03-30T01:42:29.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.812740+0000', 'last_change': '2024-03-30T01:42:22.812740+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.812740+0000', 'last_undegraded': '2024-03-30T01:42:22.812740+0000', 'last_fullsized': '2024-03-30T01:42:22.812740+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T02:59:13.222722+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.804764+0000', 'last_change': '2024-03-30T01:42:22.804764+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.804764+0000', 'last_undegraded': '2024-03-30T01:42:22.804764+0000', 'last_fullsized': '2024-03-30T01:42:22.804764+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-03-30T01:42:22.809889+0000', 'last_change': '2024-03-30T01:42:22.809889+0000', 'last_active': '2024-03-30T01:42:22.795162+0000', 'last_peered': '2024-03-30T01:42:22.795162+0000', 'last_clean': '2024-03-30T01:42:22.795162+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:22.809889+0000', 'last_undegraded': '2024-03-30T01:42:22.809889+0000', 'last_fullsized': '2024-03-30T01:42:22.809889+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:22.795162+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-31T09:57:35.672193+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-03-30T01:42:29.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-03-30T01:42:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-03-30T01:42:29.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-03-30T01:42:29.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-03-30T01:42:29.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-30T01:42:23.802198+0000', 'last_change': '2024-03-30T01:42:23.802198+0000', 'last_active': '2024-03-30T01:42:23.802198+0000', 'last_peered': '2024-03-30T01:42:23.802198+0000', 'last_clean': '2024-03-30T01:42:23.802198+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T01:42:23.802198+0000', 'last_undegraded': '2024-03-30T01:42:23.802198+0000', 'last_fullsized': '2024-03-30T01:42:23.802198+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'last_clean_scrub_stamp': '2024-03-30T01:42:23.802198+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-30T01:42:29.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-30T01:42:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.556+0000 7f6a4f839700 1 Processor -- start 2024-03-30T01:42:29.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.556+0000 7f6a4f839700 1 -- start start 2024-03-30T01:42:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.557+0000 7f6a4f839700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a480bc410 0x7f6a4806b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.557+0000 7f6a4f839700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a48066c70 con 0x7f6a4806bc50 2024-03-30T01:42:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.557+0000 7f6a4f839700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480c5ae0 con 0x7f6a4814fd00 2024-03-30T01:42:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.557+0000 7f6a4f839700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a480c60a0 con 0x7f6a480bc410 2024-03-30T01:42:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.557+0000 7f6a4ddd6700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6a4806bc50 0x7f6a480c4dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47422/0 (socket says 172.21.15.31:47422) 2024-03-30T01:42:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.557+0000 7f6a4ddd6700 1 -- 172.21.15.31:0/3913855560 learned_addr learned my addr 172.21.15.31:0/3913855560 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.557+0000 7f6a4cdd4700 1 --2- 172.21.15.31:0/3913855560 >> v2:172.21.15.59:3300/0 conn(0x7f6a480bc410 0x7f6a4806b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.557+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442569579 0 0) 0x7f6a40001490 con 0x7f6a4806bc50 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.557+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a340035b0 con 0x7f6a4806bc50 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288656017 0 0) 0x7f6a480c5ae0 con 0x7f6a4814fd00 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a40001490 con 0x7f6a4814fd00 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1677854063 0 0) 0x7f6a340035b0 con 0x7f6a4806bc50 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 >> v2:172.21.15.59:3300/0 conn(0x7f6a480bc410 msgr2=0x7f6a4806b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 --2- 172.21.15.31:0/3913855560 >> v2:172.21.15.59:3300/0 conn(0x7f6a480bc410 0x7f6a4806b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 >> v1:172.21.15.59:6789/0 conn(0x7f6a4814fd00 legacy=0x7f6a481530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480c5dc0 con 0x7f6a4806bc50 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f6a40001b80 con 0x7f6a4806bc50 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4cdd4700 1 --2- 172.21.15.31:0/3913855560 >> v2:172.21.15.59:3300/0 conn(0x7f6a480bc410 0x7f6a4806b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 >> v1:172.21.15.31:6789/0 conn(0x7f6a4806bc50 legacy=0x7f6a480c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34003b90 0x7f6a34003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a34004850 0x7f6a34004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34005210 0x7f6a3400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c60a0 con 0x7f6a34003b90 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3400c140 con 0x7f6a34004850 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3400c300 con 0x7f6a34005210 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4d5d5700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34003b90 0x7f6a34003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4cdd4700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a34004850 0x7f6a34004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4ddd6700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34005210 0x7f6a3400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4d5d5700 1 -- 172.21.15.31:0/3913855560 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34005210 msgr2=0x7f6a3400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4d5d5700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34005210 0x7f6a3400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4d5d5700 1 -- 172.21.15.31:0/3913855560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a34004850 msgr2=0x7f6a34004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4d5d5700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a34004850 0x7f6a34004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4d5d5700 1 -- 172.21.15.31:0/3913855560 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340035b0 con 0x7f6a34003b90 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4ddd6700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34005210 0x7f6a3400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4ddd6700 1 -- 172.21.15.31:0/3913855560 reap_dead start 2024-03-30T01:42:29.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a4d5d5700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34003b90 0x7f6a34003f90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6a44001940 tx=0x7f6a34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3e0542f09f907c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.558+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44001430 con 0x7f6a34003b90 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.559+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a44001d70 con 0x7f6a34003b90 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.559+0000 7f6a3ffff700 1 -- 172.21.15.31:0/3913855560 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44005530 con 0x7f6a34003b90 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.559+0000 7f6a4f839700 1 -- 172.21.15.31:0/3913855560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34003b90 msgr2=0x7f6a34003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.559+0000 7f6a4f839700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34003b90 0x7f6a34003f90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6a44001940 tx=0x7f6a34005880 comp rx=0 tx=0).stop 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.559+0000 7f6a4f839700 1 -- 172.21.15.31:0/3913855560 shutdown_connections 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.559+0000 7f6a4f839700 1 --2- 172.21.15.31:0/3913855560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a34003b90 0x7f6a34003f90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.559+0000 7f6a4f839700 1 -- 172.21.15.31:0/3913855560 >> 172.21.15.31:0/3913855560 conn(0x7f6a480c3020 msgr2=0x7f6a480c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.559+0000 7f6a4f839700 1 -- 172.21.15.31:0/3913855560 shutdown_connections 2024-03-30T01:42:29.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.559+0000 7f6a4f839700 1 -- 172.21.15.31:0/3913855560 wait complete. 2024-03-30T01:42:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.560+0000 7f6a4f839700 1 Processor -- start 2024-03-30T01:42:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4f839700 1 -- start start 2024-03-30T01:42:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4f839700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34003b90 0x7f6a4806ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4f839700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a481eaa30 0x7f6a481ede40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4f839700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4814a7a0 0x7f6a4814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4f839700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481ed9b0 con 0x7f6a4814a7a0 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4f839700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4814b160 con 0x7f6a481eaa30 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4f839700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480a7490 con 0x7f6a34003b90 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4ddd6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4814a7a0 0x7f6a4814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4ddd6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4814a7a0 0x7f6a4814ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44712/0 (socket says 172.21.15.31:44712) 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.561+0000 7f6a4ddd6700 1 -- 172.21.15.31:0/224503027 learned_addr learned my addr 172.21.15.31:0/224503027 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4d5d5700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34003b90 0x7f6a4806ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4cdd4700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a481eaa30 0x7f6a481ede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4ddd6700 1 -- 172.21.15.31:0/224503027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34003b90 msgr2=0x7f6a4806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4ddd6700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34003b90 0x7f6a4806ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4ddd6700 1 -- 172.21.15.31:0/224503027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a481eaa30 msgr2=0x7f6a481ede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4ddd6700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a481eaa30 0x7f6a481ede40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4ddd6700 1 -- 172.21.15.31:0/224503027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480a7770 con 0x7f6a4814a7a0 2024-03-30T01:42:29.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4ddd6700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4814a7a0 0x7f6a4814ac20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6a480ab930 tx=0x7f6a4000e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a3dffb700 1 -- 172.21.15.31:0/224503027 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40017070 con 0x7f6a4814a7a0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a3dffb700 1 -- 172.21.15.31:0/224503027 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4000edb0 con 0x7f6a4814a7a0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a480a7a50 con 0x7f6a4814a7a0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a3dffb700 1 -- 172.21.15.31:0/224503027 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40013420 con 0x7f6a4814a7a0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.562+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a481eaeb0 con 0x7f6a4814a7a0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.563+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a481edb60 con 0x7f6a4814a7a0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.564+0000 7f6a3dffb700 1 -- 172.21.15.31:0/224503027 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6a40013580 con 0x7f6a4814a7a0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.564+0000 7f6a3dffb700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a2c06ac90 0x7f6a2c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.565+0000 7f6a3dffb700 1 -- 172.21.15.31:0/224503027 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f6a40089200 con 0x7f6a4814a7a0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.565+0000 7f6a4d5d5700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a2c06ac90 0x7f6a2c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.565+0000 7f6a4d5d5700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a2c06ac90 0x7f6a2c06d150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6a44005040 tx=0x7f6a4400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:29.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.569+0000 7f6a3dffb700 1 -- 172.21.15.31:0/224503027 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6a481edb60 con 0x7f6a4814a7a0 2024-03-30T01:42:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.877+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6a481edb60 con 0x7f6a4814a7a0 2024-03-30T01:42:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.878+0000 7f6a3dffb700 1 -- 172.21.15.31:0/224503027 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1275 (secure 0 0 0) 0x7f6a481edb60 con 0x7f6a4814a7a0 2024-03-30T01:42:29.884 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:29.884 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":114,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":43,"num_osds":8,"num_up_osds":8,"osd_up_since":1711762846,"num_in_osds":8,"osd_in_since":1711762837,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97},{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32}],"num_pgs":193,"num_pools":5,"num_objects":24,"data_bytes":594241,"bytes_used":226234368,"bytes_avail":772867878912,"bytes_total":773094113280,"unknown_pgs_ratio":0.33160620927810669,"inactive_pgs_ratio":0.16580310463905334,"write_bytes_sec":407,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":21,"id":2,"up":1,"in":1,"max":1,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5235},{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5238}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-03-30T01:41:57.821053+0000","services":{}},"progress_events":{}} 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a2c06ac90 msgr2=0x7f6a2c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a2c06ac90 0x7f6a2c06d150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6a44005040 tx=0x7f6a4400a000 comp rx=0 tx=0).stop 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4814a7a0 msgr2=0x7f6a4814ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4814a7a0 0x7f6a4814ac20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6a480ab930 tx=0x7f6a4000e670 comp rx=0 tx=0).stop 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 shutdown_connections 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a4814a7a0 0x7f6a4814ac20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6a2c06ac90 0x7f6a2c06d150 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a34003b90 0x7f6a4806ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 --2- 172.21.15.31:0/224503027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a481eaa30 0x7f6a481ede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 >> 172.21.15.31:0/224503027 conn(0x7f6a480c3020 msgr2=0x7f6a4814c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 shutdown_connections 2024-03-30T01:42:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:29.881+0000 7f6a4f839700 1 -- 172.21.15.31:0/224503027 wait complete. 2024-03-30T01:42:29.900 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-30T01:42:29.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:42:30.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.031+0000 7f360ed09700 1 Processor -- start 2024-03-30T01:42:30.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.031+0000 7f360ed09700 1 -- start start 2024-03-30T01:42:30.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.032+0000 7f360ed09700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3608071880 0x7f360806a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:30.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.032+0000 7f360ed09700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360806ac40 0x7f360806e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:30.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.032+0000 7f360ed09700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36080c3610 con 0x7f360814e680 2024-03-30T01:42:30.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.032+0000 7f360ed09700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3608157da0 con 0x7f3608071880 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.032+0000 7f360ed09700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3608158080 con 0x7f360806ac40 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.032+0000 7f3607fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3608071880 0x7f360806a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.032+0000 7f360d2a6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360806ac40 0x7f360806e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f360caa5700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f360814e680 0x7f3608155060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:48442/0 (socket says 172.21.15.31:48442) 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f360caa5700 1 -- 172.21.15.31:0/2553914335 learned_addr learned my addr 172.21.15.31:0/2553914335 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f36077fe700 1 -- 172.21.15.31:0/2553914335 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251398704 0 0) 0x7f36080c3610 con 0x7f360814e680 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f360d2a6700 1 -- 172.21.15.31:0/2553914335 >> v2:172.21.15.59:3300/0 conn(0x7f3608071880 msgr2=0x7f360806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f360d2a6700 1 --2- 172.21.15.31:0/2553914335 >> v2:172.21.15.59:3300/0 conn(0x7f3608071880 0x7f360806a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f360d2a6700 1 -- 172.21.15.31:0/2553914335 >> v1:172.21.15.59:6789/0 conn(0x7f360814e680 legacy=0x7f3608155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f360d2a6700 1 -- 172.21.15.31:0/2553914335 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608158360 con 0x7f360806ac40 2024-03-30T01:42:30.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f360d2a6700 1 --2- 172.21.15.31:0/2553914335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360806ac40 0x7f360806e7e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3600009d90 tx=0x7f360000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90d7a8f2b9cd4db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f36077fe700 1 -- 172.21.15.31:0/2553914335 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360000e070 con 0x7f360806ac40 2024-03-30T01:42:30.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f36077fe700 1 -- 172.21.15.31:0/2553914335 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360000bb20 con 0x7f360806ac40 2024-03-30T01:42:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.033+0000 7f36077fe700 1 -- 172.21.15.31:0/2553914335 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600012430 con 0x7f360806ac40 2024-03-30T01:42:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.034+0000 7f360ed09700 1 -- 172.21.15.31:0/2553914335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360806ac40 msgr2=0x7f360806e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.034+0000 7f360ed09700 1 --2- 172.21.15.31:0/2553914335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360806ac40 0x7f360806e7e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3600009d90 tx=0x7f360000b330 comp rx=0 tx=0).stop 2024-03-30T01:42:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.034+0000 7f360ed09700 1 -- 172.21.15.31:0/2553914335 shutdown_connections 2024-03-30T01:42:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.034+0000 7f360ed09700 1 --2- 172.21.15.31:0/2553914335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360806ac40 0x7f360806e7e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.034+0000 7f360ed09700 1 --2- 172.21.15.31:0/2553914335 >> v2:172.21.15.59:3300/0 conn(0x7f3608071880 0x7f360806a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.034+0000 7f360ed09700 1 -- 172.21.15.31:0/2553914335 >> 172.21.15.31:0/2553914335 conn(0x7f360806c230 msgr2=0x7f36080c6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.034+0000 7f360ed09700 1 -- 172.21.15.31:0/2553914335 shutdown_connections 2024-03-30T01:42:30.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.034+0000 7f360ed09700 1 -- 172.21.15.31:0/2553914335 wait complete. 2024-03-30T01:42:30.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.036+0000 7f360ed09700 1 Processor -- start 2024-03-30T01:42:30.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.036+0000 7f360ed09700 1 -- start start 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f360ed09700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f360806ac40 0x7f360814ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f360ed09700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608071880 0x7f36080c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f360ed09700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360814e680 0x7f36080aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f360ed09700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36081ea620 con 0x7f3608071880 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f360ed09700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36081e96e0 con 0x7f360806ac40 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f360ed09700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36081e99a0 con 0x7f360814e680 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f3607fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608071880 0x7f36080c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f3607fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608071880 0x7f36080c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44722/0 (socket says 172.21.15.31:44722) 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f3607fff700 1 -- 172.21.15.31:0/1384118677 learned_addr learned my addr 172.21.15.31:0/1384118677 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f360d2a6700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360814e680 0x7f36080aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f360caa5700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f360806ac40 0x7f360814ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f3607fff700 1 -- 172.21.15.31:0/1384118677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360814e680 msgr2=0x7f36080aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f3607fff700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360814e680 0x7f36080aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f3607fff700 1 -- 172.21.15.31:0/1384118677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f360806ac40 msgr2=0x7f360814ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f3607fff700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f360806ac40 0x7f360814ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f3607fff700 1 -- 172.21.15.31:0/1384118677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600009a70 con 0x7f3608071880 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.037+0000 7f360d2a6700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360814e680 0x7f36080aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.038+0000 7f3607fff700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608071880 0x7f36080c6160 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f35f400e980 tx=0x7f35f400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.038+0000 7f36057fa700 1 -- 172.21.15.31:0/1384118677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4009710 con 0x7f3608071880 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.038+0000 7f36057fa700 1 -- 172.21.15.31:0/1384118677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f400c850 con 0x7f3608071880 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.038+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36081e9c80 con 0x7f3608071880 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.038+0000 7f36057fa700 1 -- 172.21.15.31:0/1384118677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400cc00 con 0x7f3608071880 2024-03-30T01:42:30.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.038+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36081ea240 con 0x7f3608071880 2024-03-30T01:42:30.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.039+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360804ea80 con 0x7f3608071880 2024-03-30T01:42:30.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.039+0000 7f36057fa700 1 -- 172.21.15.31:0/1384118677 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f35f4013470 con 0x7f3608071880 2024-03-30T01:42:30.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.040+0000 7f36057fa700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35ec06ac90 0x7f35ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:30.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.040+0000 7f36057fa700 1 -- 172.21.15.31:0/1384118677 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f35f40898b0 con 0x7f3608071880 2024-03-30T01:42:30.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.040+0000 7f360caa5700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35ec06ac90 0x7f35ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:30.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.040+0000 7f360caa5700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35ec06ac90 0x7f35ec06d150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000f60 tx=0x7f35fc007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:30.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.045+0000 7f36057fa700 1 -- 172.21.15.31:0/1384118677 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f35f4054740 con 0x7f3608071880 2024-03-30T01:42:30.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.253+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f360814ea60 con 0x7f35ec06ac90 2024-03-30T01:42:30.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.268+0000 7f36057fa700 1 -- 172.21.15.31:0/1384118677 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526896 (secure 0 0 0) 0x7f35fc005b90 con 0x7f35ec06ac90 2024-03-30T01:42:30.280 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:30.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35ec06ac90 msgr2=0x7f35ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:30.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35ec06ac90 0x7f35ec06d150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000f60 tx=0x7f35fc007c90 comp rx=0 tx=0).stop 2024-03-30T01:42:30.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608071880 msgr2=0x7f36080c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:30.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608071880 0x7f36080c6160 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f35f400e980 tx=0x7f35f400ed40 comp rx=0 tx=0).stop 2024-03-30T01:42:30.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 shutdown_connections 2024-03-30T01:42:30.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f35ec06ac90 0x7f35ec06d150 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:30.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f360814e680 0x7f36080aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:30.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f360806ac40 0x7f360814ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:30.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 --2- 172.21.15.31:0/1384118677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3608071880 0x7f36080c6160 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:30.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 >> 172.21.15.31:0/1384118677 conn(0x7f360806c230 msgr2=0x7f360814cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:30.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 shutdown_connections 2024-03-30T01:42:30.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:30.273+0000 7f360ed09700 1 -- 172.21.15.31:0/1384118677 wait complete. 2024-03-30T01:42:30.285 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:42:30.292 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-03-30T01:42:29.828341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596603,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221368,"kb_used_data":5188,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753352,"statfs":{"total":773094113280,"available":772867432448,"internally_reserved":0,"allocated":5312512,"data_stored":2984125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.019119"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282109+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:42:24.282109+0000","last_peered":"2024-03-30T01:42:24.282109+0000","last_clean":"2024-03-30T01:42:24.282109+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:42:24.282109+0000","last_undegraded":"2024-03-30T01:42:24.282109+0000","last_fullsized":"2024-03-30T01:42:24.282109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282595+0000","last_change":"2024-03-30T01:42:09.836532+0000","last_active":"2024-03-30T01:42:24.282595+0000","last_peered":"2024-03-30T01:42:24.282595+0000","last_clean":"2024-03-30T01:42:24.282595+0000","last_became_active":"2024-03-30T01:42:09.836262+0000","last_became_peered":"2024-03-30T01:42:09.836262+0000","last_unstale":"2024-03-30T01:42:24.282595+0000","last_undegraded":"2024-03-30T01:42:24.282595+0000","last_fullsized":"2024-03-30T01:42:24.282595+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:08.249431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277527+0000","last_change":"2024-03-30T01:42:10.289721+0000","last_active":"2024-03-30T01:42:24.277527+0000","last_peered":"2024-03-30T01:42:24.277527+0000","last_clean":"2024-03-30T01:42:24.277527+0000","last_became_active":"2024-03-30T01:42:10.289413+0000","last_became_peered":"2024-03-30T01:42:10.289413+0000","last_unstale":"2024-03-30T01:42:24.277527+0000","last_undegraded":"2024-03-30T01:42:24.277527+0000","last_fullsized":"2024-03-30T01:42:24.277527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"43'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606808+0000","last_change":"2024-03-30T01:42:23.832956+0000","last_active":"2024-03-30T01:42:25.606808+0000","last_peered":"2024-03-30T01:42:25.606808+0000","last_clean":"2024-03-30T01:42:25.606808+0000","last_became_active":"2024-03-30T01:42:23.832684+0000","last_became_peered":"2024-03-30T01:42:23.832684+0000","last_unstale":"2024-03-30T01:42:25.606808+0000","last_undegraded":"2024-03-30T01:42:25.606808+0000","last_fullsized":"2024-03-30T01:42:25.606808+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:34.573756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318413+0000","last_change":"2024-03-30T01:42:24.286333+0000","last_active":"2024-03-30T01:42:25.318413+0000","last_peered":"2024-03-30T01:42:25.318413+0000","last_clean":"2024-03-30T01:42:25.318413+0000","last_became_active":"2024-03-30T01:42:24.285564+0000","last_became_peered":"2024-03-30T01:42:24.285564+0000","last_unstale":"2024-03-30T01:42:25.318413+0000","last_undegraded":"2024-03-30T01:42:25.318413+0000","last_fullsized":"2024-03-30T01:42:25.318413+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:22:57.363372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291900+0000","last_change":"2024-03-30T01:42:09.829174+0000","last_active":"2024-03-30T01:42:24.291900+0000","last_peered":"2024-03-30T01:42:24.291900+0000","last_clean":"2024-03-30T01:42:24.291900+0000","last_became_active":"2024-03-30T01:42:09.828730+0000","last_became_peered":"2024-03-30T01:42:09.828730+0000","last_unstale":"2024-03-30T01:42:24.291900+0000","last_undegraded":"2024-03-30T01:42:24.291900+0000","last_fullsized":"2024-03-30T01:42:24.291900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282943+0000","last_change":"2024-03-30T01:42:10.284566+0000","last_active":"2024-03-30T01:42:24.282943+0000","last_peered":"2024-03-30T01:42:24.282943+0000","last_clean":"2024-03-30T01:42:24.282943+0000","last_became_active":"2024-03-30T01:42:10.284192+0000","last_became_peered":"2024-03-30T01:42:10.284192+0000","last_unstale":"2024-03-30T01:42:24.282943+0000","last_undegraded":"2024-03-30T01:42:24.282943+0000","last_fullsized":"2024-03-30T01:42:24.282943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:05.460383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290171+0000","last_change":"2024-03-30T01:42:23.827217+0000","last_active":"2024-03-30T01:42:24.290171+0000","last_peered":"2024-03-30T01:42:24.290171+0000","last_clean":"2024-03-30T01:42:24.290171+0000","last_became_active":"2024-03-30T01:42:23.826922+0000","last_became_peered":"2024-03-30T01:42:23.826922+0000","last_unstale":"2024-03-30T01:42:24.290171+0000","last_undegraded":"2024-03-30T01:42:24.290171+0000","last_fullsized":"2024-03-30T01:42:24.290171+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:56.846961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274839+0000","last_change":"2024-03-30T01:42:24.292886+0000","last_active":"2024-03-30T01:42:25.274839+0000","last_peered":"2024-03-30T01:42:25.274839+0000","last_clean":"2024-03-30T01:42:25.274839+0000","last_became_active":"2024-03-30T01:42:24.292613+0000","last_became_peered":"2024-03-30T01:42:24.292613+0000","last_unstale":"2024-03-30T01:42:25.274839+0000","last_undegraded":"2024-03-30T01:42:25.274839+0000","last_fullsized":"2024-03-30T01:42:25.274839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:22:35.546913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283516+0000","last_change":"2024-03-30T01:42:09.832991+0000","last_active":"2024-03-30T01:42:24.283516+0000","last_peered":"2024-03-30T01:42:24.283516+0000","last_clean":"2024-03-30T01:42:24.283516+0000","last_became_active":"2024-03-30T01:42:09.832682+0000","last_became_peered":"2024-03-30T01:42:09.832682+0000","last_unstale":"2024-03-30T01:42:24.283516+0000","last_undegraded":"2024-03-30T01:42:24.283516+0000","last_fullsized":"2024-03-30T01:42:24.283516+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:37.427434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.288679+0000","last_change":"2024-03-30T01:42:10.299170+0000","last_active":"2024-03-30T01:42:24.288679+0000","last_peered":"2024-03-30T01:42:24.288679+0000","last_clean":"2024-03-30T01:42:24.288679+0000","last_became_active":"2024-03-30T01:42:10.298846+0000","last_became_peered":"2024-03-30T01:42:10.298846+0000","last_unstale":"2024-03-30T01:42:24.288679+0000","last_undegraded":"2024-03-30T01:42:24.288679+0000","last_fullsized":"2024-03-30T01:42:24.288679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291544+0000","last_change":"2024-03-30T01:42:23.836629+0000","last_active":"2024-03-30T01:42:24.291544+0000","last_peered":"2024-03-30T01:42:24.291544+0000","last_clean":"2024-03-30T01:42:24.291544+0000","last_became_active":"2024-03-30T01:42:23.836370+0000","last_became_peered":"2024-03-30T01:42:23.836370+0000","last_unstale":"2024-03-30T01:42:24.291544+0000","last_undegraded":"2024-03-30T01:42:24.291544+0000","last_fullsized":"2024-03-30T01:42:24.291544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:43.792947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606550+0000","last_change":"2024-03-30T01:42:24.295507+0000","last_active":"2024-03-30T01:42:25.606550+0000","last_peered":"2024-03-30T01:42:25.606550+0000","last_clean":"2024-03-30T01:42:25.606550+0000","last_became_active":"2024-03-30T01:42:24.295110+0000","last_became_peered":"2024-03-30T01:42:24.295110+0000","last_unstale":"2024-03-30T01:42:25.606550+0000","last_undegraded":"2024-03-30T01:42:25.606550+0000","last_fullsized":"2024-03-30T01:42:25.606550+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:12:26.305628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273317+0000","last_change":"2024-03-30T01:42:09.822074+0000","last_active":"2024-03-30T01:42:24.273317+0000","last_peered":"2024-03-30T01:42:24.273317+0000","last_clean":"2024-03-30T01:42:24.273317+0000","last_became_active":"2024-03-30T01:42:09.821786+0000","last_became_peered":"2024-03-30T01:42:09.821786+0000","last_unstale":"2024-03-30T01:42:24.273317+0000","last_undegraded":"2024-03-30T01:42:24.273317+0000","last_fullsized":"2024-03-30T01:42:24.273317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:18.840885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.286388+0000","last_change":"2024-03-30T01:42:10.291029+0000","last_active":"2024-03-30T01:42:24.286388+0000","last_peered":"2024-03-30T01:42:24.286388+0000","last_clean":"2024-03-30T01:42:24.286388+0000","last_became_active":"2024-03-30T01:42:10.290728+0000","last_became_peered":"2024-03-30T01:42:10.290728+0000","last_unstale":"2024-03-30T01:42:24.286388+0000","last_undegraded":"2024-03-30T01:42:24.286388+0000","last_fullsized":"2024-03-30T01:42:24.286388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277359+0000","last_change":"2024-03-30T01:42:23.834875+0000","last_active":"2024-03-30T01:42:24.277359+0000","last_peered":"2024-03-30T01:42:24.277359+0000","last_clean":"2024-03-30T01:42:24.277359+0000","last_became_active":"2024-03-30T01:42:23.834636+0000","last_became_peered":"2024-03-30T01:42:23.834636+0000","last_unstale":"2024-03-30T01:42:24.277359+0000","last_undegraded":"2024-03-30T01:42:24.277359+0000","last_fullsized":"2024-03-30T01:42:24.277359+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812557+0000","last_change":"2024-03-30T01:42:24.293284+0000","last_active":"2024-03-30T01:42:25.812557+0000","last_peered":"2024-03-30T01:42:25.812557+0000","last_clean":"2024-03-30T01:42:25.812557+0000","last_became_active":"2024-03-30T01:42:24.293000+0000","last_became_peered":"2024-03-30T01:42:24.293000+0000","last_unstale":"2024-03-30T01:42:25.812557+0000","last_undegraded":"2024-03-30T01:42:25.812557+0000","last_fullsized":"2024-03-30T01:42:25.812557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:13:41.465524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.279424+0000","last_change":"2024-03-30T01:42:09.827116+0000","last_active":"2024-03-30T01:42:24.279424+0000","last_peered":"2024-03-30T01:42:24.279424+0000","last_clean":"2024-03-30T01:42:24.279424+0000","last_became_active":"2024-03-30T01:42:09.826876+0000","last_became_peered":"2024-03-30T01:42:09.826876+0000","last_unstale":"2024-03-30T01:42:24.279424+0000","last_undegraded":"2024-03-30T01:42:24.279424+0000","last_fullsized":"2024-03-30T01:42:24.279424+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:38.792762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278630+0000","last_change":"2024-03-30T01:42:10.288334+0000","last_active":"2024-03-30T01:42:24.278630+0000","last_peered":"2024-03-30T01:42:24.278630+0000","last_clean":"2024-03-30T01:42:24.278630+0000","last_became_active":"2024-03-30T01:42:10.288093+0000","last_became_peered":"2024-03-30T01:42:10.288093+0000","last_unstale":"2024-03-30T01:42:24.278630+0000","last_undegraded":"2024-03-30T01:42:24.278630+0000","last_fullsized":"2024-03-30T01:42:24.278630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283343+0000","last_change":"2024-03-30T01:42:23.826453+0000","last_active":"2024-03-30T01:42:24.283343+0000","last_peered":"2024-03-30T01:42:24.283343+0000","last_clean":"2024-03-30T01:42:24.283343+0000","last_became_active":"2024-03-30T01:42:23.826129+0000","last_became_peered":"2024-03-30T01:42:23.826129+0000","last_unstale":"2024-03-30T01:42:24.283343+0000","last_undegraded":"2024-03-30T01:42:24.283343+0000","last_fullsized":"2024-03-30T01:42:24.283343+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:44:11.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606935+0000","last_change":"2024-03-30T01:42:24.288310+0000","last_active":"2024-03-30T01:42:25.606935+0000","last_peered":"2024-03-30T01:42:25.606935+0000","last_clean":"2024-03-30T01:42:25.606935+0000","last_became_active":"2024-03-30T01:42:24.288026+0000","last_became_peered":"2024-03-30T01:42:24.288026+0000","last_unstale":"2024-03-30T01:42:25.606935+0000","last_undegraded":"2024-03-30T01:42:25.606935+0000","last_fullsized":"2024-03-30T01:42:25.606935+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:39:42.096508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280297+0000","last_change":"2024-03-30T01:42:09.830490+0000","last_active":"2024-03-30T01:42:24.280297+0000","last_peered":"2024-03-30T01:42:24.280297+0000","last_clean":"2024-03-30T01:42:24.280297+0000","last_became_active":"2024-03-30T01:42:09.830197+0000","last_became_peered":"2024-03-30T01:42:09.830197+0000","last_unstale":"2024-03-30T01:42:24.280297+0000","last_undegraded":"2024-03-30T01:42:24.280297+0000","last_fullsized":"2024-03-30T01:42:24.280297+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:46.433516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290564+0000","last_change":"2024-03-30T01:42:10.299986+0000","last_active":"2024-03-30T01:42:24.290564+0000","last_peered":"2024-03-30T01:42:24.290564+0000","last_clean":"2024-03-30T01:42:24.290564+0000","last_became_active":"2024-03-30T01:42:10.299776+0000","last_became_peered":"2024-03-30T01:42:10.299776+0000","last_unstale":"2024-03-30T01:42:24.290564+0000","last_undegraded":"2024-03-30T01:42:24.290564+0000","last_fullsized":"2024-03-30T01:42:24.290564+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.308388+0000","last_change":"2024-03-30T01:42:23.836727+0000","last_active":"2024-03-30T01:42:24.308388+0000","last_peered":"2024-03-30T01:42:24.308388+0000","last_clean":"2024-03-30T01:42:24.308388+0000","last_became_active":"2024-03-30T01:42:23.836310+0000","last_became_peered":"2024-03-30T01:42:23.836310+0000","last_unstale":"2024-03-30T01:42:24.308388+0000","last_undegraded":"2024-03-30T01:42:24.308388+0000","last_fullsized":"2024-03-30T01:42:24.308388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:58.845872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274816+0000","last_change":"2024-03-30T01:42:24.299335+0000","last_active":"2024-03-30T01:42:25.274816+0000","last_peered":"2024-03-30T01:42:25.274816+0000","last_clean":"2024-03-30T01:42:25.274816+0000","last_became_active":"2024-03-30T01:42:24.299066+0000","last_became_peered":"2024-03-30T01:42:24.299066+0000","last_unstale":"2024-03-30T01:42:25.274816+0000","last_undegraded":"2024-03-30T01:42:25.274816+0000","last_fullsized":"2024-03-30T01:42:25.274816+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:58.973438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275374+0000","last_change":"2024-03-30T01:42:09.824373+0000","last_active":"2024-03-30T01:42:24.275374+0000","last_peered":"2024-03-30T01:42:24.275374+0000","last_clean":"2024-03-30T01:42:24.275374+0000","last_became_active":"2024-03-30T01:42:09.824089+0000","last_became_peered":"2024-03-30T01:42:09.824089+0000","last_unstale":"2024-03-30T01:42:24.275374+0000","last_undegraded":"2024-03-30T01:42:24.275374+0000","last_fullsized":"2024-03-30T01:42:24.275374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:37.434399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.284258+0000","last_change":"2024-03-30T01:42:10.284232+0000","last_active":"2024-03-30T01:42:24.284258+0000","last_peered":"2024-03-30T01:42:24.284258+0000","last_clean":"2024-03-30T01:42:24.284258+0000","last_became_active":"2024-03-30T01:42:10.283909+0000","last_became_peered":"2024-03-30T01:42:10.283909+0000","last_unstale":"2024-03-30T01:42:24.284258+0000","last_undegraded":"2024-03-30T01:42:24.284258+0000","last_fullsized":"2024-03-30T01:42:24.284258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:33.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278837+0000","last_change":"2024-03-30T01:42:23.830264+0000","last_active":"2024-03-30T01:42:24.278837+0000","last_peered":"2024-03-30T01:42:24.278837+0000","last_clean":"2024-03-30T01:42:24.278837+0000","last_became_active":"2024-03-30T01:42:23.829948+0000","last_became_peered":"2024-03-30T01:42:23.829948+0000","last_unstale":"2024-03-30T01:42:24.278837+0000","last_undegraded":"2024-03-30T01:42:24.278837+0000","last_fullsized":"2024-03-30T01:42:24.278837+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:42:51.968187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812210+0000","last_change":"2024-03-30T01:42:24.298952+0000","last_active":"2024-03-30T01:42:25.812210+0000","last_peered":"2024-03-30T01:42:25.812210+0000","last_clean":"2024-03-30T01:42:25.812210+0000","last_became_active":"2024-03-30T01:42:24.298680+0000","last_became_peered":"2024-03-30T01:42:24.298680+0000","last_unstale":"2024-03-30T01:42:25.812210+0000","last_undegraded":"2024-03-30T01:42:25.812210+0000","last_fullsized":"2024-03-30T01:42:25.812210+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:11.779346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277686+0000","last_change":"2024-03-30T01:42:09.833189+0000","last_active":"2024-03-30T01:42:24.277686+0000","last_peered":"2024-03-30T01:42:24.277686+0000","last_clean":"2024-03-30T01:42:24.277686+0000","last_became_active":"2024-03-30T01:42:09.832891+0000","last_became_peered":"2024-03-30T01:42:09.832891+0000","last_unstale":"2024-03-30T01:42:24.277686+0000","last_undegraded":"2024-03-30T01:42:24.277686+0000","last_fullsized":"2024-03-30T01:42:24.277686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283440+0000","last_change":"2024-03-30T01:42:10.298216+0000","last_active":"2024-03-30T01:42:24.283440+0000","last_peered":"2024-03-30T01:42:24.283440+0000","last_clean":"2024-03-30T01:42:24.283440+0000","last_became_active":"2024-03-30T01:42:10.297976+0000","last_became_peered":"2024-03-30T01:42:10.297976+0000","last_unstale":"2024-03-30T01:42:24.283440+0000","last_undegraded":"2024-03-30T01:42:24.283440+0000","last_fullsized":"2024-03-30T01:42:24.283440+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.284594+0000","last_change":"2024-03-30T01:42:23.827398+0000","last_active":"2024-03-30T01:42:24.284594+0000","last_peered":"2024-03-30T01:42:24.284594+0000","last_clean":"2024-03-30T01:42:24.284594+0000","last_became_active":"2024-03-30T01:42:23.827120+0000","last_became_peered":"2024-03-30T01:42:23.827120+0000","last_unstale":"2024-03-30T01:42:24.284594+0000","last_undegraded":"2024-03-30T01:42:24.284594+0000","last_fullsized":"2024-03-30T01:42:24.284594+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.273929+0000","last_change":"2024-03-30T01:42:24.293160+0000","last_active":"2024-03-30T01:42:25.273929+0000","last_peered":"2024-03-30T01:42:25.273929+0000","last_clean":"2024-03-30T01:42:25.273929+0000","last_became_active":"2024-03-30T01:42:24.292443+0000","last_became_peered":"2024-03-30T01:42:24.292443+0000","last_unstale":"2024-03-30T01:42:25.273929+0000","last_undegraded":"2024-03-30T01:42:25.273929+0000","last_fullsized":"2024-03-30T01:42:25.273929+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:24.433162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273654+0000","last_change":"2024-03-30T01:42:09.829638+0000","last_active":"2024-03-30T01:42:24.273654+0000","last_peered":"2024-03-30T01:42:24.273654+0000","last_clean":"2024-03-30T01:42:24.273654+0000","last_became_active":"2024-03-30T01:42:09.829317+0000","last_became_peered":"2024-03-30T01:42:09.829317+0000","last_unstale":"2024-03-30T01:42:24.273654+0000","last_undegraded":"2024-03-30T01:42:24.273654+0000","last_fullsized":"2024-03-30T01:42:24.273654+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:33.639097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290194+0000","last_change":"2024-03-30T01:42:10.292043+0000","last_active":"2024-03-30T01:42:24.290194+0000","last_peered":"2024-03-30T01:42:24.290194+0000","last_clean":"2024-03-30T01:42:24.290194+0000","last_became_active":"2024-03-30T01:42:10.291598+0000","last_became_peered":"2024-03-30T01:42:10.291598+0000","last_unstale":"2024-03-30T01:42:24.290194+0000","last_undegraded":"2024-03-30T01:42:24.290194+0000","last_fullsized":"2024-03-30T01:42:24.290194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290056+0000","last_change":"2024-03-30T01:42:23.837223+0000","last_active":"2024-03-30T01:42:24.290056+0000","last_peered":"2024-03-30T01:42:24.290056+0000","last_clean":"2024-03-30T01:42:24.290056+0000","last_became_active":"2024-03-30T01:42:23.836973+0000","last_became_peered":"2024-03-30T01:42:23.836973+0000","last_unstale":"2024-03-30T01:42:24.290056+0000","last_undegraded":"2024-03-30T01:42:24.290056+0000","last_fullsized":"2024-03-30T01:42:24.290056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:57.966402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318600+0000","last_change":"2024-03-30T01:42:24.283055+0000","last_active":"2024-03-30T01:42:25.318600+0000","last_peered":"2024-03-30T01:42:25.318600+0000","last_clean":"2024-03-30T01:42:25.318600+0000","last_became_active":"2024-03-30T01:42:24.282764+0000","last_became_peered":"2024-03-30T01:42:24.282764+0000","last_unstale":"2024-03-30T01:42:25.318600+0000","last_undegraded":"2024-03-30T01:42:25.318600+0000","last_fullsized":"2024-03-30T01:42:25.318600+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:00.208464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282644+0000","last_change":"2024-03-30T01:42:09.829777+0000","last_active":"2024-03-30T01:42:24.282644+0000","last_peered":"2024-03-30T01:42:24.282644+0000","last_clean":"2024-03-30T01:42:24.282644+0000","last_became_active":"2024-03-30T01:42:09.829483+0000","last_became_peered":"2024-03-30T01:42:09.829483+0000","last_unstale":"2024-03-30T01:42:24.282644+0000","last_undegraded":"2024-03-30T01:42:24.282644+0000","last_fullsized":"2024-03-30T01:42:24.282644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:45.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.276850+0000","last_change":"2024-03-30T01:42:10.285649+0000","last_active":"2024-03-30T01:42:24.276850+0000","last_peered":"2024-03-30T01:42:24.276850+0000","last_clean":"2024-03-30T01:42:24.276850+0000","last_became_active":"2024-03-30T01:42:10.285297+0000","last_became_peered":"2024-03-30T01:42:10.285297+0000","last_unstale":"2024-03-30T01:42:24.276850+0000","last_undegraded":"2024-03-30T01:42:24.276850+0000","last_fullsized":"2024-03-30T01:42:24.276850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297151+0000","last_change":"2024-03-30T01:42:23.831786+0000","last_active":"2024-03-30T01:42:24.297151+0000","last_peered":"2024-03-30T01:42:24.297151+0000","last_clean":"2024-03-30T01:42:24.297151+0000","last_became_active":"2024-03-30T01:42:23.829363+0000","last_became_peered":"2024-03-30T01:42:23.829363+0000","last_unstale":"2024-03-30T01:42:24.297151+0000","last_undegraded":"2024-03-30T01:42:24.297151+0000","last_fullsized":"2024-03-30T01:42:24.297151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:19:37.521361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275209+0000","last_change":"2024-03-30T01:42:24.298420+0000","last_active":"2024-03-30T01:42:25.275209+0000","last_peered":"2024-03-30T01:42:25.275209+0000","last_clean":"2024-03-30T01:42:25.275209+0000","last_became_active":"2024-03-30T01:42:24.297784+0000","last_became_peered":"2024-03-30T01:42:24.297784+0000","last_unstale":"2024-03-30T01:42:25.275209+0000","last_undegraded":"2024-03-30T01:42:25.275209+0000","last_fullsized":"2024-03-30T01:42:25.275209+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:45.153231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277816+0000","last_change":"2024-03-30T01:42:09.826256+0000","last_active":"2024-03-30T01:42:24.277816+0000","last_peered":"2024-03-30T01:42:24.277816+0000","last_clean":"2024-03-30T01:42:24.277816+0000","last_became_active":"2024-03-30T01:42:09.825956+0000","last_became_peered":"2024-03-30T01:42:09.825956+0000","last_unstale":"2024-03-30T01:42:24.277816+0000","last_undegraded":"2024-03-30T01:42:24.277816+0000","last_fullsized":"2024-03-30T01:42:24.277816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:59:41.446927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278754+0000","last_change":"2024-03-30T01:42:10.290469+0000","last_active":"2024-03-30T01:42:24.278754+0000","last_peered":"2024-03-30T01:42:24.278754+0000","last_clean":"2024-03-30T01:42:24.278754+0000","last_became_active":"2024-03-30T01:42:10.290148+0000","last_became_peered":"2024-03-30T01:42:10.290148+0000","last_unstale":"2024-03-30T01:42:24.278754+0000","last_undegraded":"2024-03-30T01:42:24.278754+0000","last_fullsized":"2024-03-30T01:42:24.278754+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275001+0000","last_change":"2024-03-30T01:42:23.826954+0000","last_active":"2024-03-30T01:42:24.275001+0000","last_peered":"2024-03-30T01:42:24.275001+0000","last_clean":"2024-03-30T01:42:24.275001+0000","last_became_active":"2024-03-30T01:42:23.826586+0000","last_became_peered":"2024-03-30T01:42:23.826586+0000","last_unstale":"2024-03-30T01:42:24.275001+0000","last_undegraded":"2024-03-30T01:42:24.275001+0000","last_fullsized":"2024-03-30T01:42:24.275001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:54.923310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275067+0000","last_change":"2024-03-30T01:42:24.299122+0000","last_active":"2024-03-30T01:42:25.275067+0000","last_peered":"2024-03-30T01:42:25.275067+0000","last_clean":"2024-03-30T01:42:25.275067+0000","last_became_active":"2024-03-30T01:42:24.298339+0000","last_became_peered":"2024-03-30T01:42:24.298339+0000","last_unstale":"2024-03-30T01:42:25.275067+0000","last_undegraded":"2024-03-30T01:42:25.275067+0000","last_fullsized":"2024-03-30T01:42:25.275067+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:21.224256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275815+0000","last_change":"2024-03-30T01:42:09.830452+0000","last_active":"2024-03-30T01:42:24.275815+0000","last_peered":"2024-03-30T01:42:24.275815+0000","last_clean":"2024-03-30T01:42:24.275815+0000","last_became_active":"2024-03-30T01:42:09.830173+0000","last_became_peered":"2024-03-30T01:42:09.830173+0000","last_unstale":"2024-03-30T01:42:24.275815+0000","last_undegraded":"2024-03-30T01:42:24.275815+0000","last_fullsized":"2024-03-30T01:42:24.275815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:12:42.807604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282320+0000","last_change":"2024-03-30T01:42:10.279913+0000","last_active":"2024-03-30T01:42:24.282320+0000","last_peered":"2024-03-30T01:42:24.282320+0000","last_clean":"2024-03-30T01:42:24.282320+0000","last_became_active":"2024-03-30T01:42:10.279450+0000","last_became_peered":"2024-03-30T01:42:10.279450+0000","last_unstale":"2024-03-30T01:42:24.282320+0000","last_undegraded":"2024-03-30T01:42:24.282320+0000","last_fullsized":"2024-03-30T01:42:24.282320+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278669+0000","last_change":"2024-03-30T01:42:23.835387+0000","last_active":"2024-03-30T01:42:24.278669+0000","last_peered":"2024-03-30T01:42:24.278669+0000","last_clean":"2024-03-30T01:42:24.278669+0000","last_became_active":"2024-03-30T01:42:23.833427+0000","last_became_peered":"2024-03-30T01:42:23.833427+0000","last_unstale":"2024-03-30T01:42:24.278669+0000","last_undegraded":"2024-03-30T01:42:24.278669+0000","last_fullsized":"2024-03-30T01:42:24.278669+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:44.454854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606752+0000","last_change":"2024-03-30T01:42:24.300970+0000","last_active":"2024-03-30T01:42:25.606752+0000","last_peered":"2024-03-30T01:42:25.606752+0000","last_clean":"2024-03-30T01:42:25.606752+0000","last_became_active":"2024-03-30T01:42:24.300724+0000","last_became_peered":"2024-03-30T01:42:24.300724+0000","last_unstale":"2024-03-30T01:42:25.606752+0000","last_undegraded":"2024-03-30T01:42:25.606752+0000","last_fullsized":"2024-03-30T01:42:25.606752+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:24:42.987038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.287160+0000","last_change":"2024-03-30T01:42:09.828572+0000","last_active":"2024-03-30T01:42:24.287160+0000","last_peered":"2024-03-30T01:42:24.287160+0000","last_clean":"2024-03-30T01:42:24.287160+0000","last_became_active":"2024-03-30T01:42:09.828225+0000","last_became_peered":"2024-03-30T01:42:09.828225+0000","last_unstale":"2024-03-30T01:42:24.287160+0000","last_undegraded":"2024-03-30T01:42:24.287160+0000","last_fullsized":"2024-03-30T01:42:24.287160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275458+0000","last_change":"2024-03-30T01:42:10.286900+0000","last_active":"2024-03-30T01:42:24.275458+0000","last_peered":"2024-03-30T01:42:24.275458+0000","last_clean":"2024-03-30T01:42:24.275458+0000","last_became_active":"2024-03-30T01:42:10.286598+0000","last_became_peered":"2024-03-30T01:42:10.286598+0000","last_unstale":"2024-03-30T01:42:24.275458+0000","last_undegraded":"2024-03-30T01:42:24.275458+0000","last_fullsized":"2024-03-30T01:42:24.275458+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:46.592007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291128+0000","last_change":"2024-03-30T01:42:23.829204+0000","last_active":"2024-03-30T01:42:24.291128+0000","last_peered":"2024-03-30T01:42:24.291128+0000","last_clean":"2024-03-30T01:42:24.291128+0000","last_became_active":"2024-03-30T01:42:23.828938+0000","last_became_peered":"2024-03-30T01:42:23.828938+0000","last_unstale":"2024-03-30T01:42:24.291128+0000","last_undegraded":"2024-03-30T01:42:24.291128+0000","last_fullsized":"2024-03-30T01:42:24.291128+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:57.756956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.273668+0000","last_change":"2024-03-30T01:42:24.301476+0000","last_active":"2024-03-30T01:42:25.273668+0000","last_peered":"2024-03-30T01:42:25.273668+0000","last_clean":"2024-03-30T01:42:25.273668+0000","last_became_active":"2024-03-30T01:42:24.301208+0000","last_became_peered":"2024-03-30T01:42:24.301208+0000","last_unstale":"2024-03-30T01:42:25.273668+0000","last_undegraded":"2024-03-30T01:42:25.273668+0000","last_fullsized":"2024-03-30T01:42:25.273668+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:57.853997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278103+0000","last_change":"2024-03-30T01:42:09.823221+0000","last_active":"2024-03-30T01:42:24.278103+0000","last_peered":"2024-03-30T01:42:24.278103+0000","last_clean":"2024-03-30T01:42:24.278103+0000","last_became_active":"2024-03-30T01:42:09.822882+0000","last_became_peered":"2024-03-30T01:42:09.822882+0000","last_unstale":"2024-03-30T01:42:24.278103+0000","last_undegraded":"2024-03-30T01:42:24.278103+0000","last_fullsized":"2024-03-30T01:42:24.278103+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:00:31.185685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283611+0000","last_change":"2024-03-30T01:42:10.304085+0000","last_active":"2024-03-30T01:42:24.283611+0000","last_peered":"2024-03-30T01:42:24.283611+0000","last_clean":"2024-03-30T01:42:24.283611+0000","last_became_active":"2024-03-30T01:42:10.303817+0000","last_became_peered":"2024-03-30T01:42:10.303817+0000","last_unstale":"2024-03-30T01:42:24.283611+0000","last_undegraded":"2024-03-30T01:42:24.283611+0000","last_fullsized":"2024-03-30T01:42:24.283611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.303894+0000","last_change":"2024-03-30T01:42:23.837118+0000","last_active":"2024-03-30T01:42:24.303894+0000","last_peered":"2024-03-30T01:42:24.303894+0000","last_clean":"2024-03-30T01:42:24.303894+0000","last_became_active":"2024-03-30T01:42:23.836859+0000","last_became_peered":"2024-03-30T01:42:23.836859+0000","last_unstale":"2024-03-30T01:42:24.303894+0000","last_undegraded":"2024-03-30T01:42:24.303894+0000","last_fullsized":"2024-03-30T01:42:24.303894+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:06.958915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811582+0000","last_change":"2024-03-30T01:42:24.299737+0000","last_active":"2024-03-30T01:42:25.811582+0000","last_peered":"2024-03-30T01:42:25.811582+0000","last_clean":"2024-03-30T01:42:25.811582+0000","last_became_active":"2024-03-30T01:42:24.299259+0000","last_became_peered":"2024-03-30T01:42:24.299259+0000","last_unstale":"2024-03-30T01:42:25.811582+0000","last_undegraded":"2024-03-30T01:42:25.811582+0000","last_fullsized":"2024-03-30T01:42:25.811582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:14.820447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283642+0000","last_change":"2024-03-30T01:42:09.829713+0000","last_active":"2024-03-30T01:42:24.283642+0000","last_peered":"2024-03-30T01:42:24.283642+0000","last_clean":"2024-03-30T01:42:24.283642+0000","last_became_active":"2024-03-30T01:42:09.829375+0000","last_became_peered":"2024-03-30T01:42:09.829375+0000","last_unstale":"2024-03-30T01:42:24.283642+0000","last_undegraded":"2024-03-30T01:42:24.283642+0000","last_fullsized":"2024-03-30T01:42:24.283642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:49:51.601663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277836+0000","last_change":"2024-03-30T01:42:10.295516+0000","last_active":"2024-03-30T01:42:24.277836+0000","last_peered":"2024-03-30T01:42:24.277836+0000","last_clean":"2024-03-30T01:42:24.277836+0000","last_became_active":"2024-03-30T01:42:10.295133+0000","last_became_peered":"2024-03-30T01:42:10.295133+0000","last_unstale":"2024-03-30T01:42:24.277836+0000","last_undegraded":"2024-03-30T01:42:24.277836+0000","last_fullsized":"2024-03-30T01:42:24.277836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273944+0000","last_change":"2024-03-30T01:42:23.826615+0000","last_active":"2024-03-30T01:42:24.273944+0000","last_peered":"2024-03-30T01:42:24.273944+0000","last_clean":"2024-03-30T01:42:24.273944+0000","last_became_active":"2024-03-30T01:42:23.826332+0000","last_became_peered":"2024-03-30T01:42:23.826332+0000","last_unstale":"2024-03-30T01:42:24.273944+0000","last_undegraded":"2024-03-30T01:42:24.273944+0000","last_fullsized":"2024-03-30T01:42:24.273944+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:46.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274114+0000","last_change":"2024-03-30T01:42:24.297682+0000","last_active":"2024-03-30T01:42:25.274114+0000","last_peered":"2024-03-30T01:42:25.274114+0000","last_clean":"2024-03-30T01:42:25.274114+0000","last_became_active":"2024-03-30T01:42:24.297410+0000","last_became_peered":"2024-03-30T01:42:24.297410+0000","last_unstale":"2024-03-30T01:42:25.274114+0000","last_undegraded":"2024-03-30T01:42:25.274114+0000","last_fullsized":"2024-03-30T01:42:25.274114+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:58.973438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277619+0000","last_change":"2024-03-30T01:42:09.826895+0000","last_active":"2024-03-30T01:42:24.277619+0000","last_peered":"2024-03-30T01:42:24.277619+0000","last_clean":"2024-03-30T01:42:24.277619+0000","last_became_active":"2024-03-30T01:42:09.826609+0000","last_became_peered":"2024-03-30T01:42:09.826609+0000","last_unstale":"2024-03-30T01:42:24.277619+0000","last_undegraded":"2024-03-30T01:42:24.277619+0000","last_fullsized":"2024-03-30T01:42:24.277619+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:42:59.036073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291816+0000","last_change":"2024-03-30T01:42:10.291564+0000","last_active":"2024-03-30T01:42:24.291816+0000","last_peered":"2024-03-30T01:42:24.291816+0000","last_clean":"2024-03-30T01:42:24.291816+0000","last_became_active":"2024-03-30T01:42:10.291285+0000","last_became_peered":"2024-03-30T01:42:10.291285+0000","last_unstale":"2024-03-30T01:42:24.291816+0000","last_undegraded":"2024-03-30T01:42:24.291816+0000","last_fullsized":"2024-03-30T01:42:24.291816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.301787+0000","last_change":"2024-03-30T01:42:23.829970+0000","last_active":"2024-03-30T01:42:24.301787+0000","last_peered":"2024-03-30T01:42:24.301787+0000","last_clean":"2024-03-30T01:42:24.301787+0000","last_became_active":"2024-03-30T01:42:23.829639+0000","last_became_peered":"2024-03-30T01:42:23.829639+0000","last_unstale":"2024-03-30T01:42:24.301787+0000","last_undegraded":"2024-03-30T01:42:24.301787+0000","last_fullsized":"2024-03-30T01:42:24.301787+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:49.116227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813166+0000","last_change":"2024-03-30T01:42:24.302754+0000","last_active":"2024-03-30T01:42:25.813166+0000","last_peered":"2024-03-30T01:42:25.813166+0000","last_clean":"2024-03-30T01:42:25.813166+0000","last_became_active":"2024-03-30T01:42:24.302449+0000","last_became_peered":"2024-03-30T01:42:24.302449+0000","last_unstale":"2024-03-30T01:42:25.813166+0000","last_undegraded":"2024-03-30T01:42:25.813166+0000","last_fullsized":"2024-03-30T01:42:25.813166+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:57.853997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":23,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282222+0000","last_change":"2024-03-30T01:42:09.831869+0000","last_active":"2024-03-30T01:42:24.282222+0000","last_peered":"2024-03-30T01:42:24.282222+0000","last_clean":"2024-03-30T01:42:24.282222+0000","last_became_active":"2024-03-30T01:42:09.831562+0000","last_became_peered":"2024-03-30T01:42:09.831562+0000","last_unstale":"2024-03-30T01:42:24.282222+0000","last_undegraded":"2024-03-30T01:42:24.282222+0000","last_fullsized":"2024-03-30T01:42:24.282222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:35.938001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290305+0000","last_change":"2024-03-30T01:42:10.288399+0000","last_active":"2024-03-30T01:42:24.290305+0000","last_peered":"2024-03-30T01:42:24.290305+0000","last_clean":"2024-03-30T01:42:24.290305+0000","last_became_active":"2024-03-30T01:42:10.287890+0000","last_became_peered":"2024-03-30T01:42:10.287890+0000","last_unstale":"2024-03-30T01:42:24.290305+0000","last_undegraded":"2024-03-30T01:42:24.290305+0000","last_fullsized":"2024-03-30T01:42:24.290305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273363+0000","last_change":"2024-03-30T01:42:23.832729+0000","last_active":"2024-03-30T01:42:24.273363+0000","last_peered":"2024-03-30T01:42:24.273363+0000","last_clean":"2024-03-30T01:42:24.273363+0000","last_became_active":"2024-03-30T01:42:23.832474+0000","last_became_peered":"2024-03-30T01:42:23.832474+0000","last_unstale":"2024-03-30T01:42:24.273363+0000","last_undegraded":"2024-03-30T01:42:24.273363+0000","last_fullsized":"2024-03-30T01:42:24.273363+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:59.368657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606839+0000","last_change":"2024-03-30T01:42:24.296757+0000","last_active":"2024-03-30T01:42:25.606839+0000","last_peered":"2024-03-30T01:42:25.606839+0000","last_clean":"2024-03-30T01:42:25.606839+0000","last_became_active":"2024-03-30T01:42:24.296481+0000","last_became_peered":"2024-03-30T01:42:24.296481+0000","last_unstale":"2024-03-30T01:42:25.606839+0000","last_undegraded":"2024-03-30T01:42:25.606839+0000","last_fullsized":"2024-03-30T01:42:25.606839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:50:12.648133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273090+0000","last_change":"2024-03-30T01:42:09.820451+0000","last_active":"2024-03-30T01:42:24.273090+0000","last_peered":"2024-03-30T01:42:24.273090+0000","last_clean":"2024-03-30T01:42:24.273090+0000","last_became_active":"2024-03-30T01:42:09.820152+0000","last_became_peered":"2024-03-30T01:42:09.820152+0000","last_unstale":"2024-03-30T01:42:24.273090+0000","last_undegraded":"2024-03-30T01:42:24.273090+0000","last_fullsized":"2024-03-30T01:42:24.273090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:04.153303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282489+0000","last_change":"2024-03-30T01:42:10.279681+0000","last_active":"2024-03-30T01:42:24.282489+0000","last_peered":"2024-03-30T01:42:24.282489+0000","last_clean":"2024-03-30T01:42:24.282489+0000","last_became_active":"2024-03-30T01:42:10.279333+0000","last_became_peered":"2024-03-30T01:42:10.279333+0000","last_unstale":"2024-03-30T01:42:24.282489+0000","last_undegraded":"2024-03-30T01:42:24.282489+0000","last_fullsized":"2024-03-30T01:42:24.282489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.300487+0000","last_change":"2024-03-30T01:42:23.833668+0000","last_active":"2024-03-30T01:42:24.300487+0000","last_peered":"2024-03-30T01:42:24.300487+0000","last_clean":"2024-03-30T01:42:24.300487+0000","last_became_active":"2024-03-30T01:42:23.832884+0000","last_became_peered":"2024-03-30T01:42:23.832884+0000","last_unstale":"2024-03-30T01:42:24.300487+0000","last_undegraded":"2024-03-30T01:42:24.300487+0000","last_fullsized":"2024-03-30T01:42:24.300487+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.379389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275298+0000","last_change":"2024-03-30T01:42:24.292573+0000","last_active":"2024-03-30T01:42:25.275298+0000","last_peered":"2024-03-30T01:42:25.275298+0000","last_clean":"2024-03-30T01:42:25.275298+0000","last_became_active":"2024-03-30T01:42:24.292282+0000","last_became_peered":"2024-03-30T01:42:24.292282+0000","last_unstale":"2024-03-30T01:42:25.275298+0000","last_undegraded":"2024-03-30T01:42:25.275298+0000","last_fullsized":"2024-03-30T01:42:25.275298+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:35.580791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.285427+0000","last_change":"2024-03-30T01:42:09.829194+0000","last_active":"2024-03-30T01:42:24.285427+0000","last_peered":"2024-03-30T01:42:24.285427+0000","last_clean":"2024-03-30T01:42:24.285427+0000","last_became_active":"2024-03-30T01:42:09.828257+0000","last_became_peered":"2024-03-30T01:42:09.828257+0000","last_unstale":"2024-03-30T01:42:24.285427+0000","last_undegraded":"2024-03-30T01:42:24.285427+0000","last_fullsized":"2024-03-30T01:42:24.285427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.289884+0000","last_change":"2024-03-30T01:42:10.288326+0000","last_active":"2024-03-30T01:42:24.289884+0000","last_peered":"2024-03-30T01:42:24.289884+0000","last_clean":"2024-03-30T01:42:24.289884+0000","last_became_active":"2024-03-30T01:42:10.288050+0000","last_became_peered":"2024-03-30T01:42:10.288050+0000","last_unstale":"2024-03-30T01:42:24.289884+0000","last_undegraded":"2024-03-30T01:42:24.289884+0000","last_fullsized":"2024-03-30T01:42:24.289884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"42'8","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.309918+0000","last_change":"2024-03-30T01:42:23.836559+0000","last_active":"2024-03-30T01:42:24.309918+0000","last_peered":"2024-03-30T01:42:24.309918+0000","last_clean":"2024-03-30T01:42:24.309918+0000","last_became_active":"2024-03-30T01:42:23.836272+0000","last_became_peered":"2024-03-30T01:42:23.836272+0000","last_unstale":"2024-03-30T01:42:24.309918+0000","last_undegraded":"2024-03-30T01:42:24.309918+0000","last_fullsized":"2024-03-30T01:42:24.309918+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:20.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.607152+0000","last_change":"2024-03-30T01:42:24.296963+0000","last_active":"2024-03-30T01:42:25.607152+0000","last_peered":"2024-03-30T01:42:25.607152+0000","last_clean":"2024-03-30T01:42:25.607152+0000","last_became_active":"2024-03-30T01:42:24.288206+0000","last_became_peered":"2024-03-30T01:42:24.288206+0000","last_unstale":"2024-03-30T01:42:25.607152+0000","last_undegraded":"2024-03-30T01:42:25.607152+0000","last_fullsized":"2024-03-30T01:42:25.607152+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:14:43.146891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.289989+0000","last_change":"2024-03-30T01:42:09.832318+0000","last_active":"2024-03-30T01:42:24.289989+0000","last_peered":"2024-03-30T01:42:24.289989+0000","last_clean":"2024-03-30T01:42:24.289989+0000","last_became_active":"2024-03-30T01:42:09.831964+0000","last_became_peered":"2024-03-30T01:42:09.831964+0000","last_unstale":"2024-03-30T01:42:24.289989+0000","last_undegraded":"2024-03-30T01:42:24.289989+0000","last_fullsized":"2024-03-30T01:42:24.289989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:05.700970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278549+0000","last_change":"2024-03-30T01:42:10.285053+0000","last_active":"2024-03-30T01:42:24.278549+0000","last_peered":"2024-03-30T01:42:24.278549+0000","last_clean":"2024-03-30T01:42:24.278549+0000","last_became_active":"2024-03-30T01:42:10.284814+0000","last_became_peered":"2024-03-30T01:42:10.284814+0000","last_unstale":"2024-03-30T01:42:24.278549+0000","last_undegraded":"2024-03-30T01:42:24.278549+0000","last_fullsized":"2024-03-30T01:42:24.278549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:28.875197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283983+0000","last_change":"2024-03-30T01:42:23.822556+0000","last_active":"2024-03-30T01:42:24.283983+0000","last_peered":"2024-03-30T01:42:24.283983+0000","last_clean":"2024-03-30T01:42:24.283983+0000","last_became_active":"2024-03-30T01:42:23.822232+0000","last_became_peered":"2024-03-30T01:42:23.822232+0000","last_unstale":"2024-03-30T01:42:24.283983+0000","last_undegraded":"2024-03-30T01:42:24.283983+0000","last_fullsized":"2024-03-30T01:42:24.283983+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:29:59.201428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812571+0000","last_change":"2024-03-30T01:42:24.301362+0000","last_active":"2024-03-30T01:42:25.812571+0000","last_peered":"2024-03-30T01:42:25.812571+0000","last_clean":"2024-03-30T01:42:25.812571+0000","last_became_active":"2024-03-30T01:42:24.301060+0000","last_became_peered":"2024-03-30T01:42:24.301060+0000","last_unstale":"2024-03-30T01:42:25.812571+0000","last_undegraded":"2024-03-30T01:42:25.812571+0000","last_fullsized":"2024-03-30T01:42:25.812571+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:15:46.581141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"39'8","reported_seq":23,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291223+0000","last_change":"2024-03-30T01:42:09.823152+0000","last_active":"2024-03-30T01:42:24.291223+0000","last_peered":"2024-03-30T01:42:24.291223+0000","last_clean":"2024-03-30T01:42:24.291223+0000","last_became_active":"2024-03-30T01:42:09.822826+0000","last_became_peered":"2024-03-30T01:42:09.822826+0000","last_unstale":"2024-03-30T01:42:24.291223+0000","last_undegraded":"2024-03-30T01:42:24.291223+0000","last_fullsized":"2024-03-30T01:42:24.291223+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:17.799215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283446+0000","last_change":"2024-03-30T01:42:10.296944+0000","last_active":"2024-03-30T01:42:24.283446+0000","last_peered":"2024-03-30T01:42:24.283446+0000","last_clean":"2024-03-30T01:42:24.283446+0000","last_became_active":"2024-03-30T01:42:10.296714+0000","last_became_peered":"2024-03-30T01:42:10.296714+0000","last_unstale":"2024-03-30T01:42:24.283446+0000","last_undegraded":"2024-03-30T01:42:24.283446+0000","last_fullsized":"2024-03-30T01:42:24.283446+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283769+0000","last_change":"2024-03-30T01:42:23.836656+0000","last_active":"2024-03-30T01:42:24.283769+0000","last_peered":"2024-03-30T01:42:24.283769+0000","last_clean":"2024-03-30T01:42:24.283769+0000","last_became_active":"2024-03-30T01:42:23.836352+0000","last_became_peered":"2024-03-30T01:42:23.836352+0000","last_unstale":"2024-03-30T01:42:24.283769+0000","last_undegraded":"2024-03-30T01:42:24.283769+0000","last_fullsized":"2024-03-30T01:42:24.283769+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275090+0000","last_change":"2024-03-30T01:42:24.288096+0000","last_active":"2024-03-30T01:42:25.275090+0000","last_peered":"2024-03-30T01:42:25.275090+0000","last_clean":"2024-03-30T01:42:25.275090+0000","last_became_active":"2024-03-30T01:42:24.287765+0000","last_became_peered":"2024-03-30T01:42:24.287765+0000","last_unstale":"2024-03-30T01:42:25.275090+0000","last_undegraded":"2024-03-30T01:42:25.275090+0000","last_fullsized":"2024-03-30T01:42:25.275090+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:57.853997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277360+0000","last_change":"2024-03-30T01:42:09.833439+0000","last_active":"2024-03-30T01:42:24.277360+0000","last_peered":"2024-03-30T01:42:24.277360+0000","last_clean":"2024-03-30T01:42:24.277360+0000","last_became_active":"2024-03-30T01:42:09.833137+0000","last_became_peered":"2024-03-30T01:42:09.833137+0000","last_unstale":"2024-03-30T01:42:24.277360+0000","last_undegraded":"2024-03-30T01:42:24.277360+0000","last_fullsized":"2024-03-30T01:42:24.277360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280507+0000","last_change":"2024-03-30T01:42:10.292182+0000","last_active":"2024-03-30T01:42:24.280507+0000","last_peered":"2024-03-30T01:42:24.280507+0000","last_clean":"2024-03-30T01:42:24.280507+0000","last_became_active":"2024-03-30T01:42:10.291868+0000","last_became_peered":"2024-03-30T01:42:10.291868+0000","last_unstale":"2024-03-30T01:42:24.280507+0000","last_undegraded":"2024-03-30T01:42:24.280507+0000","last_fullsized":"2024-03-30T01:42:24.280507+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282905+0000","last_change":"2024-03-30T01:42:23.829400+0000","last_active":"2024-03-30T01:42:24.282905+0000","last_peered":"2024-03-30T01:42:24.282905+0000","last_clean":"2024-03-30T01:42:24.282905+0000","last_became_active":"2024-03-30T01:42:23.829058+0000","last_became_peered":"2024-03-30T01:42:23.829058+0000","last_unstale":"2024-03-30T01:42:24.282905+0000","last_undegraded":"2024-03-30T01:42:24.282905+0000","last_fullsized":"2024-03-30T01:42:24.282905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:12.384136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811906+0000","last_change":"2024-03-30T01:42:24.300333+0000","last_active":"2024-03-30T01:42:25.811906+0000","last_peered":"2024-03-30T01:42:25.811906+0000","last_clean":"2024-03-30T01:42:25.811906+0000","last_became_active":"2024-03-30T01:42:24.300127+0000","last_became_peered":"2024-03-30T01:42:24.300127+0000","last_unstale":"2024-03-30T01:42:25.811906+0000","last_undegraded":"2024-03-30T01:42:25.811906+0000","last_fullsized":"2024-03-30T01:42:25.811906+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:44.799982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277811+0000","last_change":"2024-03-30T01:42:09.826561+0000","last_active":"2024-03-30T01:42:24.277811+0000","last_peered":"2024-03-30T01:42:24.277811+0000","last_clean":"2024-03-30T01:42:24.277811+0000","last_became_active":"2024-03-30T01:42:09.826059+0000","last_became_peered":"2024-03-30T01:42:09.826059+0000","last_unstale":"2024-03-30T01:42:24.277811+0000","last_undegraded":"2024-03-30T01:42:24.277811+0000","last_fullsized":"2024-03-30T01:42:24.277811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.284182+0000","last_change":"2024-03-30T01:42:10.284755+0000","last_active":"2024-03-30T01:42:24.284182+0000","last_peered":"2024-03-30T01:42:24.284182+0000","last_clean":"2024-03-30T01:42:24.284182+0000","last_became_active":"2024-03-30T01:42:10.284421+0000","last_became_peered":"2024-03-30T01:42:10.284421+0000","last_unstale":"2024-03-30T01:42:24.284182+0000","last_undegraded":"2024-03-30T01:42:24.284182+0000","last_fullsized":"2024-03-30T01:42:24.284182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"42'6","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.312657+0000","last_change":"2024-03-30T01:42:23.837809+0000","last_active":"2024-03-30T01:42:24.312657+0000","last_peered":"2024-03-30T01:42:24.312657+0000","last_clean":"2024-03-30T01:42:24.312657+0000","last_became_active":"2024-03-30T01:42:23.837550+0000","last_became_peered":"2024-03-30T01:42:23.837550+0000","last_unstale":"2024-03-30T01:42:24.312657+0000","last_undegraded":"2024-03-30T01:42:24.312657+0000","last_fullsized":"2024-03-30T01:42:24.312657+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:44.146196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812751+0000","last_change":"2024-03-30T01:42:24.291310+0000","last_active":"2024-03-30T01:42:25.812751+0000","last_peered":"2024-03-30T01:42:25.812751+0000","last_clean":"2024-03-30T01:42:25.812751+0000","last_became_active":"2024-03-30T01:42:24.291034+0000","last_became_peered":"2024-03-30T01:42:24.291034+0000","last_unstale":"2024-03-30T01:42:25.812751+0000","last_undegraded":"2024-03-30T01:42:25.812751+0000","last_fullsized":"2024-03-30T01:42:25.812751+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:58.973438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297262+0000","last_change":"2024-03-30T01:42:09.828624+0000","last_active":"2024-03-30T01:42:24.297262+0000","last_peered":"2024-03-30T01:42:24.297262+0000","last_clean":"2024-03-30T01:42:24.297262+0000","last_became_active":"2024-03-30T01:42:09.828341+0000","last_became_peered":"2024-03-30T01:42:09.828341+0000","last_unstale":"2024-03-30T01:42:24.297262+0000","last_undegraded":"2024-03-30T01:42:24.297262+0000","last_fullsized":"2024-03-30T01:42:24.297262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:32.138427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275251+0000","last_change":"2024-03-30T01:42:10.295259+0000","last_active":"2024-03-30T01:42:24.275251+0000","last_peered":"2024-03-30T01:42:24.275251+0000","last_clean":"2024-03-30T01:42:24.275251+0000","last_became_active":"2024-03-30T01:42:10.295054+0000","last_became_peered":"2024-03-30T01:42:10.295054+0000","last_unstale":"2024-03-30T01:42:24.275251+0000","last_undegraded":"2024-03-30T01:42:24.275251+0000","last_fullsized":"2024-03-30T01:42:24.275251+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:36.942583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"42'3","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.301595+0000","last_change":"2024-03-30T01:42:23.832976+0000","last_active":"2024-03-30T01:42:24.301595+0000","last_peered":"2024-03-30T01:42:24.301595+0000","last_clean":"2024-03-30T01:42:24.301595+0000","last_became_active":"2024-03-30T01:42:23.832769+0000","last_became_peered":"2024-03-30T01:42:23.832769+0000","last_unstale":"2024-03-30T01:42:24.301595+0000","last_undegraded":"2024-03-30T01:42:24.301595+0000","last_fullsized":"2024-03-30T01:42:24.301595+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:54:59.010522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606373+0000","last_change":"2024-03-30T01:42:24.304851+0000","last_active":"2024-03-30T01:42:25.606373+0000","last_peered":"2024-03-30T01:42:25.606373+0000","last_clean":"2024-03-30T01:42:25.606373+0000","last_became_active":"2024-03-30T01:42:24.301522+0000","last_became_peered":"2024-03-30T01:42:24.301522+0000","last_unstale":"2024-03-30T01:42:25.606373+0000","last_undegraded":"2024-03-30T01:42:25.606373+0000","last_fullsized":"2024-03-30T01:42:25.606373+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:39.518546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283467+0000","last_change":"2024-03-30T01:42:09.820512+0000","last_active":"2024-03-30T01:42:24.283467+0000","last_peered":"2024-03-30T01:42:24.283467+0000","last_clean":"2024-03-30T01:42:24.283467+0000","last_became_active":"2024-03-30T01:42:09.820253+0000","last_became_peered":"2024-03-30T01:42:09.820253+0000","last_unstale":"2024-03-30T01:42:24.283467+0000","last_undegraded":"2024-03-30T01:42:24.283467+0000","last_fullsized":"2024-03-30T01:42:24.283467+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.455801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290107+0000","last_change":"2024-03-30T01:42:10.291872+0000","last_active":"2024-03-30T01:42:24.290107+0000","last_peered":"2024-03-30T01:42:24.290107+0000","last_clean":"2024-03-30T01:42:24.290107+0000","last_became_active":"2024-03-30T01:42:10.291199+0000","last_became_peered":"2024-03-30T01:42:10.291199+0000","last_unstale":"2024-03-30T01:42:24.290107+0000","last_undegraded":"2024-03-30T01:42:24.290107+0000","last_fullsized":"2024-03-30T01:42:24.290107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297204+0000","last_change":"2024-03-30T01:42:23.824219+0000","last_active":"2024-03-30T01:42:24.297204+0000","last_peered":"2024-03-30T01:42:24.297204+0000","last_clean":"2024-03-30T01:42:24.297204+0000","last_became_active":"2024-03-30T01:42:23.823949+0000","last_became_peered":"2024-03-30T01:42:23.823949+0000","last_unstale":"2024-03-30T01:42:24.297204+0000","last_undegraded":"2024-03-30T01:42:24.297204+0000","last_fullsized":"2024-03-30T01:42:24.297204+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813275+0000","last_change":"2024-03-30T01:42:24.301996+0000","last_active":"2024-03-30T01:42:25.813275+0000","last_peered":"2024-03-30T01:42:25.813275+0000","last_clean":"2024-03-30T01:42:25.813275+0000","last_became_active":"2024-03-30T01:42:24.301718+0000","last_became_peered":"2024-03-30T01:42:24.301718+0000","last_unstale":"2024-03-30T01:42:25.813275+0000","last_undegraded":"2024-03-30T01:42:25.813275+0000","last_fullsized":"2024-03-30T01:42:25.813275+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:45.153231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.286403+0000","last_change":"2024-03-30T01:42:09.826326+0000","last_active":"2024-03-30T01:42:24.286403+0000","last_peered":"2024-03-30T01:42:24.286403+0000","last_clean":"2024-03-30T01:42:24.286403+0000","last_became_active":"2024-03-30T01:42:09.826050+0000","last_became_peered":"2024-03-30T01:42:09.826050+0000","last_unstale":"2024-03-30T01:42:24.286403+0000","last_undegraded":"2024-03-30T01:42:24.286403+0000","last_fullsized":"2024-03-30T01:42:24.286403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.288773+0000","last_change":"2024-03-30T01:42:10.303359+0000","last_active":"2024-03-30T01:42:24.288773+0000","last_peered":"2024-03-30T01:42:24.288773+0000","last_clean":"2024-03-30T01:42:24.288773+0000","last_became_active":"2024-03-30T01:42:10.303039+0000","last_became_peered":"2024-03-30T01:42:10.303039+0000","last_unstale":"2024-03-30T01:42:24.288773+0000","last_undegraded":"2024-03-30T01:42:24.288773+0000","last_fullsized":"2024-03-30T01:42:24.288773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.300271+0000","last_change":"2024-03-30T01:42:23.830543+0000","last_active":"2024-03-30T01:42:24.300271+0000","last_peered":"2024-03-30T01:42:24.300271+0000","last_clean":"2024-03-30T01:42:24.300271+0000","last_became_active":"2024-03-30T01:42:23.830218+0000","last_became_peered":"2024-03-30T01:42:23.830218+0000","last_unstale":"2024-03-30T01:42:24.300271+0000","last_undegraded":"2024-03-30T01:42:24.300271+0000","last_fullsized":"2024-03-30T01:42:24.300271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:36.452658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812429+0000","last_change":"2024-03-30T01:42:24.302598+0000","last_active":"2024-03-30T01:42:25.812429+0000","last_peered":"2024-03-30T01:42:25.812429+0000","last_clean":"2024-03-30T01:42:25.812429+0000","last_became_active":"2024-03-30T01:42:24.302365+0000","last_became_peered":"2024-03-30T01:42:24.302365+0000","last_unstale":"2024-03-30T01:42:25.812429+0000","last_undegraded":"2024-03-30T01:42:25.812429+0000","last_fullsized":"2024-03-30T01:42:25.812429+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:45.153231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282526+0000","last_change":"2024-03-30T01:42:09.831981+0000","last_active":"2024-03-30T01:42:24.282526+0000","last_peered":"2024-03-30T01:42:24.282526+0000","last_clean":"2024-03-30T01:42:24.282526+0000","last_became_active":"2024-03-30T01:42:09.831647+0000","last_became_peered":"2024-03-30T01:42:09.831647+0000","last_unstale":"2024-03-30T01:42:24.282526+0000","last_undegraded":"2024-03-30T01:42:24.282526+0000","last_fullsized":"2024-03-30T01:42:24.282526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:02.883477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277377+0000","last_change":"2024-03-30T01:42:10.289213+0000","last_active":"2024-03-30T01:42:24.277377+0000","last_peered":"2024-03-30T01:42:24.277377+0000","last_clean":"2024-03-30T01:42:24.277377+0000","last_became_active":"2024-03-30T01:42:10.288677+0000","last_became_peered":"2024-03-30T01:42:10.288677+0000","last_unstale":"2024-03-30T01:42:24.277377+0000","last_undegraded":"2024-03-30T01:42:24.277377+0000","last_fullsized":"2024-03-30T01:42:24.277377+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278504+0000","last_change":"2024-03-30T01:42:23.829725+0000","last_active":"2024-03-30T01:42:24.278504+0000","last_peered":"2024-03-30T01:42:24.278504+0000","last_clean":"2024-03-30T01:42:24.278504+0000","last_became_active":"2024-03-30T01:42:23.829414+0000","last_became_peered":"2024-03-30T01:42:23.829414+0000","last_unstale":"2024-03-30T01:42:24.278504+0000","last_undegraded":"2024-03-30T01:42:24.278504+0000","last_fullsized":"2024-03-30T01:42:24.278504+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:32.897671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813494+0000","last_change":"2024-03-30T01:42:24.306123+0000","last_active":"2024-03-30T01:42:25.813494+0000","last_peered":"2024-03-30T01:42:25.813494+0000","last_clean":"2024-03-30T01:42:25.813494+0000","last_became_active":"2024-03-30T01:42:24.305878+0000","last_became_peered":"2024-03-30T01:42:24.305878+0000","last_unstale":"2024-03-30T01:42:25.813494+0000","last_undegraded":"2024-03-30T01:42:25.813494+0000","last_fullsized":"2024-03-30T01:42:25.813494+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:22:35.546913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277010+0000","last_change":"2024-03-30T01:42:09.831496+0000","last_active":"2024-03-30T01:42:24.277010+0000","last_peered":"2024-03-30T01:42:24.277010+0000","last_clean":"2024-03-30T01:42:24.277010+0000","last_became_active":"2024-03-30T01:42:09.831197+0000","last_became_peered":"2024-03-30T01:42:09.831197+0000","last_unstale":"2024-03-30T01:42:24.277010+0000","last_undegraded":"2024-03-30T01:42:24.277010+0000","last_fullsized":"2024-03-30T01:42:24.277010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278647+0000","last_change":"2024-03-30T01:42:10.287526+0000","last_active":"2024-03-30T01:42:24.278647+0000","last_peered":"2024-03-30T01:42:24.278647+0000","last_clean":"2024-03-30T01:42:24.278647+0000","last_became_active":"2024-03-30T01:42:10.287255+0000","last_became_peered":"2024-03-30T01:42:10.287255+0000","last_unstale":"2024-03-30T01:42:24.278647+0000","last_undegraded":"2024-03-30T01:42:24.278647+0000","last_fullsized":"2024-03-30T01:42:24.278647+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273224+0000","last_change":"2024-03-30T01:42:23.825156+0000","last_active":"2024-03-30T01:42:24.273224+0000","last_peered":"2024-03-30T01:42:24.273224+0000","last_clean":"2024-03-30T01:42:24.273224+0000","last_became_active":"2024-03-30T01:42:23.824623+0000","last_became_peered":"2024-03-30T01:42:23.824623+0000","last_unstale":"2024-03-30T01:42:24.273224+0000","last_undegraded":"2024-03-30T01:42:24.273224+0000","last_fullsized":"2024-03-30T01:42:24.273224+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:35.849177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812469+0000","last_change":"2024-03-30T01:42:24.299646+0000","last_active":"2024-03-30T01:42:25.812469+0000","last_peered":"2024-03-30T01:42:25.812469+0000","last_clean":"2024-03-30T01:42:25.812469+0000","last_became_active":"2024-03-30T01:42:24.298957+0000","last_became_peered":"2024-03-30T01:42:24.298957+0000","last_unstale":"2024-03-30T01:42:25.812469+0000","last_undegraded":"2024-03-30T01:42:25.812469+0000","last_fullsized":"2024-03-30T01:42:25.812469+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:49:25.818990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.289951+0000","last_change":"2024-03-30T01:42:09.824809+0000","last_active":"2024-03-30T01:42:24.289951+0000","last_peered":"2024-03-30T01:42:24.289951+0000","last_clean":"2024-03-30T01:42:24.289951+0000","last_became_active":"2024-03-30T01:42:09.824493+0000","last_became_peered":"2024-03-30T01:42:09.824493+0000","last_unstale":"2024-03-30T01:42:24.289951+0000","last_undegraded":"2024-03-30T01:42:24.289951+0000","last_fullsized":"2024-03-30T01:42:24.289951+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297494+0000","last_change":"2024-03-30T01:42:10.291979+0000","last_active":"2024-03-30T01:42:24.297494+0000","last_peered":"2024-03-30T01:42:24.297494+0000","last_clean":"2024-03-30T01:42:24.297494+0000","last_became_active":"2024-03-30T01:42:10.291708+0000","last_became_peered":"2024-03-30T01:42:10.291708+0000","last_unstale":"2024-03-30T01:42:24.297494+0000","last_undegraded":"2024-03-30T01:42:24.297494+0000","last_fullsized":"2024-03-30T01:42:24.297494+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.299720+0000","last_change":"2024-03-30T01:42:23.828108+0000","last_active":"2024-03-30T01:42:24.299720+0000","last_peered":"2024-03-30T01:42:24.299720+0000","last_clean":"2024-03-30T01:42:24.299720+0000","last_became_active":"2024-03-30T01:42:23.827597+0000","last_became_peered":"2024-03-30T01:42:23.827597+0000","last_unstale":"2024-03-30T01:42:24.299720+0000","last_undegraded":"2024-03-30T01:42:24.299720+0000","last_fullsized":"2024-03-30T01:42:24.299720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275252+0000","last_change":"2024-03-30T01:42:24.292625+0000","last_active":"2024-03-30T01:42:25.275252+0000","last_peered":"2024-03-30T01:42:25.275252+0000","last_clean":"2024-03-30T01:42:25.275252+0000","last_became_active":"2024-03-30T01:42:24.292322+0000","last_became_peered":"2024-03-30T01:42:24.292322+0000","last_unstale":"2024-03-30T01:42:25.275252+0000","last_undegraded":"2024-03-30T01:42:25.275252+0000","last_fullsized":"2024-03-30T01:42:25.275252+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:44.799982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290658+0000","last_change":"2024-03-30T01:42:09.829349+0000","last_active":"2024-03-30T01:42:24.290658+0000","last_peered":"2024-03-30T01:42:24.290658+0000","last_clean":"2024-03-30T01:42:24.290658+0000","last_became_active":"2024-03-30T01:42:09.828931+0000","last_became_peered":"2024-03-30T01:42:09.828931+0000","last_unstale":"2024-03-30T01:42:24.290658+0000","last_undegraded":"2024-03-30T01:42:24.290658+0000","last_fullsized":"2024-03-30T01:42:24.290658+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283722+0000","last_change":"2024-03-30T01:42:10.300292+0000","last_active":"2024-03-30T01:42:24.283722+0000","last_peered":"2024-03-30T01:42:24.283722+0000","last_clean":"2024-03-30T01:42:24.283722+0000","last_became_active":"2024-03-30T01:42:10.299867+0000","last_became_peered":"2024-03-30T01:42:10.299867+0000","last_unstale":"2024-03-30T01:42:24.283722+0000","last_undegraded":"2024-03-30T01:42:24.283722+0000","last_fullsized":"2024-03-30T01:42:24.283722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"42'4","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.298762+0000","last_change":"2024-03-30T01:42:23.822358+0000","last_active":"2024-03-30T01:42:24.298762+0000","last_peered":"2024-03-30T01:42:24.298762+0000","last_clean":"2024-03-30T01:42:24.298762+0000","last_became_active":"2024-03-30T01:42:23.822053+0000","last_became_peered":"2024-03-30T01:42:23.822053+0000","last_unstale":"2024-03-30T01:42:24.298762+0000","last_undegraded":"2024-03-30T01:42:24.298762+0000","last_fullsized":"2024-03-30T01:42:24.298762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:13.222722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812080+0000","last_change":"2024-03-30T01:42:24.292298+0000","last_active":"2024-03-30T01:42:25.812080+0000","last_peered":"2024-03-30T01:42:25.812080+0000","last_clean":"2024-03-30T01:42:25.812080+0000","last_became_active":"2024-03-30T01:42:24.292030+0000","last_became_peered":"2024-03-30T01:42:24.292030+0000","last_unstale":"2024-03-30T01:42:25.812080+0000","last_undegraded":"2024-03-30T01:42:25.812080+0000","last_fullsized":"2024-03-30T01:42:25.812080+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:44.799982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280617+0000","last_change":"2024-03-30T01:42:09.829899+0000","last_active":"2024-03-30T01:42:24.280617+0000","last_peered":"2024-03-30T01:42:24.280617+0000","last_clean":"2024-03-30T01:42:24.280617+0000","last_became_active":"2024-03-30T01:42:09.829601+0000","last_became_peered":"2024-03-30T01:42:09.829601+0000","last_unstale":"2024-03-30T01:42:24.280617+0000","last_undegraded":"2024-03-30T01:42:24.280617+0000","last_fullsized":"2024-03-30T01:42:24.280617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:30.887459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290890+0000","last_change":"2024-03-30T01:42:10.298381+0000","last_active":"2024-03-30T01:42:24.290890+0000","last_peered":"2024-03-30T01:42:24.290890+0000","last_clean":"2024-03-30T01:42:24.290890+0000","last_became_active":"2024-03-30T01:42:10.298109+0000","last_became_peered":"2024-03-30T01:42:10.298109+0000","last_unstale":"2024-03-30T01:42:24.290890+0000","last_undegraded":"2024-03-30T01:42:24.290890+0000","last_fullsized":"2024-03-30T01:42:24.290890+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.295295+0000","last_change":"2024-03-30T01:42:23.840369+0000","last_active":"2024-03-30T01:42:24.295295+0000","last_peered":"2024-03-30T01:42:24.295295+0000","last_clean":"2024-03-30T01:42:24.295295+0000","last_became_active":"2024-03-30T01:42:23.840105+0000","last_became_peered":"2024-03-30T01:42:23.840105+0000","last_unstale":"2024-03-30T01:42:24.295295+0000","last_undegraded":"2024-03-30T01:42:24.295295+0000","last_fullsized":"2024-03-30T01:42:24.295295+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811796+0000","last_change":"2024-03-30T01:42:24.299326+0000","last_active":"2024-03-30T01:42:25.811796+0000","last_peered":"2024-03-30T01:42:25.811796+0000","last_clean":"2024-03-30T01:42:25.811796+0000","last_became_active":"2024-03-30T01:42:24.299030+0000","last_became_peered":"2024-03-30T01:42:24.299030+0000","last_unstale":"2024-03-30T01:42:25.811796+0000","last_undegraded":"2024-03-30T01:42:25.811796+0000","last_fullsized":"2024-03-30T01:42:25.811796+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:57.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.274118+0000","last_change":"2024-03-30T01:42:09.824850+0000","last_active":"2024-03-30T01:42:24.274118+0000","last_peered":"2024-03-30T01:42:24.274118+0000","last_clean":"2024-03-30T01:42:24.274118+0000","last_became_active":"2024-03-30T01:42:09.824537+0000","last_became_peered":"2024-03-30T01:42:09.824537+0000","last_unstale":"2024-03-30T01:42:24.274118+0000","last_undegraded":"2024-03-30T01:42:24.274118+0000","last_fullsized":"2024-03-30T01:42:24.274118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:17.759503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282490+0000","last_change":"2024-03-30T01:42:10.285919+0000","last_active":"2024-03-30T01:42:24.282490+0000","last_peered":"2024-03-30T01:42:24.282490+0000","last_clean":"2024-03-30T01:42:24.282490+0000","last_became_active":"2024-03-30T01:42:10.285629+0000","last_became_peered":"2024-03-30T01:42:10.285629+0000","last_unstale":"2024-03-30T01:42:24.282490+0000","last_undegraded":"2024-03-30T01:42:24.282490+0000","last_fullsized":"2024-03-30T01:42:24.282490+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.300062+0000","last_change":"2024-03-30T01:42:23.831900+0000","last_active":"2024-03-30T01:42:24.300062+0000","last_peered":"2024-03-30T01:42:24.300062+0000","last_clean":"2024-03-30T01:42:24.300062+0000","last_became_active":"2024-03-30T01:42:23.831395+0000","last_became_peered":"2024-03-30T01:42:23.831395+0000","last_unstale":"2024-03-30T01:42:24.300062+0000","last_undegraded":"2024-03-30T01:42:24.300062+0000","last_fullsized":"2024-03-30T01:42:24.300062+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:35.672193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318446+0000","last_change":"2024-03-30T01:42:24.296156+0000","last_active":"2024-03-30T01:42:25.318446+0000","last_peered":"2024-03-30T01:42:25.318446+0000","last_clean":"2024-03-30T01:42:25.318446+0000","last_became_active":"2024-03-30T01:42:24.295854+0000","last_became_peered":"2024-03-30T01:42:24.295854+0000","last_unstale":"2024-03-30T01:42:25.318446+0000","last_undegraded":"2024-03-30T01:42:25.318446+0000","last_fullsized":"2024-03-30T01:42:25.318446+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:13.391171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813761+0000","last_change":"2024-03-30T01:42:24.297566+0000","last_active":"2024-03-30T01:42:25.813761+0000","last_peered":"2024-03-30T01:42:25.813761+0000","last_clean":"2024-03-30T01:42:25.813761+0000","last_became_active":"2024-03-30T01:42:24.297251+0000","last_became_peered":"2024-03-30T01:42:24.297251+0000","last_unstale":"2024-03-30T01:42:25.813761+0000","last_undegraded":"2024-03-30T01:42:25.813761+0000","last_fullsized":"2024-03-30T01:42:25.813761+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:57.853997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.286985+0000","last_change":"2024-03-30T01:42:10.297189+0000","last_active":"2024-03-30T01:42:24.286985+0000","last_peered":"2024-03-30T01:42:24.286985+0000","last_clean":"2024-03-30T01:42:24.286985+0000","last_became_active":"2024-03-30T01:42:10.296821+0000","last_became_peered":"2024-03-30T01:42:10.296821+0000","last_unstale":"2024-03-30T01:42:24.286985+0000","last_undegraded":"2024-03-30T01:42:24.286985+0000","last_fullsized":"2024-03-30T01:42:24.286985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606988+0000","last_change":"2024-03-30T01:42:24.296647+0000","last_active":"2024-03-30T01:42:25.606988+0000","last_peered":"2024-03-30T01:42:25.606988+0000","last_clean":"2024-03-30T01:42:25.606988+0000","last_became_active":"2024-03-30T01:42:24.288648+0000","last_became_peered":"2024-03-30T01:42:24.288648+0000","last_unstale":"2024-03-30T01:42:25.606988+0000","last_undegraded":"2024-03-30T01:42:25.606988+0000","last_fullsized":"2024-03-30T01:42:25.606988+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:10.469402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283864+0000","last_change":"2024-03-30T01:42:10.274200+0000","last_active":"2024-03-30T01:42:24.283864+0000","last_peered":"2024-03-30T01:42:24.283864+0000","last_clean":"2024-03-30T01:42:24.283864+0000","last_became_active":"2024-03-30T01:42:10.273923+0000","last_became_peered":"2024-03-30T01:42:10.273923+0000","last_unstale":"2024-03-30T01:42:24.283864+0000","last_undegraded":"2024-03-30T01:42:24.283864+0000","last_fullsized":"2024-03-30T01:42:24.283864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:44.766475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318538+0000","last_change":"2024-03-30T01:42:24.298867+0000","last_active":"2024-03-30T01:42:25.318538+0000","last_peered":"2024-03-30T01:42:25.318538+0000","last_clean":"2024-03-30T01:42:25.318538+0000","last_became_active":"2024-03-30T01:42:24.298557+0000","last_became_peered":"2024-03-30T01:42:24.298557+0000","last_unstale":"2024-03-30T01:42:25.318538+0000","last_undegraded":"2024-03-30T01:42:25.318538+0000","last_fullsized":"2024-03-30T01:42:25.318538+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:02:13.092965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273427+0000","last_change":"2024-03-30T01:42:10.290078+0000","last_active":"2024-03-30T01:42:24.273427+0000","last_peered":"2024-03-30T01:42:24.273427+0000","last_clean":"2024-03-30T01:42:24.273427+0000","last_became_active":"2024-03-30T01:42:10.289127+0000","last_became_peered":"2024-03-30T01:42:10.289127+0000","last_unstale":"2024-03-30T01:42:24.273427+0000","last_undegraded":"2024-03-30T01:42:24.273427+0000","last_fullsized":"2024-03-30T01:42:24.273427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:36.303285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606622+0000","last_change":"2024-03-30T01:42:24.288253+0000","last_active":"2024-03-30T01:42:25.606622+0000","last_peered":"2024-03-30T01:42:25.606622+0000","last_clean":"2024-03-30T01:42:25.606622+0000","last_became_active":"2024-03-30T01:42:24.287967+0000","last_became_peered":"2024-03-30T01:42:24.287967+0000","last_unstale":"2024-03-30T01:42:25.606622+0000","last_undegraded":"2024-03-30T01:42:25.606622+0000","last_fullsized":"2024-03-30T01:42:25.606622+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:01.354401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275460+0000","last_change":"2024-03-30T01:42:10.291099+0000","last_active":"2024-03-30T01:42:24.275460+0000","last_peered":"2024-03-30T01:42:24.275460+0000","last_clean":"2024-03-30T01:42:24.275460+0000","last_became_active":"2024-03-30T01:42:10.290802+0000","last_became_peered":"2024-03-30T01:42:10.290802+0000","last_unstale":"2024-03-30T01:42:24.275460+0000","last_undegraded":"2024-03-30T01:42:24.275460+0000","last_fullsized":"2024-03-30T01:42:24.275460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:13.070677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811731+0000","last_change":"2024-03-30T01:42:24.302481+0000","last_active":"2024-03-30T01:42:25.811731+0000","last_peered":"2024-03-30T01:42:25.811731+0000","last_clean":"2024-03-30T01:42:25.811731+0000","last_became_active":"2024-03-30T01:42:24.302183+0000","last_became_peered":"2024-03-30T01:42:24.302183+0000","last_unstale":"2024-03-30T01:42:25.811731+0000","last_undegraded":"2024-03-30T01:42:25.811731+0000","last_fullsized":"2024-03-30T01:42:25.811731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:34.136863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277627+0000","last_change":"2024-03-30T01:42:10.288633+0000","last_active":"2024-03-30T01:42:24.277627+0000","last_peered":"2024-03-30T01:42:24.277627+0000","last_clean":"2024-03-30T01:42:24.277627+0000","last_became_active":"2024-03-30T01:42:10.288326+0000","last_became_peered":"2024-03-30T01:42:10.288326+0000","last_unstale":"2024-03-30T01:42:24.277627+0000","last_undegraded":"2024-03-30T01:42:24.277627+0000","last_fullsized":"2024-03-30T01:42:24.277627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318329+0000","last_change":"2024-03-30T01:42:24.295937+0000","last_active":"2024-03-30T01:42:25.318329+0000","last_peered":"2024-03-30T01:42:25.318329+0000","last_clean":"2024-03-30T01:42:25.318329+0000","last_became_active":"2024-03-30T01:42:24.295604+0000","last_became_peered":"2024-03-30T01:42:24.295604+0000","last_unstale":"2024-03-30T01:42:25.318329+0000","last_undegraded":"2024-03-30T01:42:25.318329+0000","last_fullsized":"2024-03-30T01:42:25.318329+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:00.017558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.274341+0000","last_change":"2024-03-30T01:42:10.274845+0000","last_active":"2024-03-30T01:42:24.274341+0000","last_peered":"2024-03-30T01:42:24.274341+0000","last_clean":"2024-03-30T01:42:24.274341+0000","last_became_active":"2024-03-30T01:42:10.274566+0000","last_became_peered":"2024-03-30T01:42:10.274566+0000","last_unstale":"2024-03-30T01:42:24.274341+0000","last_undegraded":"2024-03-30T01:42:24.274341+0000","last_fullsized":"2024-03-30T01:42:24.274341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:32.801396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606081+0000","last_change":"2024-03-30T01:42:24.295712+0000","last_active":"2024-03-30T01:42:25.606081+0000","last_peered":"2024-03-30T01:42:25.606081+0000","last_clean":"2024-03-30T01:42:25.606081+0000","last_became_active":"2024-03-30T01:42:24.295459+0000","last_became_peered":"2024-03-30T01:42:24.295459+0000","last_unstale":"2024-03-30T01:42:25.606081+0000","last_undegraded":"2024-03-30T01:42:25.606081+0000","last_fullsized":"2024-03-30T01:42:25.606081+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:51.251893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290183+0000","last_change":"2024-03-30T01:42:10.290378+0000","last_active":"2024-03-30T01:42:24.290183+0000","last_peered":"2024-03-30T01:42:24.290183+0000","last_clean":"2024-03-30T01:42:24.290183+0000","last_became_active":"2024-03-30T01:42:10.290022+0000","last_became_peered":"2024-03-30T01:42:10.290022+0000","last_unstale":"2024-03-30T01:42:24.290183+0000","last_undegraded":"2024-03-30T01:42:24.290183+0000","last_fullsized":"2024-03-30T01:42:24.290183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275857+0000","last_change":"2024-03-30T01:42:24.292681+0000","last_active":"2024-03-30T01:42:25.275857+0000","last_peered":"2024-03-30T01:42:25.275857+0000","last_clean":"2024-03-30T01:42:25.275857+0000","last_became_active":"2024-03-30T01:42:24.292407+0000","last_became_peered":"2024-03-30T01:42:24.292407+0000","last_unstale":"2024-03-30T01:42:25.275857+0000","last_undegraded":"2024-03-30T01:42:25.275857+0000","last_fullsized":"2024-03-30T01:42:25.275857+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:33:13.335957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273835+0000","last_change":"2024-03-30T01:42:10.285378+0000","last_active":"2024-03-30T01:42:24.273835+0000","last_peered":"2024-03-30T01:42:24.273835+0000","last_clean":"2024-03-30T01:42:24.273835+0000","last_became_active":"2024-03-30T01:42:10.285105+0000","last_became_peered":"2024-03-30T01:42:10.285105+0000","last_unstale":"2024-03-30T01:42:24.273835+0000","last_undegraded":"2024-03-30T01:42:24.273835+0000","last_fullsized":"2024-03-30T01:42:24.273835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:52.824468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812316+0000","last_change":"2024-03-30T01:42:24.299833+0000","last_active":"2024-03-30T01:42:25.812316+0000","last_peered":"2024-03-30T01:42:25.812316+0000","last_clean":"2024-03-30T01:42:25.812316+0000","last_became_active":"2024-03-30T01:42:24.299327+0000","last_became_peered":"2024-03-30T01:42:24.299327+0000","last_unstale":"2024-03-30T01:42:25.812316+0000","last_undegraded":"2024-03-30T01:42:25.812316+0000","last_fullsized":"2024-03-30T01:42:25.812316+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:11.669273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.274758+0000","last_change":"2024-03-30T01:42:10.286887+0000","last_active":"2024-03-30T01:42:24.274758+0000","last_peered":"2024-03-30T01:42:24.274758+0000","last_clean":"2024-03-30T01:42:24.274758+0000","last_became_active":"2024-03-30T01:42:10.278107+0000","last_became_peered":"2024-03-30T01:42:10.278107+0000","last_unstale":"2024-03-30T01:42:24.274758+0000","last_undegraded":"2024-03-30T01:42:24.274758+0000","last_fullsized":"2024-03-30T01:42:24.274758+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.607019+0000","last_change":"2024-03-30T01:42:24.304463+0000","last_active":"2024-03-30T01:42:25.607019+0000","last_peered":"2024-03-30T01:42:25.607019+0000","last_clean":"2024-03-30T01:42:25.607019+0000","last_became_active":"2024-03-30T01:42:24.304127+0000","last_became_peered":"2024-03-30T01:42:24.304127+0000","last_unstale":"2024-03-30T01:42:25.607019+0000","last_undegraded":"2024-03-30T01:42:25.607019+0000","last_fullsized":"2024-03-30T01:42:25.607019+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:30.527405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.284166+0000","last_change":"2024-03-30T01:42:10.281331+0000","last_active":"2024-03-30T01:42:24.284166+0000","last_peered":"2024-03-30T01:42:24.284166+0000","last_clean":"2024-03-30T01:42:24.284166+0000","last_became_active":"2024-03-30T01:42:10.281011+0000","last_became_peered":"2024-03-30T01:42:10.281011+0000","last_unstale":"2024-03-30T01:42:24.284166+0000","last_undegraded":"2024-03-30T01:42:24.284166+0000","last_fullsized":"2024-03-30T01:42:24.284166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.288897+0000","last_change":"2024-03-30T01:42:10.299194+0000","last_active":"2024-03-30T01:42:24.288897+0000","last_peered":"2024-03-30T01:42:24.288897+0000","last_clean":"2024-03-30T01:42:24.288897+0000","last_became_active":"2024-03-30T01:42:10.298900+0000","last_became_peered":"2024-03-30T01:42:10.298900+0000","last_unstale":"2024-03-30T01:42:24.288897+0000","last_undegraded":"2024-03-30T01:42:24.288897+0000","last_fullsized":"2024-03-30T01:42:24.288897+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318379+0000","last_change":"2024-03-30T01:42:24.278409+0000","last_active":"2024-03-30T01:42:25.318379+0000","last_peered":"2024-03-30T01:42:25.318379+0000","last_clean":"2024-03-30T01:42:25.318379+0000","last_became_active":"2024-03-30T01:42:24.278120+0000","last_became_peered":"2024-03-30T01:42:24.278120+0000","last_unstale":"2024-03-30T01:42:25.318379+0000","last_undegraded":"2024-03-30T01:42:25.318379+0000","last_fullsized":"2024-03-30T01:42:25.318379+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:51.209327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275167+0000","last_change":"2024-03-30T01:42:24.307026+0000","last_active":"2024-03-30T01:42:25.275167+0000","last_peered":"2024-03-30T01:42:25.275167+0000","last_clean":"2024-03-30T01:42:25.275167+0000","last_became_active":"2024-03-30T01:42:24.306782+0000","last_became_peered":"2024-03-30T01:42:24.306782+0000","last_unstale":"2024-03-30T01:42:25.275167+0000","last_undegraded":"2024-03-30T01:42:25.275167+0000","last_fullsized":"2024-03-30T01:42:25.275167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:22:35.546913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.276861+0000","last_change":"2024-03-30T01:42:10.304038+0000","last_active":"2024-03-30T01:42:24.276861+0000","last_peered":"2024-03-30T01:42:24.276861+0000","last_clean":"2024-03-30T01:42:24.276861+0000","last_became_active":"2024-03-30T01:42:10.303747+0000","last_became_peered":"2024-03-30T01:42:10.303747+0000","last_unstale":"2024-03-30T01:42:24.276861+0000","last_undegraded":"2024-03-30T01:42:24.276861+0000","last_fullsized":"2024-03-30T01:42:24.276861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.274030+0000","last_change":"2024-03-30T01:42:10.300160+0000","last_active":"2024-03-30T01:42:24.274030+0000","last_peered":"2024-03-30T01:42:24.274030+0000","last_clean":"2024-03-30T01:42:24.274030+0000","last_became_active":"2024-03-30T01:42:10.299728+0000","last_became_peered":"2024-03-30T01:42:10.299728+0000","last_unstale":"2024-03-30T01:42:24.274030+0000","last_undegraded":"2024-03-30T01:42:24.274030+0000","last_fullsized":"2024-03-30T01:42:24.274030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812787+0000","last_change":"2024-03-30T01:42:24.307796+0000","last_active":"2024-03-30T01:42:25.812787+0000","last_peered":"2024-03-30T01:42:25.812787+0000","last_clean":"2024-03-30T01:42:25.812787+0000","last_became_active":"2024-03-30T01:42:24.307592+0000","last_became_peered":"2024-03-30T01:42:24.307592+0000","last_unstale":"2024-03-30T01:42:25.812787+0000","last_undegraded":"2024-03-30T01:42:25.812787+0000","last_fullsized":"2024-03-30T01:42:25.812787+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:21.224256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278021+0000","last_change":"2024-03-30T01:42:10.292351+0000","last_active":"2024-03-30T01:42:24.278021+0000","last_peered":"2024-03-30T01:42:24.278021+0000","last_clean":"2024-03-30T01:42:24.278021+0000","last_became_active":"2024-03-30T01:42:10.292040+0000","last_became_peered":"2024-03-30T01:42:10.292040+0000","last_unstale":"2024-03-30T01:42:24.278021+0000","last_undegraded":"2024-03-30T01:42:24.278021+0000","last_fullsized":"2024-03-30T01:42:24.278021+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:09.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606379+0000","last_change":"2024-03-30T01:42:24.300935+0000","last_active":"2024-03-30T01:42:25.606379+0000","last_peered":"2024-03-30T01:42:25.606379+0000","last_clean":"2024-03-30T01:42:25.606379+0000","last_became_active":"2024-03-30T01:42:24.300663+0000","last_became_peered":"2024-03-30T01:42:24.300663+0000","last_unstale":"2024-03-30T01:42:25.606379+0000","last_undegraded":"2024-03-30T01:42:25.606379+0000","last_fullsized":"2024-03-30T01:42:25.606379+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:12:17.200114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273912+0000","last_change":"2024-03-30T01:42:10.300393+0000","last_active":"2024-03-30T01:42:24.273912+0000","last_peered":"2024-03-30T01:42:24.273912+0000","last_clean":"2024-03-30T01:42:24.273912+0000","last_became_active":"2024-03-30T01:42:10.299795+0000","last_became_peered":"2024-03-30T01:42:10.299795+0000","last_unstale":"2024-03-30T01:42:24.273912+0000","last_undegraded":"2024-03-30T01:42:24.273912+0000","last_fullsized":"2024-03-30T01:42:24.273912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:21.256043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274563+0000","last_change":"2024-03-30T01:42:24.295808+0000","last_active":"2024-03-30T01:42:25.274563+0000","last_peered":"2024-03-30T01:42:25.274563+0000","last_clean":"2024-03-30T01:42:25.274563+0000","last_became_active":"2024-03-30T01:42:24.295490+0000","last_became_peered":"2024-03-30T01:42:24.295490+0000","last_unstale":"2024-03-30T01:42:25.274563+0000","last_undegraded":"2024-03-30T01:42:25.274563+0000","last_fullsized":"2024-03-30T01:42:25.274563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:33.904706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283022+0000","last_change":"2024-03-30T01:42:10.285733+0000","last_active":"2024-03-30T01:42:24.283022+0000","last_peered":"2024-03-30T01:42:24.283022+0000","last_clean":"2024-03-30T01:42:24.283022+0000","last_became_active":"2024-03-30T01:42:10.285428+0000","last_became_peered":"2024-03-30T01:42:10.285428+0000","last_unstale":"2024-03-30T01:42:24.283022+0000","last_undegraded":"2024-03-30T01:42:24.283022+0000","last_fullsized":"2024-03-30T01:42:24.283022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606152+0000","last_change":"2024-03-30T01:42:24.296607+0000","last_active":"2024-03-30T01:42:25.606152+0000","last_peered":"2024-03-30T01:42:25.606152+0000","last_clean":"2024-03-30T01:42:25.606152+0000","last_became_active":"2024-03-30T01:42:24.288603+0000","last_became_peered":"2024-03-30T01:42:24.288603+0000","last_unstale":"2024-03-30T01:42:25.606152+0000","last_undegraded":"2024-03-30T01:42:25.606152+0000","last_fullsized":"2024-03-30T01:42:25.606152+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:18.158278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813829+0000","last_change":"2024-03-30T01:42:24.297299+0000","last_active":"2024-03-30T01:42:25.813829+0000","last_peered":"2024-03-30T01:42:25.813829+0000","last_clean":"2024-03-30T01:42:25.813829+0000","last_became_active":"2024-03-30T01:42:24.297027+0000","last_became_peered":"2024-03-30T01:42:24.297027+0000","last_unstale":"2024-03-30T01:42:25.813829+0000","last_undegraded":"2024-03-30T01:42:25.813829+0000","last_fullsized":"2024-03-30T01:42:25.813829+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:58.973438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282495+0000","last_change":"2024-03-30T01:42:10.284419+0000","last_active":"2024-03-30T01:42:24.282495+0000","last_peered":"2024-03-30T01:42:24.282495+0000","last_clean":"2024-03-30T01:42:24.282495+0000","last_became_active":"2024-03-30T01:42:10.284098+0000","last_became_peered":"2024-03-30T01:42:10.284098+0000","last_unstale":"2024-03-30T01:42:24.282495+0000","last_undegraded":"2024-03-30T01:42:24.282495+0000","last_fullsized":"2024-03-30T01:42:24.282495+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274959+0000","last_change":"2024-03-30T01:42:24.299819+0000","last_active":"2024-03-30T01:42:25.274959+0000","last_peered":"2024-03-30T01:42:25.274959+0000","last_clean":"2024-03-30T01:42:25.274959+0000","last_became_active":"2024-03-30T01:42:24.299346+0000","last_became_peered":"2024-03-30T01:42:24.299346+0000","last_unstale":"2024-03-30T01:42:25.274959+0000","last_undegraded":"2024-03-30T01:42:25.274959+0000","last_fullsized":"2024-03-30T01:42:25.274959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:45.153231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277529+0000","last_change":"2024-03-30T01:42:10.293953+0000","last_active":"2024-03-30T01:42:24.277529+0000","last_peered":"2024-03-30T01:42:24.277529+0000","last_clean":"2024-03-30T01:42:24.277529+0000","last_became_active":"2024-03-30T01:42:10.293641+0000","last_became_peered":"2024-03-30T01:42:10.293641+0000","last_unstale":"2024-03-30T01:42:24.277529+0000","last_undegraded":"2024-03-30T01:42:24.277529+0000","last_fullsized":"2024-03-30T01:42:24.277529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813811+0000","last_change":"2024-03-30T01:42:24.291685+0000","last_active":"2024-03-30T01:42:25.813811+0000","last_peered":"2024-03-30T01:42:25.813811+0000","last_clean":"2024-03-30T01:42:25.813811+0000","last_became_active":"2024-03-30T01:42:24.291392+0000","last_became_peered":"2024-03-30T01:42:24.291392+0000","last_unstale":"2024-03-30T01:42:25.813811+0000","last_undegraded":"2024-03-30T01:42:25.813811+0000","last_fullsized":"2024-03-30T01:42:25.813811+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:24:32.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283414+0000","last_change":"2024-03-30T01:42:10.294667+0000","last_active":"2024-03-30T01:42:24.283414+0000","last_peered":"2024-03-30T01:42:24.283414+0000","last_clean":"2024-03-30T01:42:24.283414+0000","last_became_active":"2024-03-30T01:42:10.294330+0000","last_became_peered":"2024-03-30T01:42:10.294330+0000","last_unstale":"2024-03-30T01:42:24.283414+0000","last_undegraded":"2024-03-30T01:42:24.283414+0000","last_fullsized":"2024-03-30T01:42:24.283414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606681+0000","last_change":"2024-03-30T01:42:24.296243+0000","last_active":"2024-03-30T01:42:25.606681+0000","last_peered":"2024-03-30T01:42:25.606681+0000","last_clean":"2024-03-30T01:42:25.606681+0000","last_became_active":"2024-03-30T01:42:24.295980+0000","last_became_peered":"2024-03-30T01:42:24.295980+0000","last_unstale":"2024-03-30T01:42:25.606681+0000","last_undegraded":"2024-03-30T01:42:25.606681+0000","last_fullsized":"2024-03-30T01:42:25.606681+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:20:21.487199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.276735+0000","last_change":"2024-03-30T01:42:10.295808+0000","last_active":"2024-03-30T01:42:24.276735+0000","last_peered":"2024-03-30T01:42:24.276735+0000","last_clean":"2024-03-30T01:42:24.276735+0000","last_became_active":"2024-03-30T01:42:10.295460+0000","last_became_peered":"2024-03-30T01:42:10.295460+0000","last_unstale":"2024-03-30T01:42:24.276735+0000","last_undegraded":"2024-03-30T01:42:24.276735+0000","last_fullsized":"2024-03-30T01:42:24.276735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.273874+0000","last_change":"2024-03-30T01:42:24.295697+0000","last_active":"2024-03-30T01:42:25.273874+0000","last_peered":"2024-03-30T01:42:25.273874+0000","last_clean":"2024-03-30T01:42:25.273874+0000","last_became_active":"2024-03-30T01:42:24.295417+0000","last_became_peered":"2024-03-30T01:42:24.295417+0000","last_unstale":"2024-03-30T01:42:25.273874+0000","last_undegraded":"2024-03-30T01:42:25.273874+0000","last_fullsized":"2024-03-30T01:42:25.273874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:44.799982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278835+0000","last_change":"2024-03-30T01:42:10.294103+0000","last_active":"2024-03-30T01:42:24.278835+0000","last_peered":"2024-03-30T01:42:24.278835+0000","last_clean":"2024-03-30T01:42:24.278835+0000","last_became_active":"2024-03-30T01:42:10.293833+0000","last_became_peered":"2024-03-30T01:42:10.293833+0000","last_unstale":"2024-03-30T01:42:24.278835+0000","last_undegraded":"2024-03-30T01:42:24.278835+0000","last_fullsized":"2024-03-30T01:42:24.278835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812622+0000","last_change":"2024-03-30T01:42:24.292418+0000","last_active":"2024-03-30T01:42:25.812622+0000","last_peered":"2024-03-30T01:42:25.812622+0000","last_clean":"2024-03-30T01:42:25.812622+0000","last_became_active":"2024-03-30T01:42:24.292134+0000","last_became_peered":"2024-03-30T01:42:24.292134+0000","last_unstale":"2024-03-30T01:42:25.812622+0000","last_undegraded":"2024-03-30T01:42:25.812622+0000","last_fullsized":"2024-03-30T01:42:25.812622+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:21.224256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277244+0000","last_change":"2024-03-30T01:42:10.296160+0000","last_active":"2024-03-30T01:42:24.277244+0000","last_peered":"2024-03-30T01:42:24.277244+0000","last_clean":"2024-03-30T01:42:24.277244+0000","last_became_active":"2024-03-30T01:42:10.295867+0000","last_became_peered":"2024-03-30T01:42:10.295867+0000","last_unstale":"2024-03-30T01:42:24.277244+0000","last_undegraded":"2024-03-30T01:42:24.277244+0000","last_fullsized":"2024-03-30T01:42:24.277244+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811563+0000","last_change":"2024-03-30T01:42:24.302182+0000","last_active":"2024-03-30T01:42:25.811563+0000","last_peered":"2024-03-30T01:42:25.811563+0000","last_clean":"2024-03-30T01:42:25.811563+0000","last_became_active":"2024-03-30T01:42:24.301909+0000","last_became_peered":"2024-03-30T01:42:24.301909+0000","last_unstale":"2024-03-30T01:42:25.811563+0000","last_undegraded":"2024-03-30T01:42:25.811563+0000","last_fullsized":"2024-03-30T01:42:25.811563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:11:54.175140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277512+0000","last_change":"2024-03-30T01:42:10.295963+0000","last_active":"2024-03-30T01:42:24.277512+0000","last_peered":"2024-03-30T01:42:24.277512+0000","last_clean":"2024-03-30T01:42:24.277512+0000","last_became_active":"2024-03-30T01:42:10.295545+0000","last_became_peered":"2024-03-30T01:42:10.295545+0000","last_unstale":"2024-03-30T01:42:24.277512+0000","last_undegraded":"2024-03-30T01:42:24.277512+0000","last_fullsized":"2024-03-30T01:42:24.277512+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606146+0000","last_change":"2024-03-30T01:42:24.288825+0000","last_active":"2024-03-30T01:42:25.606146+0000","last_peered":"2024-03-30T01:42:25.606146+0000","last_clean":"2024-03-30T01:42:25.606146+0000","last_became_active":"2024-03-30T01:42:24.288324+0000","last_became_peered":"2024-03-30T01:42:24.288324+0000","last_unstale":"2024-03-30T01:42:25.606146+0000","last_undegraded":"2024-03-30T01:42:25.606146+0000","last_fullsized":"2024-03-30T01:42:25.606146+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:34.136863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280197+0000","last_change":"2024-03-30T01:42:10.285717+0000","last_active":"2024-03-30T01:42:24.280197+0000","last_peered":"2024-03-30T01:42:24.280197+0000","last_clean":"2024-03-30T01:42:24.280197+0000","last_became_active":"2024-03-30T01:42:10.285437+0000","last_became_peered":"2024-03-30T01:42:10.285437+0000","last_unstale":"2024-03-30T01:42:24.280197+0000","last_undegraded":"2024-03-30T01:42:24.280197+0000","last_fullsized":"2024-03-30T01:42:24.280197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274662+0000","last_change":"2024-03-30T01:42:24.306285+0000","last_active":"2024-03-30T01:42:25.274662+0000","last_peered":"2024-03-30T01:42:25.274662+0000","last_clean":"2024-03-30T01:42:25.274662+0000","last_became_active":"2024-03-30T01:42:24.305317+0000","last_became_peered":"2024-03-30T01:42:24.305317+0000","last_unstale":"2024-03-30T01:42:25.274662+0000","last_undegraded":"2024-03-30T01:42:25.274662+0000","last_fullsized":"2024-03-30T01:42:25.274662+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:24:32.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291778+0000","last_change":"2024-03-30T01:42:10.298731+0000","last_active":"2024-03-30T01:42:24.291778+0000","last_peered":"2024-03-30T01:42:24.291778+0000","last_clean":"2024-03-30T01:42:24.291778+0000","last_became_active":"2024-03-30T01:42:10.298525+0000","last_became_peered":"2024-03-30T01:42:10.298525+0000","last_unstale":"2024-03-30T01:42:24.291778+0000","last_undegraded":"2024-03-30T01:42:24.291778+0000","last_fullsized":"2024-03-30T01:42:24.291778+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812533+0000","last_change":"2024-03-30T01:42:24.306384+0000","last_active":"2024-03-30T01:42:25.812533+0000","last_peered":"2024-03-30T01:42:25.812533+0000","last_clean":"2024-03-30T01:42:25.812533+0000","last_became_active":"2024-03-30T01:42:24.305978+0000","last_became_peered":"2024-03-30T01:42:24.305978+0000","last_unstale":"2024-03-30T01:42:25.812533+0000","last_undegraded":"2024-03-30T01:42:25.812533+0000","last_fullsized":"2024-03-30T01:42:25.812533+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:39:42.096508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277222+0000","last_change":"2024-03-30T01:42:10.289068+0000","last_active":"2024-03-30T01:42:24.277222+0000","last_peered":"2024-03-30T01:42:24.277222+0000","last_clean":"2024-03-30T01:42:24.277222+0000","last_became_active":"2024-03-30T01:42:10.288282+0000","last_became_peered":"2024-03-30T01:42:10.288282+0000","last_unstale":"2024-03-30T01:42:24.277222+0000","last_undegraded":"2024-03-30T01:42:24.277222+0000","last_fullsized":"2024-03-30T01:42:24.277222+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812132+0000","last_change":"2024-03-30T01:42:24.289951+0000","last_active":"2024-03-30T01:42:25.812132+0000","last_peered":"2024-03-30T01:42:25.812132+0000","last_clean":"2024-03-30T01:42:25.812132+0000","last_became_active":"2024-03-30T01:42:24.289446+0000","last_became_peered":"2024-03-30T01:42:24.289446+0000","last_unstale":"2024-03-30T01:42:25.812132+0000","last_undegraded":"2024-03-30T01:42:25.812132+0000","last_fullsized":"2024-03-30T01:42:25.812132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:08.115455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.285326+0000","last_change":"2024-03-30T01:42:10.287001+0000","last_active":"2024-03-30T01:42:24.285326+0000","last_peered":"2024-03-30T01:42:24.285326+0000","last_clean":"2024-03-30T01:42:24.285326+0000","last_became_active":"2024-03-30T01:42:10.286700+0000","last_became_peered":"2024-03-30T01:42:10.286700+0000","last_unstale":"2024-03-30T01:42:24.285326+0000","last_undegraded":"2024-03-30T01:42:24.285326+0000","last_fullsized":"2024-03-30T01:42:24.285326+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.276236+0000","last_change":"2024-03-30T01:42:24.292271+0000","last_active":"2024-03-30T01:42:25.276236+0000","last_peered":"2024-03-30T01:42:25.276236+0000","last_clean":"2024-03-30T01:42:25.276236+0000","last_became_active":"2024-03-30T01:42:24.292005+0000","last_became_peered":"2024-03-30T01:42:24.292005+0000","last_unstale":"2024-03-30T01:42:25.276236+0000","last_undegraded":"2024-03-30T01:42:25.276236+0000","last_fullsized":"2024-03-30T01:42:25.276236+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:43.323501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275640+0000","last_change":"2024-03-30T01:42:10.299868+0000","last_active":"2024-03-30T01:42:24.275640+0000","last_peered":"2024-03-30T01:42:24.275640+0000","last_clean":"2024-03-30T01:42:24.275640+0000","last_became_active":"2024-03-30T01:42:10.299612+0000","last_became_peered":"2024-03-30T01:42:10.299612+0000","last_unstale":"2024-03-30T01:42:24.275640+0000","last_undegraded":"2024-03-30T01:42:24.275640+0000","last_fullsized":"2024-03-30T01:42:24.275640+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:49.471045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606566+0000","last_change":"2024-03-30T01:42:24.301867+0000","last_active":"2024-03-30T01:42:25.606566+0000","last_peered":"2024-03-30T01:42:25.606566+0000","last_clean":"2024-03-30T01:42:25.606566+0000","last_became_active":"2024-03-30T01:42:24.301239+0000","last_became_peered":"2024-03-30T01:42:24.301239+0000","last_unstale":"2024-03-30T01:42:25.606566+0000","last_undegraded":"2024-03-30T01:42:25.606566+0000","last_fullsized":"2024-03-30T01:42:25.606566+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:00.869560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275157+0000","last_change":"2024-03-30T01:42:10.294277+0000","last_active":"2024-03-30T01:42:24.275157+0000","last_peered":"2024-03-30T01:42:24.275157+0000","last_clean":"2024-03-30T01:42:24.275157+0000","last_became_active":"2024-03-30T01:42:10.293995+0000","last_became_peered":"2024-03-30T01:42:10.293995+0000","last_unstale":"2024-03-30T01:42:24.275157+0000","last_undegraded":"2024-03-30T01:42:24.275157+0000","last_fullsized":"2024-03-30T01:42:24.275157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:03.888060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274283+0000","last_change":"2024-03-30T01:42:24.306793+0000","last_active":"2024-03-30T01:42:25.274283+0000","last_peered":"2024-03-30T01:42:25.274283+0000","last_clean":"2024-03-30T01:42:25.274283+0000","last_became_active":"2024-03-30T01:42:24.306099+0000","last_became_peered":"2024-03-30T01:42:24.306099+0000","last_unstale":"2024-03-30T01:42:25.274283+0000","last_undegraded":"2024-03-30T01:42:25.274283+0000","last_fullsized":"2024-03-30T01:42:25.274283+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:43:06.308084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280198+0000","last_change":"2024-03-30T01:42:10.285050+0000","last_active":"2024-03-30T01:42:24.280198+0000","last_peered":"2024-03-30T01:42:24.280198+0000","last_clean":"2024-03-30T01:42:24.280198+0000","last_became_active":"2024-03-30T01:42:10.284771+0000","last_became_peered":"2024-03-30T01:42:10.284771+0000","last_unstale":"2024-03-30T01:42:24.280198+0000","last_undegraded":"2024-03-30T01:42:24.280198+0000","last_fullsized":"2024-03-30T01:42:24.280198+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.273680+0000","last_change":"2024-03-30T01:42:24.305711+0000","last_active":"2024-03-30T01:42:25.273680+0000","last_peered":"2024-03-30T01:42:25.273680+0000","last_clean":"2024-03-30T01:42:25.273680+0000","last_became_active":"2024-03-30T01:42:24.305425+0000","last_became_peered":"2024-03-30T01:42:24.305425+0000","last_unstale":"2024-03-30T01:42:25.273680+0000","last_undegraded":"2024-03-30T01:42:25.273680+0000","last_fullsized":"2024-03-30T01:42:25.273680+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:03:25.829367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291583+0000","last_change":"2024-03-30T01:42:10.294965+0000","last_active":"2024-03-30T01:42:24.291583+0000","last_peered":"2024-03-30T01:42:24.291583+0000","last_clean":"2024-03-30T01:42:24.291583+0000","last_became_active":"2024-03-30T01:42:10.294685+0000","last_became_peered":"2024-03-30T01:42:10.294685+0000","last_unstale":"2024-03-30T01:42:24.291583+0000","last_undegraded":"2024-03-30T01:42:24.291583+0000","last_fullsized":"2024-03-30T01:42:24.291583+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812681+0000","last_change":"2024-03-30T01:42:24.291578+0000","last_active":"2024-03-30T01:42:25.812681+0000","last_peered":"2024-03-30T01:42:25.812681+0000","last_clean":"2024-03-30T01:42:25.812681+0000","last_became_active":"2024-03-30T01:42:24.291309+0000","last_became_peered":"2024-03-30T01:42:24.291309+0000","last_unstale":"2024-03-30T01:42:25.812681+0000","last_undegraded":"2024-03-30T01:42:25.812681+0000","last_fullsized":"2024-03-30T01:42:25.812681+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:03:25.829367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297193+0000","last_change":"2024-03-30T01:42:10.288258+0000","last_active":"2024-03-30T01:42:24.297193+0000","last_peered":"2024-03-30T01:42:24.297193+0000","last_clean":"2024-03-30T01:42:24.297193+0000","last_became_active":"2024-03-30T01:42:10.287993+0000","last_became_peered":"2024-03-30T01:42:10.287993+0000","last_unstale":"2024-03-30T01:42:24.297193+0000","last_undegraded":"2024-03-30T01:42:24.297193+0000","last_fullsized":"2024-03-30T01:42:24.297193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1118208,"data_stored":817623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46600000000000003}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.623},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.41499999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.2009999999999996,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.442}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.63600000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":225220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.501}]},{"osd":1,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.3250000000000002,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.45700000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.38600000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.40100000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56399999999999995}]},{"osd":7,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":224236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":15.038,"5min":15.038,"15min":15.038},"last":0.46600000000000003}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":11.858000000000001,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.42899999999999999}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.372},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1289999999999996,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":11.99,"5min":11.99,"15min":11.99},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.624}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":15.006,"5min":15.006,"15min":15.006},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":11.926,"5min":11.926,"15min":11.926},"last":0.71499999999999997}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":14.974,"5min":14.974,"15min":14.974},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.960000000000001,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":14.923999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.66500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1101824,"data_stored":813475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.312},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.946999999999999,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.436}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":12.983000000000001,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.2279999999999998,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.61199999999999999}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.3869999999999996,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":5.468,"5min":5.468,"15min":5.468},"last":0.57099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":225620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":12.478999999999999,"5min":12.478999999999999,"15min":12.478999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.40799999999999997}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":12.425000000000001,"5min":12.425000000000001,"15min":12.425000000000001},"last":0.32000000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.434},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":13.217000000000001,"5min":13.217000000000001,"15min":13.217000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":8.7400000000000002,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":8.5640000000000001,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":8.7759999999999998,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":8.5190000000000001,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.66800000000000004}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.621},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":8.5939999999999994,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":0.68300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":228681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.379},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.34399999999999997}]},{"osd":1,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.30499999999999999}]},{"osd":2,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.44800000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.64600000000000002}]},{"osd":5,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.751}]},{"osd":6,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.73599999999999999}]},{"osd":7,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.77300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":225172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.58899999999999997}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":9.3970000000000002,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.45100000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":5.5289999999999999,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":9.452,"5min":9.452,"15min":9.452},"last":0.621}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.54900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":224098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":6.6760000000000002,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7480000000000002,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.7709999999999999,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.8410000000000002,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.747}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64400000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.629},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.76500000000000001}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:42:33.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:42:33.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.442+0000 7f4c4ba56700 1 Processor -- start 2024-03-30T01:42:33.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.442+0000 7f4c4ba56700 1 -- start start 2024-03-30T01:42:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c4ba56700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c4406b400 0x7f4c44151a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c4ba56700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4c440c4f60 0x7f4c440c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c4ba56700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4c44151f40 0x7f4c440c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c4ba56700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c44154830 con 0x7f4c440c4f60 2024-03-30T01:42:33.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c4ba56700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c440c5380 con 0x7f4c44151f40 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c4ba56700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c7fd0 con 0x7f4c4406b400 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c48ff1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4c440c4f60 0x7f4c440c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c48ff1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4c440c4f60 0x7f4c440c5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44734/0 (socket says 172.21.15.31:44734) 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c48ff1700 1 -- 172.21.15.31:0/3554003070 learned_addr learned my addr 172.21.15.31:0/3554003070 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c497f2700 1 --2- 172.21.15.31:0/3554003070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c4406b400 0x7f4c44151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c49ff3700 1 --2- 172.21.15.31:0/3554003070 >> v2:172.21.15.59:3300/0 conn(0x7f4c44151f40 0x7f4c440c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c48ff1700 1 -- 172.21.15.31:0/3554003070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c4406b400 msgr2=0x7f4c44151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c48ff1700 1 --2- 172.21.15.31:0/3554003070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c4406b400 0x7f4c44151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c48ff1700 1 -- 172.21.15.31:0/3554003070 >> v2:172.21.15.59:3300/0 conn(0x7f4c44151f40 msgr2=0x7f4c440c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c48ff1700 1 --2- 172.21.15.31:0/3554003070 >> v2:172.21.15.59:3300/0 conn(0x7f4c44151f40 0x7f4c440c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.443+0000 7f4c48ff1700 1 -- 172.21.15.31:0/3554003070 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c44154a20 con 0x7f4c440c4f60 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.444+0000 7f4c48ff1700 1 --2- 172.21.15.31:0/3554003070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440c4f60 0x7f4c440c5d70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4c4408b1f0 tx=0x7f4c34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400a619f52f5dd3f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.444+0000 7f4c3bfff700 1 -- 172.21.15.31:0/3554003070 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34011830 con 0x7f4c440c4f60 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.444+0000 7f4c3bfff700 1 -- 172.21.15.31:0/3554003070 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c34011990 con 0x7f4c440c4f60 2024-03-30T01:42:33.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.444+0000 7f4c3bfff700 1 -- 172.21.15.31:0/3554003070 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34011d20 con 0x7f4c440c4f60 2024-03-30T01:42:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.444+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3554003070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440c4f60 msgr2=0x7f4c440c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.444+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3554003070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440c4f60 0x7f4c440c5d70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4c4408b1f0 tx=0x7f4c34009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.445+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3554003070 shutdown_connections 2024-03-30T01:42:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.445+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3554003070 >> v2:172.21.15.59:3300/0 conn(0x7f4c44151f40 0x7f4c440c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.445+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3554003070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c4406b400 0x7f4c44151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.445+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3554003070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c440c4f60 0x7f4c440c5d70 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.445+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3554003070 >> 172.21.15.31:0/3554003070 conn(0x7f4c440ca0d0 msgr2=0x7f4c440ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.445+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3554003070 shutdown_connections 2024-03-30T01:42:33.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.445+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3554003070 wait complete. 2024-03-30T01:42:33.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.446+0000 7f4c4ba56700 1 Processor -- start 2024-03-30T01:42:33.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.446+0000 7f4c4ba56700 1 -- start start 2024-03-30T01:42:33.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.447+0000 7f4c4ba56700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c4406b400 0x7f4c441e7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.447+0000 7f4c4ba56700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c440c4f60 0x7f4c441ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.447+0000 7f4c4ba56700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c44151f40 0x7f4c441ed920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.447+0000 7f4c4ba56700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441ede60 con 0x7f4c4406b400 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.447+0000 7f4c4ba56700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440bf740 con 0x7f4c44151f40 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.447+0000 7f4c4ba56700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c0e40 con 0x7f4c440c4f60 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.447+0000 7f4c497f2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c4406b400 0x7f4c441e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.447+0000 7f4c497f2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c4406b400 0x7f4c441e7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44748/0 (socket says 172.21.15.31:44748) 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.447+0000 7f4c497f2700 1 -- 172.21.15.31:0/3650297159 learned_addr learned my addr 172.21.15.31:0/3650297159 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c48ff1700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c440c4f60 0x7f4c441ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c49ff3700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c44151f40 0x7f4c441ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c497f2700 1 -- 172.21.15.31:0/3650297159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c440c4f60 msgr2=0x7f4c441ed3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c497f2700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c440c4f60 0x7f4c441ed3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c497f2700 1 -- 172.21.15.31:0/3650297159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c44151f40 msgr2=0x7f4c441ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c497f2700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c44151f40 0x7f4c441ed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c497f2700 1 -- 172.21.15.31:0/3650297159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009550 con 0x7f4c4406b400 2024-03-30T01:42:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c497f2700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c4406b400 0x7f4c441e7400 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000e930 tx=0x7f4c4000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c39ffb700 1 -- 172.21.15.31:0/3650297159 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40009690 con 0x7f4c4406b400 2024-03-30T01:42:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c39ffb700 1 -- 172.21.15.31:0/3650297159 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4000c850 con 0x7f4c4406b400 2024-03-30T01:42:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c441587b0 con 0x7f4c4406b400 2024-03-30T01:42:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c39ffb700 1 -- 172.21.15.31:0/3650297159 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000cbe0 con 0x7f4c4406b400 2024-03-30T01:42:33.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.448+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c44158a00 con 0x7f4c4406b400 2024-03-30T01:42:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.449+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4404ea80 con 0x7f4c4406b400 2024-03-30T01:42:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.450+0000 7f4c39ffb700 1 -- 172.21.15.31:0/3650297159 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4c4000cd40 con 0x7f4c4406b400 2024-03-30T01:42:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.450+0000 7f4c39ffb700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4c2806ac90 0x7f4c2806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.450+0000 7f4c39ffb700 1 -- 172.21.15.31:0/3650297159 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f4c40089c40 con 0x7f4c4406b400 2024-03-30T01:42:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.450+0000 7f4c48ff1700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4c2806ac90 0x7f4c2806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.451+0000 7f4c48ff1700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4c2806ac90 0x7f4c2806d150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c4408c2c0 tx=0x7f4c34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:33.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.455+0000 7f4c39ffb700 1 -- 172.21.15.31:0/3650297159 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4c40054970 con 0x7f4c4406b400 2024-03-30T01:42:33.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.664+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c440c67f0 con 0x7f4c2806ac90 2024-03-30T01:42:33.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.679+0000 7f4c39ffb700 1 -- 172.21.15.31:0/3650297159 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526896 (secure 0 0 0) 0x7f4c34011b00 con 0x7f4c2806ac90 2024-03-30T01:42:33.690 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:33.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4c2806ac90 msgr2=0x7f4c2806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:33.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4c2806ac90 0x7f4c2806d150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c4408c2c0 tx=0x7f4c34008030 comp rx=0 tx=0).stop 2024-03-30T01:42:33.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c4406b400 msgr2=0x7f4c441e7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:33.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c4406b400 0x7f4c441e7400 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000e930 tx=0x7f4c4000ecf0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 shutdown_connections 2024-03-30T01:42:33.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c4406b400 0x7f4c441e7400 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4c2806ac90 0x7f4c2806d150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c44151f40 0x7f4c441ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 --2- 172.21.15.31:0/3650297159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c440c4f60 0x7f4c441ed3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 >> 172.21.15.31:0/3650297159 conn(0x7f4c440ca0d0 msgr2=0x7f4c44153190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:33.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 shutdown_connections 2024-03-30T01:42:33.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.683+0000 7f4c4ba56700 1 -- 172.21.15.31:0/3650297159 wait complete. 2024-03-30T01:42:33.696 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:42:33.702 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-03-30T01:42:31.829025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596603,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221368,"kb_used_data":5188,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753352,"statfs":{"total":773094113280,"available":772867432448,"internally_reserved":0,"allocated":5312512,"data_stored":2984125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001915"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282109+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:42:24.282109+0000","last_peered":"2024-03-30T01:42:24.282109+0000","last_clean":"2024-03-30T01:42:24.282109+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:42:24.282109+0000","last_undegraded":"2024-03-30T01:42:24.282109+0000","last_fullsized":"2024-03-30T01:42:24.282109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282595+0000","last_change":"2024-03-30T01:42:09.836532+0000","last_active":"2024-03-30T01:42:24.282595+0000","last_peered":"2024-03-30T01:42:24.282595+0000","last_clean":"2024-03-30T01:42:24.282595+0000","last_became_active":"2024-03-30T01:42:09.836262+0000","last_became_peered":"2024-03-30T01:42:09.836262+0000","last_unstale":"2024-03-30T01:42:24.282595+0000","last_undegraded":"2024-03-30T01:42:24.282595+0000","last_fullsized":"2024-03-30T01:42:24.282595+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:08.249431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277527+0000","last_change":"2024-03-30T01:42:10.289721+0000","last_active":"2024-03-30T01:42:24.277527+0000","last_peered":"2024-03-30T01:42:24.277527+0000","last_clean":"2024-03-30T01:42:24.277527+0000","last_became_active":"2024-03-30T01:42:10.289413+0000","last_became_peered":"2024-03-30T01:42:10.289413+0000","last_unstale":"2024-03-30T01:42:24.277527+0000","last_undegraded":"2024-03-30T01:42:24.277527+0000","last_fullsized":"2024-03-30T01:42:24.277527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"43'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606808+0000","last_change":"2024-03-30T01:42:23.832956+0000","last_active":"2024-03-30T01:42:25.606808+0000","last_peered":"2024-03-30T01:42:25.606808+0000","last_clean":"2024-03-30T01:42:25.606808+0000","last_became_active":"2024-03-30T01:42:23.832684+0000","last_became_peered":"2024-03-30T01:42:23.832684+0000","last_unstale":"2024-03-30T01:42:25.606808+0000","last_undegraded":"2024-03-30T01:42:25.606808+0000","last_fullsized":"2024-03-30T01:42:25.606808+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:34.573756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318413+0000","last_change":"2024-03-30T01:42:24.286333+0000","last_active":"2024-03-30T01:42:25.318413+0000","last_peered":"2024-03-30T01:42:25.318413+0000","last_clean":"2024-03-30T01:42:25.318413+0000","last_became_active":"2024-03-30T01:42:24.285564+0000","last_became_peered":"2024-03-30T01:42:24.285564+0000","last_unstale":"2024-03-30T01:42:25.318413+0000","last_undegraded":"2024-03-30T01:42:25.318413+0000","last_fullsized":"2024-03-30T01:42:25.318413+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:22:57.363372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291900+0000","last_change":"2024-03-30T01:42:09.829174+0000","last_active":"2024-03-30T01:42:24.291900+0000","last_peered":"2024-03-30T01:42:24.291900+0000","last_clean":"2024-03-30T01:42:24.291900+0000","last_became_active":"2024-03-30T01:42:09.828730+0000","last_became_peered":"2024-03-30T01:42:09.828730+0000","last_unstale":"2024-03-30T01:42:24.291900+0000","last_undegraded":"2024-03-30T01:42:24.291900+0000","last_fullsized":"2024-03-30T01:42:24.291900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282943+0000","last_change":"2024-03-30T01:42:10.284566+0000","last_active":"2024-03-30T01:42:24.282943+0000","last_peered":"2024-03-30T01:42:24.282943+0000","last_clean":"2024-03-30T01:42:24.282943+0000","last_became_active":"2024-03-30T01:42:10.284192+0000","last_became_peered":"2024-03-30T01:42:10.284192+0000","last_unstale":"2024-03-30T01:42:24.282943+0000","last_undegraded":"2024-03-30T01:42:24.282943+0000","last_fullsized":"2024-03-30T01:42:24.282943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:05.460383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290171+0000","last_change":"2024-03-30T01:42:23.827217+0000","last_active":"2024-03-30T01:42:24.290171+0000","last_peered":"2024-03-30T01:42:24.290171+0000","last_clean":"2024-03-30T01:42:24.290171+0000","last_became_active":"2024-03-30T01:42:23.826922+0000","last_became_peered":"2024-03-30T01:42:23.826922+0000","last_unstale":"2024-03-30T01:42:24.290171+0000","last_undegraded":"2024-03-30T01:42:24.290171+0000","last_fullsized":"2024-03-30T01:42:24.290171+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:56.846961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274839+0000","last_change":"2024-03-30T01:42:24.292886+0000","last_active":"2024-03-30T01:42:25.274839+0000","last_peered":"2024-03-30T01:42:25.274839+0000","last_clean":"2024-03-30T01:42:25.274839+0000","last_became_active":"2024-03-30T01:42:24.292613+0000","last_became_peered":"2024-03-30T01:42:24.292613+0000","last_unstale":"2024-03-30T01:42:25.274839+0000","last_undegraded":"2024-03-30T01:42:25.274839+0000","last_fullsized":"2024-03-30T01:42:25.274839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:22:35.546913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283516+0000","last_change":"2024-03-30T01:42:09.832991+0000","last_active":"2024-03-30T01:42:24.283516+0000","last_peered":"2024-03-30T01:42:24.283516+0000","last_clean":"2024-03-30T01:42:24.283516+0000","last_became_active":"2024-03-30T01:42:09.832682+0000","last_became_peered":"2024-03-30T01:42:09.832682+0000","last_unstale":"2024-03-30T01:42:24.283516+0000","last_undegraded":"2024-03-30T01:42:24.283516+0000","last_fullsized":"2024-03-30T01:42:24.283516+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:37.427434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.288679+0000","last_change":"2024-03-30T01:42:10.299170+0000","last_active":"2024-03-30T01:42:24.288679+0000","last_peered":"2024-03-30T01:42:24.288679+0000","last_clean":"2024-03-30T01:42:24.288679+0000","last_became_active":"2024-03-30T01:42:10.298846+0000","last_became_peered":"2024-03-30T01:42:10.298846+0000","last_unstale":"2024-03-30T01:42:24.288679+0000","last_undegraded":"2024-03-30T01:42:24.288679+0000","last_fullsized":"2024-03-30T01:42:24.288679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291544+0000","last_change":"2024-03-30T01:42:23.836629+0000","last_active":"2024-03-30T01:42:24.291544+0000","last_peered":"2024-03-30T01:42:24.291544+0000","last_clean":"2024-03-30T01:42:24.291544+0000","last_became_active":"2024-03-30T01:42:23.836370+0000","last_became_peered":"2024-03-30T01:42:23.836370+0000","last_unstale":"2024-03-30T01:42:24.291544+0000","last_undegraded":"2024-03-30T01:42:24.291544+0000","last_fullsized":"2024-03-30T01:42:24.291544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:43.792947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606550+0000","last_change":"2024-03-30T01:42:24.295507+0000","last_active":"2024-03-30T01:42:25.606550+0000","last_peered":"2024-03-30T01:42:25.606550+0000","last_clean":"2024-03-30T01:42:25.606550+0000","last_became_active":"2024-03-30T01:42:24.295110+0000","last_became_peered":"2024-03-30T01:42:24.295110+0000","last_unstale":"2024-03-30T01:42:25.606550+0000","last_undegraded":"2024-03-30T01:42:25.606550+0000","last_fullsized":"2024-03-30T01:42:25.606550+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:12:26.305628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273317+0000","last_change":"2024-03-30T01:42:09.822074+0000","last_active":"2024-03-30T01:42:24.273317+0000","last_peered":"2024-03-30T01:42:24.273317+0000","last_clean":"2024-03-30T01:42:24.273317+0000","last_became_active":"2024-03-30T01:42:09.821786+0000","last_became_peered":"2024-03-30T01:42:09.821786+0000","last_unstale":"2024-03-30T01:42:24.273317+0000","last_undegraded":"2024-03-30T01:42:24.273317+0000","last_fullsized":"2024-03-30T01:42:24.273317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:18.840885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.286388+0000","last_change":"2024-03-30T01:42:10.291029+0000","last_active":"2024-03-30T01:42:24.286388+0000","last_peered":"2024-03-30T01:42:24.286388+0000","last_clean":"2024-03-30T01:42:24.286388+0000","last_became_active":"2024-03-30T01:42:10.290728+0000","last_became_peered":"2024-03-30T01:42:10.290728+0000","last_unstale":"2024-03-30T01:42:24.286388+0000","last_undegraded":"2024-03-30T01:42:24.286388+0000","last_fullsized":"2024-03-30T01:42:24.286388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277359+0000","last_change":"2024-03-30T01:42:23.834875+0000","last_active":"2024-03-30T01:42:24.277359+0000","last_peered":"2024-03-30T01:42:24.277359+0000","last_clean":"2024-03-30T01:42:24.277359+0000","last_became_active":"2024-03-30T01:42:23.834636+0000","last_became_peered":"2024-03-30T01:42:23.834636+0000","last_unstale":"2024-03-30T01:42:24.277359+0000","last_undegraded":"2024-03-30T01:42:24.277359+0000","last_fullsized":"2024-03-30T01:42:24.277359+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812557+0000","last_change":"2024-03-30T01:42:24.293284+0000","last_active":"2024-03-30T01:42:25.812557+0000","last_peered":"2024-03-30T01:42:25.812557+0000","last_clean":"2024-03-30T01:42:25.812557+0000","last_became_active":"2024-03-30T01:42:24.293000+0000","last_became_peered":"2024-03-30T01:42:24.293000+0000","last_unstale":"2024-03-30T01:42:25.812557+0000","last_undegraded":"2024-03-30T01:42:25.812557+0000","last_fullsized":"2024-03-30T01:42:25.812557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:13:41.465524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.279424+0000","last_change":"2024-03-30T01:42:09.827116+0000","last_active":"2024-03-30T01:42:24.279424+0000","last_peered":"2024-03-30T01:42:24.279424+0000","last_clean":"2024-03-30T01:42:24.279424+0000","last_became_active":"2024-03-30T01:42:09.826876+0000","last_became_peered":"2024-03-30T01:42:09.826876+0000","last_unstale":"2024-03-30T01:42:24.279424+0000","last_undegraded":"2024-03-30T01:42:24.279424+0000","last_fullsized":"2024-03-30T01:42:24.279424+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:38.792762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278630+0000","last_change":"2024-03-30T01:42:10.288334+0000","last_active":"2024-03-30T01:42:24.278630+0000","last_peered":"2024-03-30T01:42:24.278630+0000","last_clean":"2024-03-30T01:42:24.278630+0000","last_became_active":"2024-03-30T01:42:10.288093+0000","last_became_peered":"2024-03-30T01:42:10.288093+0000","last_unstale":"2024-03-30T01:42:24.278630+0000","last_undegraded":"2024-03-30T01:42:24.278630+0000","last_fullsized":"2024-03-30T01:42:24.278630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283343+0000","last_change":"2024-03-30T01:42:23.826453+0000","last_active":"2024-03-30T01:42:24.283343+0000","last_peered":"2024-03-30T01:42:24.283343+0000","last_clean":"2024-03-30T01:42:24.283343+0000","last_became_active":"2024-03-30T01:42:23.826129+0000","last_became_peered":"2024-03-30T01:42:23.826129+0000","last_unstale":"2024-03-30T01:42:24.283343+0000","last_undegraded":"2024-03-30T01:42:24.283343+0000","last_fullsized":"2024-03-30T01:42:24.283343+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:44:11.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606935+0000","last_change":"2024-03-30T01:42:24.288310+0000","last_active":"2024-03-30T01:42:25.606935+0000","last_peered":"2024-03-30T01:42:25.606935+0000","last_clean":"2024-03-30T01:42:25.606935+0000","last_became_active":"2024-03-30T01:42:24.288026+0000","last_became_peered":"2024-03-30T01:42:24.288026+0000","last_unstale":"2024-03-30T01:42:25.606935+0000","last_undegraded":"2024-03-30T01:42:25.606935+0000","last_fullsized":"2024-03-30T01:42:25.606935+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:39:42.096508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280297+0000","last_change":"2024-03-30T01:42:09.830490+0000","last_active":"2024-03-30T01:42:24.280297+0000","last_peered":"2024-03-30T01:42:24.280297+0000","last_clean":"2024-03-30T01:42:24.280297+0000","last_became_active":"2024-03-30T01:42:09.830197+0000","last_became_peered":"2024-03-30T01:42:09.830197+0000","last_unstale":"2024-03-30T01:42:24.280297+0000","last_undegraded":"2024-03-30T01:42:24.280297+0000","last_fullsized":"2024-03-30T01:42:24.280297+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:46.433516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290564+0000","last_change":"2024-03-30T01:42:10.299986+0000","last_active":"2024-03-30T01:42:24.290564+0000","last_peered":"2024-03-30T01:42:24.290564+0000","last_clean":"2024-03-30T01:42:24.290564+0000","last_became_active":"2024-03-30T01:42:10.299776+0000","last_became_peered":"2024-03-30T01:42:10.299776+0000","last_unstale":"2024-03-30T01:42:24.290564+0000","last_undegraded":"2024-03-30T01:42:24.290564+0000","last_fullsized":"2024-03-30T01:42:24.290564+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.308388+0000","last_change":"2024-03-30T01:42:23.836727+0000","last_active":"2024-03-30T01:42:24.308388+0000","last_peered":"2024-03-30T01:42:24.308388+0000","last_clean":"2024-03-30T01:42:24.308388+0000","last_became_active":"2024-03-30T01:42:23.836310+0000","last_became_peered":"2024-03-30T01:42:23.836310+0000","last_unstale":"2024-03-30T01:42:24.308388+0000","last_undegraded":"2024-03-30T01:42:24.308388+0000","last_fullsized":"2024-03-30T01:42:24.308388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:58.845872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274816+0000","last_change":"2024-03-30T01:42:24.299335+0000","last_active":"2024-03-30T01:42:25.274816+0000","last_peered":"2024-03-30T01:42:25.274816+0000","last_clean":"2024-03-30T01:42:25.274816+0000","last_became_active":"2024-03-30T01:42:24.299066+0000","last_became_peered":"2024-03-30T01:42:24.299066+0000","last_unstale":"2024-03-30T01:42:25.274816+0000","last_undegraded":"2024-03-30T01:42:25.274816+0000","last_fullsized":"2024-03-30T01:42:25.274816+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:58.973438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275374+0000","last_change":"2024-03-30T01:42:09.824373+0000","last_active":"2024-03-30T01:42:24.275374+0000","last_peered":"2024-03-30T01:42:24.275374+0000","last_clean":"2024-03-30T01:42:24.275374+0000","last_became_active":"2024-03-30T01:42:09.824089+0000","last_became_peered":"2024-03-30T01:42:09.824089+0000","last_unstale":"2024-03-30T01:42:24.275374+0000","last_undegraded":"2024-03-30T01:42:24.275374+0000","last_fullsized":"2024-03-30T01:42:24.275374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:37.434399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.284258+0000","last_change":"2024-03-30T01:42:10.284232+0000","last_active":"2024-03-30T01:42:24.284258+0000","last_peered":"2024-03-30T01:42:24.284258+0000","last_clean":"2024-03-30T01:42:24.284258+0000","last_became_active":"2024-03-30T01:42:10.283909+0000","last_became_peered":"2024-03-30T01:42:10.283909+0000","last_unstale":"2024-03-30T01:42:24.284258+0000","last_undegraded":"2024-03-30T01:42:24.284258+0000","last_fullsized":"2024-03-30T01:42:24.284258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:33.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278837+0000","last_change":"2024-03-30T01:42:23.830264+0000","last_active":"2024-03-30T01:42:24.278837+0000","last_peered":"2024-03-30T01:42:24.278837+0000","last_clean":"2024-03-30T01:42:24.278837+0000","last_became_active":"2024-03-30T01:42:23.829948+0000","last_became_peered":"2024-03-30T01:42:23.829948+0000","last_unstale":"2024-03-30T01:42:24.278837+0000","last_undegraded":"2024-03-30T01:42:24.278837+0000","last_fullsized":"2024-03-30T01:42:24.278837+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:42:51.968187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812210+0000","last_change":"2024-03-30T01:42:24.298952+0000","last_active":"2024-03-30T01:42:25.812210+0000","last_peered":"2024-03-30T01:42:25.812210+0000","last_clean":"2024-03-30T01:42:25.812210+0000","last_became_active":"2024-03-30T01:42:24.298680+0000","last_became_peered":"2024-03-30T01:42:24.298680+0000","last_unstale":"2024-03-30T01:42:25.812210+0000","last_undegraded":"2024-03-30T01:42:25.812210+0000","last_fullsized":"2024-03-30T01:42:25.812210+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:27:11.779346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277686+0000","last_change":"2024-03-30T01:42:09.833189+0000","last_active":"2024-03-30T01:42:24.277686+0000","last_peered":"2024-03-30T01:42:24.277686+0000","last_clean":"2024-03-30T01:42:24.277686+0000","last_became_active":"2024-03-30T01:42:09.832891+0000","last_became_peered":"2024-03-30T01:42:09.832891+0000","last_unstale":"2024-03-30T01:42:24.277686+0000","last_undegraded":"2024-03-30T01:42:24.277686+0000","last_fullsized":"2024-03-30T01:42:24.277686+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283440+0000","last_change":"2024-03-30T01:42:10.298216+0000","last_active":"2024-03-30T01:42:24.283440+0000","last_peered":"2024-03-30T01:42:24.283440+0000","last_clean":"2024-03-30T01:42:24.283440+0000","last_became_active":"2024-03-30T01:42:10.297976+0000","last_became_peered":"2024-03-30T01:42:10.297976+0000","last_unstale":"2024-03-30T01:42:24.283440+0000","last_undegraded":"2024-03-30T01:42:24.283440+0000","last_fullsized":"2024-03-30T01:42:24.283440+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.284594+0000","last_change":"2024-03-30T01:42:23.827398+0000","last_active":"2024-03-30T01:42:24.284594+0000","last_peered":"2024-03-30T01:42:24.284594+0000","last_clean":"2024-03-30T01:42:24.284594+0000","last_became_active":"2024-03-30T01:42:23.827120+0000","last_became_peered":"2024-03-30T01:42:23.827120+0000","last_unstale":"2024-03-30T01:42:24.284594+0000","last_undegraded":"2024-03-30T01:42:24.284594+0000","last_fullsized":"2024-03-30T01:42:24.284594+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.273929+0000","last_change":"2024-03-30T01:42:24.293160+0000","last_active":"2024-03-30T01:42:25.273929+0000","last_peered":"2024-03-30T01:42:25.273929+0000","last_clean":"2024-03-30T01:42:25.273929+0000","last_became_active":"2024-03-30T01:42:24.292443+0000","last_became_peered":"2024-03-30T01:42:24.292443+0000","last_unstale":"2024-03-30T01:42:25.273929+0000","last_undegraded":"2024-03-30T01:42:25.273929+0000","last_fullsized":"2024-03-30T01:42:25.273929+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:24.433162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273654+0000","last_change":"2024-03-30T01:42:09.829638+0000","last_active":"2024-03-30T01:42:24.273654+0000","last_peered":"2024-03-30T01:42:24.273654+0000","last_clean":"2024-03-30T01:42:24.273654+0000","last_became_active":"2024-03-30T01:42:09.829317+0000","last_became_peered":"2024-03-30T01:42:09.829317+0000","last_unstale":"2024-03-30T01:42:24.273654+0000","last_undegraded":"2024-03-30T01:42:24.273654+0000","last_fullsized":"2024-03-30T01:42:24.273654+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:33.639097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290194+0000","last_change":"2024-03-30T01:42:10.292043+0000","last_active":"2024-03-30T01:42:24.290194+0000","last_peered":"2024-03-30T01:42:24.290194+0000","last_clean":"2024-03-30T01:42:24.290194+0000","last_became_active":"2024-03-30T01:42:10.291598+0000","last_became_peered":"2024-03-30T01:42:10.291598+0000","last_unstale":"2024-03-30T01:42:24.290194+0000","last_undegraded":"2024-03-30T01:42:24.290194+0000","last_fullsized":"2024-03-30T01:42:24.290194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290056+0000","last_change":"2024-03-30T01:42:23.837223+0000","last_active":"2024-03-30T01:42:24.290056+0000","last_peered":"2024-03-30T01:42:24.290056+0000","last_clean":"2024-03-30T01:42:24.290056+0000","last_became_active":"2024-03-30T01:42:23.836973+0000","last_became_peered":"2024-03-30T01:42:23.836973+0000","last_unstale":"2024-03-30T01:42:24.290056+0000","last_undegraded":"2024-03-30T01:42:24.290056+0000","last_fullsized":"2024-03-30T01:42:24.290056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:57.966402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318600+0000","last_change":"2024-03-30T01:42:24.283055+0000","last_active":"2024-03-30T01:42:25.318600+0000","last_peered":"2024-03-30T01:42:25.318600+0000","last_clean":"2024-03-30T01:42:25.318600+0000","last_became_active":"2024-03-30T01:42:24.282764+0000","last_became_peered":"2024-03-30T01:42:24.282764+0000","last_unstale":"2024-03-30T01:42:25.318600+0000","last_undegraded":"2024-03-30T01:42:25.318600+0000","last_fullsized":"2024-03-30T01:42:25.318600+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:00.208464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282644+0000","last_change":"2024-03-30T01:42:09.829777+0000","last_active":"2024-03-30T01:42:24.282644+0000","last_peered":"2024-03-30T01:42:24.282644+0000","last_clean":"2024-03-30T01:42:24.282644+0000","last_became_active":"2024-03-30T01:42:09.829483+0000","last_became_peered":"2024-03-30T01:42:09.829483+0000","last_unstale":"2024-03-30T01:42:24.282644+0000","last_undegraded":"2024-03-30T01:42:24.282644+0000","last_fullsized":"2024-03-30T01:42:24.282644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:45.587424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.276850+0000","last_change":"2024-03-30T01:42:10.285649+0000","last_active":"2024-03-30T01:42:24.276850+0000","last_peered":"2024-03-30T01:42:24.276850+0000","last_clean":"2024-03-30T01:42:24.276850+0000","last_became_active":"2024-03-30T01:42:10.285297+0000","last_became_peered":"2024-03-30T01:42:10.285297+0000","last_unstale":"2024-03-30T01:42:24.276850+0000","last_undegraded":"2024-03-30T01:42:24.276850+0000","last_fullsized":"2024-03-30T01:42:24.276850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297151+0000","last_change":"2024-03-30T01:42:23.831786+0000","last_active":"2024-03-30T01:42:24.297151+0000","last_peered":"2024-03-30T01:42:24.297151+0000","last_clean":"2024-03-30T01:42:24.297151+0000","last_became_active":"2024-03-30T01:42:23.829363+0000","last_became_peered":"2024-03-30T01:42:23.829363+0000","last_unstale":"2024-03-30T01:42:24.297151+0000","last_undegraded":"2024-03-30T01:42:24.297151+0000","last_fullsized":"2024-03-30T01:42:24.297151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:19:37.521361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275209+0000","last_change":"2024-03-30T01:42:24.298420+0000","last_active":"2024-03-30T01:42:25.275209+0000","last_peered":"2024-03-30T01:42:25.275209+0000","last_clean":"2024-03-30T01:42:25.275209+0000","last_became_active":"2024-03-30T01:42:24.297784+0000","last_became_peered":"2024-03-30T01:42:24.297784+0000","last_unstale":"2024-03-30T01:42:25.275209+0000","last_undegraded":"2024-03-30T01:42:25.275209+0000","last_fullsized":"2024-03-30T01:42:25.275209+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:45.153231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277816+0000","last_change":"2024-03-30T01:42:09.826256+0000","last_active":"2024-03-30T01:42:24.277816+0000","last_peered":"2024-03-30T01:42:24.277816+0000","last_clean":"2024-03-30T01:42:24.277816+0000","last_became_active":"2024-03-30T01:42:09.825956+0000","last_became_peered":"2024-03-30T01:42:09.825956+0000","last_unstale":"2024-03-30T01:42:24.277816+0000","last_undegraded":"2024-03-30T01:42:24.277816+0000","last_fullsized":"2024-03-30T01:42:24.277816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:59:41.446927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278754+0000","last_change":"2024-03-30T01:42:10.290469+0000","last_active":"2024-03-30T01:42:24.278754+0000","last_peered":"2024-03-30T01:42:24.278754+0000","last_clean":"2024-03-30T01:42:24.278754+0000","last_became_active":"2024-03-30T01:42:10.290148+0000","last_became_peered":"2024-03-30T01:42:10.290148+0000","last_unstale":"2024-03-30T01:42:24.278754+0000","last_undegraded":"2024-03-30T01:42:24.278754+0000","last_fullsized":"2024-03-30T01:42:24.278754+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275001+0000","last_change":"2024-03-30T01:42:23.826954+0000","last_active":"2024-03-30T01:42:24.275001+0000","last_peered":"2024-03-30T01:42:24.275001+0000","last_clean":"2024-03-30T01:42:24.275001+0000","last_became_active":"2024-03-30T01:42:23.826586+0000","last_became_peered":"2024-03-30T01:42:23.826586+0000","last_unstale":"2024-03-30T01:42:24.275001+0000","last_undegraded":"2024-03-30T01:42:24.275001+0000","last_fullsized":"2024-03-30T01:42:24.275001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:54.923310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275067+0000","last_change":"2024-03-30T01:42:24.299122+0000","last_active":"2024-03-30T01:42:25.275067+0000","last_peered":"2024-03-30T01:42:25.275067+0000","last_clean":"2024-03-30T01:42:25.275067+0000","last_became_active":"2024-03-30T01:42:24.298339+0000","last_became_peered":"2024-03-30T01:42:24.298339+0000","last_unstale":"2024-03-30T01:42:25.275067+0000","last_undegraded":"2024-03-30T01:42:25.275067+0000","last_fullsized":"2024-03-30T01:42:25.275067+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:21.224256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275815+0000","last_change":"2024-03-30T01:42:09.830452+0000","last_active":"2024-03-30T01:42:24.275815+0000","last_peered":"2024-03-30T01:42:24.275815+0000","last_clean":"2024-03-30T01:42:24.275815+0000","last_became_active":"2024-03-30T01:42:09.830173+0000","last_became_peered":"2024-03-30T01:42:09.830173+0000","last_unstale":"2024-03-30T01:42:24.275815+0000","last_undegraded":"2024-03-30T01:42:24.275815+0000","last_fullsized":"2024-03-30T01:42:24.275815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:12:42.807604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282320+0000","last_change":"2024-03-30T01:42:10.279913+0000","last_active":"2024-03-30T01:42:24.282320+0000","last_peered":"2024-03-30T01:42:24.282320+0000","last_clean":"2024-03-30T01:42:24.282320+0000","last_became_active":"2024-03-30T01:42:10.279450+0000","last_became_peered":"2024-03-30T01:42:10.279450+0000","last_unstale":"2024-03-30T01:42:24.282320+0000","last_undegraded":"2024-03-30T01:42:24.282320+0000","last_fullsized":"2024-03-30T01:42:24.282320+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278669+0000","last_change":"2024-03-30T01:42:23.835387+0000","last_active":"2024-03-30T01:42:24.278669+0000","last_peered":"2024-03-30T01:42:24.278669+0000","last_clean":"2024-03-30T01:42:24.278669+0000","last_became_active":"2024-03-30T01:42:23.833427+0000","last_became_peered":"2024-03-30T01:42:23.833427+0000","last_unstale":"2024-03-30T01:42:24.278669+0000","last_undegraded":"2024-03-30T01:42:24.278669+0000","last_fullsized":"2024-03-30T01:42:24.278669+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:44.454854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606752+0000","last_change":"2024-03-30T01:42:24.300970+0000","last_active":"2024-03-30T01:42:25.606752+0000","last_peered":"2024-03-30T01:42:25.606752+0000","last_clean":"2024-03-30T01:42:25.606752+0000","last_became_active":"2024-03-30T01:42:24.300724+0000","last_became_peered":"2024-03-30T01:42:24.300724+0000","last_unstale":"2024-03-30T01:42:25.606752+0000","last_undegraded":"2024-03-30T01:42:25.606752+0000","last_fullsized":"2024-03-30T01:42:25.606752+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:24:42.987038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.287160+0000","last_change":"2024-03-30T01:42:09.828572+0000","last_active":"2024-03-30T01:42:24.287160+0000","last_peered":"2024-03-30T01:42:24.287160+0000","last_clean":"2024-03-30T01:42:24.287160+0000","last_became_active":"2024-03-30T01:42:09.828225+0000","last_became_peered":"2024-03-30T01:42:09.828225+0000","last_unstale":"2024-03-30T01:42:24.287160+0000","last_undegraded":"2024-03-30T01:42:24.287160+0000","last_fullsized":"2024-03-30T01:42:24.287160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275458+0000","last_change":"2024-03-30T01:42:10.286900+0000","last_active":"2024-03-30T01:42:24.275458+0000","last_peered":"2024-03-30T01:42:24.275458+0000","last_clean":"2024-03-30T01:42:24.275458+0000","last_became_active":"2024-03-30T01:42:10.286598+0000","last_became_peered":"2024-03-30T01:42:10.286598+0000","last_unstale":"2024-03-30T01:42:24.275458+0000","last_undegraded":"2024-03-30T01:42:24.275458+0000","last_fullsized":"2024-03-30T01:42:24.275458+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:37:46.592007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291128+0000","last_change":"2024-03-30T01:42:23.829204+0000","last_active":"2024-03-30T01:42:24.291128+0000","last_peered":"2024-03-30T01:42:24.291128+0000","last_clean":"2024-03-30T01:42:24.291128+0000","last_became_active":"2024-03-30T01:42:23.828938+0000","last_became_peered":"2024-03-30T01:42:23.828938+0000","last_unstale":"2024-03-30T01:42:24.291128+0000","last_undegraded":"2024-03-30T01:42:24.291128+0000","last_fullsized":"2024-03-30T01:42:24.291128+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:57.756956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.273668+0000","last_change":"2024-03-30T01:42:24.301476+0000","last_active":"2024-03-30T01:42:25.273668+0000","last_peered":"2024-03-30T01:42:25.273668+0000","last_clean":"2024-03-30T01:42:25.273668+0000","last_became_active":"2024-03-30T01:42:24.301208+0000","last_became_peered":"2024-03-30T01:42:24.301208+0000","last_unstale":"2024-03-30T01:42:25.273668+0000","last_undegraded":"2024-03-30T01:42:25.273668+0000","last_fullsized":"2024-03-30T01:42:25.273668+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:57.853997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278103+0000","last_change":"2024-03-30T01:42:09.823221+0000","last_active":"2024-03-30T01:42:24.278103+0000","last_peered":"2024-03-30T01:42:24.278103+0000","last_clean":"2024-03-30T01:42:24.278103+0000","last_became_active":"2024-03-30T01:42:09.822882+0000","last_became_peered":"2024-03-30T01:42:09.822882+0000","last_unstale":"2024-03-30T01:42:24.278103+0000","last_undegraded":"2024-03-30T01:42:24.278103+0000","last_fullsized":"2024-03-30T01:42:24.278103+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:00:31.185685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283611+0000","last_change":"2024-03-30T01:42:10.304085+0000","last_active":"2024-03-30T01:42:24.283611+0000","last_peered":"2024-03-30T01:42:24.283611+0000","last_clean":"2024-03-30T01:42:24.283611+0000","last_became_active":"2024-03-30T01:42:10.303817+0000","last_became_peered":"2024-03-30T01:42:10.303817+0000","last_unstale":"2024-03-30T01:42:24.283611+0000","last_undegraded":"2024-03-30T01:42:24.283611+0000","last_fullsized":"2024-03-30T01:42:24.283611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.303894+0000","last_change":"2024-03-30T01:42:23.837118+0000","last_active":"2024-03-30T01:42:24.303894+0000","last_peered":"2024-03-30T01:42:24.303894+0000","last_clean":"2024-03-30T01:42:24.303894+0000","last_became_active":"2024-03-30T01:42:23.836859+0000","last_became_peered":"2024-03-30T01:42:23.836859+0000","last_unstale":"2024-03-30T01:42:24.303894+0000","last_undegraded":"2024-03-30T01:42:24.303894+0000","last_fullsized":"2024-03-30T01:42:24.303894+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:06.958915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811582+0000","last_change":"2024-03-30T01:42:24.299737+0000","last_active":"2024-03-30T01:42:25.811582+0000","last_peered":"2024-03-30T01:42:25.811582+0000","last_clean":"2024-03-30T01:42:25.811582+0000","last_became_active":"2024-03-30T01:42:24.299259+0000","last_became_peered":"2024-03-30T01:42:24.299259+0000","last_unstale":"2024-03-30T01:42:25.811582+0000","last_undegraded":"2024-03-30T01:42:25.811582+0000","last_fullsized":"2024-03-30T01:42:25.811582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:51:14.820447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283642+0000","last_change":"2024-03-30T01:42:09.829713+0000","last_active":"2024-03-30T01:42:24.283642+0000","last_peered":"2024-03-30T01:42:24.283642+0000","last_clean":"2024-03-30T01:42:24.283642+0000","last_became_active":"2024-03-30T01:42:09.829375+0000","last_became_peered":"2024-03-30T01:42:09.829375+0000","last_unstale":"2024-03-30T01:42:24.283642+0000","last_undegraded":"2024-03-30T01:42:24.283642+0000","last_fullsized":"2024-03-30T01:42:24.283642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:49:51.601663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277836+0000","last_change":"2024-03-30T01:42:10.295516+0000","last_active":"2024-03-30T01:42:24.277836+0000","last_peered":"2024-03-30T01:42:24.277836+0000","last_clean":"2024-03-30T01:42:24.277836+0000","last_became_active":"2024-03-30T01:42:10.295133+0000","last_became_peered":"2024-03-30T01:42:10.295133+0000","last_unstale":"2024-03-30T01:42:24.277836+0000","last_undegraded":"2024-03-30T01:42:24.277836+0000","last_fullsized":"2024-03-30T01:42:24.277836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273944+0000","last_change":"2024-03-30T01:42:23.826615+0000","last_active":"2024-03-30T01:42:24.273944+0000","last_peered":"2024-03-30T01:42:24.273944+0000","last_clean":"2024-03-30T01:42:24.273944+0000","last_became_active":"2024-03-30T01:42:23.826332+0000","last_became_peered":"2024-03-30T01:42:23.826332+0000","last_unstale":"2024-03-30T01:42:24.273944+0000","last_undegraded":"2024-03-30T01:42:24.273944+0000","last_fullsized":"2024-03-30T01:42:24.273944+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:46.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274114+0000","last_change":"2024-03-30T01:42:24.297682+0000","last_active":"2024-03-30T01:42:25.274114+0000","last_peered":"2024-03-30T01:42:25.274114+0000","last_clean":"2024-03-30T01:42:25.274114+0000","last_became_active":"2024-03-30T01:42:24.297410+0000","last_became_peered":"2024-03-30T01:42:24.297410+0000","last_unstale":"2024-03-30T01:42:25.274114+0000","last_undegraded":"2024-03-30T01:42:25.274114+0000","last_fullsized":"2024-03-30T01:42:25.274114+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:58.973438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277619+0000","last_change":"2024-03-30T01:42:09.826895+0000","last_active":"2024-03-30T01:42:24.277619+0000","last_peered":"2024-03-30T01:42:24.277619+0000","last_clean":"2024-03-30T01:42:24.277619+0000","last_became_active":"2024-03-30T01:42:09.826609+0000","last_became_peered":"2024-03-30T01:42:09.826609+0000","last_unstale":"2024-03-30T01:42:24.277619+0000","last_undegraded":"2024-03-30T01:42:24.277619+0000","last_fullsized":"2024-03-30T01:42:24.277619+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:42:59.036073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291816+0000","last_change":"2024-03-30T01:42:10.291564+0000","last_active":"2024-03-30T01:42:24.291816+0000","last_peered":"2024-03-30T01:42:24.291816+0000","last_clean":"2024-03-30T01:42:24.291816+0000","last_became_active":"2024-03-30T01:42:10.291285+0000","last_became_peered":"2024-03-30T01:42:10.291285+0000","last_unstale":"2024-03-30T01:42:24.291816+0000","last_undegraded":"2024-03-30T01:42:24.291816+0000","last_fullsized":"2024-03-30T01:42:24.291816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.301787+0000","last_change":"2024-03-30T01:42:23.829970+0000","last_active":"2024-03-30T01:42:24.301787+0000","last_peered":"2024-03-30T01:42:24.301787+0000","last_clean":"2024-03-30T01:42:24.301787+0000","last_became_active":"2024-03-30T01:42:23.829639+0000","last_became_peered":"2024-03-30T01:42:23.829639+0000","last_unstale":"2024-03-30T01:42:24.301787+0000","last_undegraded":"2024-03-30T01:42:24.301787+0000","last_fullsized":"2024-03-30T01:42:24.301787+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:49.116227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813166+0000","last_change":"2024-03-30T01:42:24.302754+0000","last_active":"2024-03-30T01:42:25.813166+0000","last_peered":"2024-03-30T01:42:25.813166+0000","last_clean":"2024-03-30T01:42:25.813166+0000","last_became_active":"2024-03-30T01:42:24.302449+0000","last_became_peered":"2024-03-30T01:42:24.302449+0000","last_unstale":"2024-03-30T01:42:25.813166+0000","last_undegraded":"2024-03-30T01:42:25.813166+0000","last_fullsized":"2024-03-30T01:42:25.813166+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:57.853997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":23,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282222+0000","last_change":"2024-03-30T01:42:09.831869+0000","last_active":"2024-03-30T01:42:24.282222+0000","last_peered":"2024-03-30T01:42:24.282222+0000","last_clean":"2024-03-30T01:42:24.282222+0000","last_became_active":"2024-03-30T01:42:09.831562+0000","last_became_peered":"2024-03-30T01:42:09.831562+0000","last_unstale":"2024-03-30T01:42:24.282222+0000","last_undegraded":"2024-03-30T01:42:24.282222+0000","last_fullsized":"2024-03-30T01:42:24.282222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:35.938001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290305+0000","last_change":"2024-03-30T01:42:10.288399+0000","last_active":"2024-03-30T01:42:24.290305+0000","last_peered":"2024-03-30T01:42:24.290305+0000","last_clean":"2024-03-30T01:42:24.290305+0000","last_became_active":"2024-03-30T01:42:10.287890+0000","last_became_peered":"2024-03-30T01:42:10.287890+0000","last_unstale":"2024-03-30T01:42:24.290305+0000","last_undegraded":"2024-03-30T01:42:24.290305+0000","last_fullsized":"2024-03-30T01:42:24.290305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:35:11.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273363+0000","last_change":"2024-03-30T01:42:23.832729+0000","last_active":"2024-03-30T01:42:24.273363+0000","last_peered":"2024-03-30T01:42:24.273363+0000","last_clean":"2024-03-30T01:42:24.273363+0000","last_became_active":"2024-03-30T01:42:23.832474+0000","last_became_peered":"2024-03-30T01:42:23.832474+0000","last_unstale":"2024-03-30T01:42:24.273363+0000","last_undegraded":"2024-03-30T01:42:24.273363+0000","last_fullsized":"2024-03-30T01:42:24.273363+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:59.368657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606839+0000","last_change":"2024-03-30T01:42:24.296757+0000","last_active":"2024-03-30T01:42:25.606839+0000","last_peered":"2024-03-30T01:42:25.606839+0000","last_clean":"2024-03-30T01:42:25.606839+0000","last_became_active":"2024-03-30T01:42:24.296481+0000","last_became_peered":"2024-03-30T01:42:24.296481+0000","last_unstale":"2024-03-30T01:42:25.606839+0000","last_undegraded":"2024-03-30T01:42:25.606839+0000","last_fullsized":"2024-03-30T01:42:25.606839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:50:12.648133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273090+0000","last_change":"2024-03-30T01:42:09.820451+0000","last_active":"2024-03-30T01:42:24.273090+0000","last_peered":"2024-03-30T01:42:24.273090+0000","last_clean":"2024-03-30T01:42:24.273090+0000","last_became_active":"2024-03-30T01:42:09.820152+0000","last_became_peered":"2024-03-30T01:42:09.820152+0000","last_unstale":"2024-03-30T01:42:24.273090+0000","last_undegraded":"2024-03-30T01:42:24.273090+0000","last_fullsized":"2024-03-30T01:42:24.273090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:04.153303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282489+0000","last_change":"2024-03-30T01:42:10.279681+0000","last_active":"2024-03-30T01:42:24.282489+0000","last_peered":"2024-03-30T01:42:24.282489+0000","last_clean":"2024-03-30T01:42:24.282489+0000","last_became_active":"2024-03-30T01:42:10.279333+0000","last_became_peered":"2024-03-30T01:42:10.279333+0000","last_unstale":"2024-03-30T01:42:24.282489+0000","last_undegraded":"2024-03-30T01:42:24.282489+0000","last_fullsized":"2024-03-30T01:42:24.282489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.300487+0000","last_change":"2024-03-30T01:42:23.833668+0000","last_active":"2024-03-30T01:42:24.300487+0000","last_peered":"2024-03-30T01:42:24.300487+0000","last_clean":"2024-03-30T01:42:24.300487+0000","last_became_active":"2024-03-30T01:42:23.832884+0000","last_became_peered":"2024-03-30T01:42:23.832884+0000","last_unstale":"2024-03-30T01:42:24.300487+0000","last_undegraded":"2024-03-30T01:42:24.300487+0000","last_fullsized":"2024-03-30T01:42:24.300487+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.379389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275298+0000","last_change":"2024-03-30T01:42:24.292573+0000","last_active":"2024-03-30T01:42:25.275298+0000","last_peered":"2024-03-30T01:42:25.275298+0000","last_clean":"2024-03-30T01:42:25.275298+0000","last_became_active":"2024-03-30T01:42:24.292282+0000","last_became_peered":"2024-03-30T01:42:24.292282+0000","last_unstale":"2024-03-30T01:42:25.275298+0000","last_undegraded":"2024-03-30T01:42:25.275298+0000","last_fullsized":"2024-03-30T01:42:25.275298+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:35.580791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.285427+0000","last_change":"2024-03-30T01:42:09.829194+0000","last_active":"2024-03-30T01:42:24.285427+0000","last_peered":"2024-03-30T01:42:24.285427+0000","last_clean":"2024-03-30T01:42:24.285427+0000","last_became_active":"2024-03-30T01:42:09.828257+0000","last_became_peered":"2024-03-30T01:42:09.828257+0000","last_unstale":"2024-03-30T01:42:24.285427+0000","last_undegraded":"2024-03-30T01:42:24.285427+0000","last_fullsized":"2024-03-30T01:42:24.285427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.289884+0000","last_change":"2024-03-30T01:42:10.288326+0000","last_active":"2024-03-30T01:42:24.289884+0000","last_peered":"2024-03-30T01:42:24.289884+0000","last_clean":"2024-03-30T01:42:24.289884+0000","last_became_active":"2024-03-30T01:42:10.288050+0000","last_became_peered":"2024-03-30T01:42:10.288050+0000","last_unstale":"2024-03-30T01:42:24.289884+0000","last_undegraded":"2024-03-30T01:42:24.289884+0000","last_fullsized":"2024-03-30T01:42:24.289884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"42'8","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.309918+0000","last_change":"2024-03-30T01:42:23.836559+0000","last_active":"2024-03-30T01:42:24.309918+0000","last_peered":"2024-03-30T01:42:24.309918+0000","last_clean":"2024-03-30T01:42:24.309918+0000","last_became_active":"2024-03-30T01:42:23.836272+0000","last_became_peered":"2024-03-30T01:42:23.836272+0000","last_unstale":"2024-03-30T01:42:24.309918+0000","last_undegraded":"2024-03-30T01:42:24.309918+0000","last_fullsized":"2024-03-30T01:42:24.309918+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:20.217220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.607152+0000","last_change":"2024-03-30T01:42:24.296963+0000","last_active":"2024-03-30T01:42:25.607152+0000","last_peered":"2024-03-30T01:42:25.607152+0000","last_clean":"2024-03-30T01:42:25.607152+0000","last_became_active":"2024-03-30T01:42:24.288206+0000","last_became_peered":"2024-03-30T01:42:24.288206+0000","last_unstale":"2024-03-30T01:42:25.607152+0000","last_undegraded":"2024-03-30T01:42:25.607152+0000","last_fullsized":"2024-03-30T01:42:25.607152+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:14:43.146891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.289989+0000","last_change":"2024-03-30T01:42:09.832318+0000","last_active":"2024-03-30T01:42:24.289989+0000","last_peered":"2024-03-30T01:42:24.289989+0000","last_clean":"2024-03-30T01:42:24.289989+0000","last_became_active":"2024-03-30T01:42:09.831964+0000","last_became_peered":"2024-03-30T01:42:09.831964+0000","last_unstale":"2024-03-30T01:42:24.289989+0000","last_undegraded":"2024-03-30T01:42:24.289989+0000","last_fullsized":"2024-03-30T01:42:24.289989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:05.700970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278549+0000","last_change":"2024-03-30T01:42:10.285053+0000","last_active":"2024-03-30T01:42:24.278549+0000","last_peered":"2024-03-30T01:42:24.278549+0000","last_clean":"2024-03-30T01:42:24.278549+0000","last_became_active":"2024-03-30T01:42:10.284814+0000","last_became_peered":"2024-03-30T01:42:10.284814+0000","last_unstale":"2024-03-30T01:42:24.278549+0000","last_undegraded":"2024-03-30T01:42:24.278549+0000","last_fullsized":"2024-03-30T01:42:24.278549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:28.875197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283983+0000","last_change":"2024-03-30T01:42:23.822556+0000","last_active":"2024-03-30T01:42:24.283983+0000","last_peered":"2024-03-30T01:42:24.283983+0000","last_clean":"2024-03-30T01:42:24.283983+0000","last_became_active":"2024-03-30T01:42:23.822232+0000","last_became_peered":"2024-03-30T01:42:23.822232+0000","last_unstale":"2024-03-30T01:42:24.283983+0000","last_undegraded":"2024-03-30T01:42:24.283983+0000","last_fullsized":"2024-03-30T01:42:24.283983+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:29:59.201428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812571+0000","last_change":"2024-03-30T01:42:24.301362+0000","last_active":"2024-03-30T01:42:25.812571+0000","last_peered":"2024-03-30T01:42:25.812571+0000","last_clean":"2024-03-30T01:42:25.812571+0000","last_became_active":"2024-03-30T01:42:24.301060+0000","last_became_peered":"2024-03-30T01:42:24.301060+0000","last_unstale":"2024-03-30T01:42:25.812571+0000","last_undegraded":"2024-03-30T01:42:25.812571+0000","last_fullsized":"2024-03-30T01:42:25.812571+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:15:46.581141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"39'8","reported_seq":23,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291223+0000","last_change":"2024-03-30T01:42:09.823152+0000","last_active":"2024-03-30T01:42:24.291223+0000","last_peered":"2024-03-30T01:42:24.291223+0000","last_clean":"2024-03-30T01:42:24.291223+0000","last_became_active":"2024-03-30T01:42:09.822826+0000","last_became_peered":"2024-03-30T01:42:09.822826+0000","last_unstale":"2024-03-30T01:42:24.291223+0000","last_undegraded":"2024-03-30T01:42:24.291223+0000","last_fullsized":"2024-03-30T01:42:24.291223+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:17.799215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283446+0000","last_change":"2024-03-30T01:42:10.296944+0000","last_active":"2024-03-30T01:42:24.283446+0000","last_peered":"2024-03-30T01:42:24.283446+0000","last_clean":"2024-03-30T01:42:24.283446+0000","last_became_active":"2024-03-30T01:42:10.296714+0000","last_became_peered":"2024-03-30T01:42:10.296714+0000","last_unstale":"2024-03-30T01:42:24.283446+0000","last_undegraded":"2024-03-30T01:42:24.283446+0000","last_fullsized":"2024-03-30T01:42:24.283446+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283769+0000","last_change":"2024-03-30T01:42:23.836656+0000","last_active":"2024-03-30T01:42:24.283769+0000","last_peered":"2024-03-30T01:42:24.283769+0000","last_clean":"2024-03-30T01:42:24.283769+0000","last_became_active":"2024-03-30T01:42:23.836352+0000","last_became_peered":"2024-03-30T01:42:23.836352+0000","last_unstale":"2024-03-30T01:42:24.283769+0000","last_undegraded":"2024-03-30T01:42:24.283769+0000","last_fullsized":"2024-03-30T01:42:24.283769+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275090+0000","last_change":"2024-03-30T01:42:24.288096+0000","last_active":"2024-03-30T01:42:25.275090+0000","last_peered":"2024-03-30T01:42:25.275090+0000","last_clean":"2024-03-30T01:42:25.275090+0000","last_became_active":"2024-03-30T01:42:24.287765+0000","last_became_peered":"2024-03-30T01:42:24.287765+0000","last_unstale":"2024-03-30T01:42:25.275090+0000","last_undegraded":"2024-03-30T01:42:25.275090+0000","last_fullsized":"2024-03-30T01:42:25.275090+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:57.853997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277360+0000","last_change":"2024-03-30T01:42:09.833439+0000","last_active":"2024-03-30T01:42:24.277360+0000","last_peered":"2024-03-30T01:42:24.277360+0000","last_clean":"2024-03-30T01:42:24.277360+0000","last_became_active":"2024-03-30T01:42:09.833137+0000","last_became_peered":"2024-03-30T01:42:09.833137+0000","last_unstale":"2024-03-30T01:42:24.277360+0000","last_undegraded":"2024-03-30T01:42:24.277360+0000","last_fullsized":"2024-03-30T01:42:24.277360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280507+0000","last_change":"2024-03-30T01:42:10.292182+0000","last_active":"2024-03-30T01:42:24.280507+0000","last_peered":"2024-03-30T01:42:24.280507+0000","last_clean":"2024-03-30T01:42:24.280507+0000","last_became_active":"2024-03-30T01:42:10.291868+0000","last_became_peered":"2024-03-30T01:42:10.291868+0000","last_unstale":"2024-03-30T01:42:24.280507+0000","last_undegraded":"2024-03-30T01:42:24.280507+0000","last_fullsized":"2024-03-30T01:42:24.280507+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:00.232241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282905+0000","last_change":"2024-03-30T01:42:23.829400+0000","last_active":"2024-03-30T01:42:24.282905+0000","last_peered":"2024-03-30T01:42:24.282905+0000","last_clean":"2024-03-30T01:42:24.282905+0000","last_became_active":"2024-03-30T01:42:23.829058+0000","last_became_peered":"2024-03-30T01:42:23.829058+0000","last_unstale":"2024-03-30T01:42:24.282905+0000","last_undegraded":"2024-03-30T01:42:24.282905+0000","last_fullsized":"2024-03-30T01:42:24.282905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:12.384136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811906+0000","last_change":"2024-03-30T01:42:24.300333+0000","last_active":"2024-03-30T01:42:25.811906+0000","last_peered":"2024-03-30T01:42:25.811906+0000","last_clean":"2024-03-30T01:42:25.811906+0000","last_became_active":"2024-03-30T01:42:24.300127+0000","last_became_peered":"2024-03-30T01:42:24.300127+0000","last_unstale":"2024-03-30T01:42:25.811906+0000","last_undegraded":"2024-03-30T01:42:25.811906+0000","last_fullsized":"2024-03-30T01:42:25.811906+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:44.799982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277811+0000","last_change":"2024-03-30T01:42:09.826561+0000","last_active":"2024-03-30T01:42:24.277811+0000","last_peered":"2024-03-30T01:42:24.277811+0000","last_clean":"2024-03-30T01:42:24.277811+0000","last_became_active":"2024-03-30T01:42:09.826059+0000","last_became_peered":"2024-03-30T01:42:09.826059+0000","last_unstale":"2024-03-30T01:42:24.277811+0000","last_undegraded":"2024-03-30T01:42:24.277811+0000","last_fullsized":"2024-03-30T01:42:24.277811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:12.066094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.284182+0000","last_change":"2024-03-30T01:42:10.284755+0000","last_active":"2024-03-30T01:42:24.284182+0000","last_peered":"2024-03-30T01:42:24.284182+0000","last_clean":"2024-03-30T01:42:24.284182+0000","last_became_active":"2024-03-30T01:42:10.284421+0000","last_became_peered":"2024-03-30T01:42:10.284421+0000","last_unstale":"2024-03-30T01:42:24.284182+0000","last_undegraded":"2024-03-30T01:42:24.284182+0000","last_fullsized":"2024-03-30T01:42:24.284182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:42:18.803797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"42'6","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.312657+0000","last_change":"2024-03-30T01:42:23.837809+0000","last_active":"2024-03-30T01:42:24.312657+0000","last_peered":"2024-03-30T01:42:24.312657+0000","last_clean":"2024-03-30T01:42:24.312657+0000","last_became_active":"2024-03-30T01:42:23.837550+0000","last_became_peered":"2024-03-30T01:42:23.837550+0000","last_unstale":"2024-03-30T01:42:24.312657+0000","last_undegraded":"2024-03-30T01:42:24.312657+0000","last_fullsized":"2024-03-30T01:42:24.312657+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:44.146196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812751+0000","last_change":"2024-03-30T01:42:24.291310+0000","last_active":"2024-03-30T01:42:25.812751+0000","last_peered":"2024-03-30T01:42:25.812751+0000","last_clean":"2024-03-30T01:42:25.812751+0000","last_became_active":"2024-03-30T01:42:24.291034+0000","last_became_peered":"2024-03-30T01:42:24.291034+0000","last_unstale":"2024-03-30T01:42:25.812751+0000","last_undegraded":"2024-03-30T01:42:25.812751+0000","last_fullsized":"2024-03-30T01:42:25.812751+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:58.973438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297262+0000","last_change":"2024-03-30T01:42:09.828624+0000","last_active":"2024-03-30T01:42:24.297262+0000","last_peered":"2024-03-30T01:42:24.297262+0000","last_clean":"2024-03-30T01:42:24.297262+0000","last_became_active":"2024-03-30T01:42:09.828341+0000","last_became_peered":"2024-03-30T01:42:09.828341+0000","last_unstale":"2024-03-30T01:42:24.297262+0000","last_undegraded":"2024-03-30T01:42:24.297262+0000","last_fullsized":"2024-03-30T01:42:24.297262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:32.138427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275251+0000","last_change":"2024-03-30T01:42:10.295259+0000","last_active":"2024-03-30T01:42:24.275251+0000","last_peered":"2024-03-30T01:42:24.275251+0000","last_clean":"2024-03-30T01:42:24.275251+0000","last_became_active":"2024-03-30T01:42:10.295054+0000","last_became_peered":"2024-03-30T01:42:10.295054+0000","last_unstale":"2024-03-30T01:42:24.275251+0000","last_undegraded":"2024-03-30T01:42:24.275251+0000","last_fullsized":"2024-03-30T01:42:24.275251+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:22:36.942583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"42'3","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.301595+0000","last_change":"2024-03-30T01:42:23.832976+0000","last_active":"2024-03-30T01:42:24.301595+0000","last_peered":"2024-03-30T01:42:24.301595+0000","last_clean":"2024-03-30T01:42:24.301595+0000","last_became_active":"2024-03-30T01:42:23.832769+0000","last_became_peered":"2024-03-30T01:42:23.832769+0000","last_unstale":"2024-03-30T01:42:24.301595+0000","last_undegraded":"2024-03-30T01:42:24.301595+0000","last_fullsized":"2024-03-30T01:42:24.301595+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:54:59.010522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606373+0000","last_change":"2024-03-30T01:42:24.304851+0000","last_active":"2024-03-30T01:42:25.606373+0000","last_peered":"2024-03-30T01:42:25.606373+0000","last_clean":"2024-03-30T01:42:25.606373+0000","last_became_active":"2024-03-30T01:42:24.301522+0000","last_became_peered":"2024-03-30T01:42:24.301522+0000","last_unstale":"2024-03-30T01:42:25.606373+0000","last_undegraded":"2024-03-30T01:42:25.606373+0000","last_fullsized":"2024-03-30T01:42:25.606373+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:39.518546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283467+0000","last_change":"2024-03-30T01:42:09.820512+0000","last_active":"2024-03-30T01:42:24.283467+0000","last_peered":"2024-03-30T01:42:24.283467+0000","last_clean":"2024-03-30T01:42:24.283467+0000","last_became_active":"2024-03-30T01:42:09.820253+0000","last_became_peered":"2024-03-30T01:42:09.820253+0000","last_unstale":"2024-03-30T01:42:24.283467+0000","last_undegraded":"2024-03-30T01:42:24.283467+0000","last_fullsized":"2024-03-30T01:42:24.283467+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:04.455801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290107+0000","last_change":"2024-03-30T01:42:10.291872+0000","last_active":"2024-03-30T01:42:24.290107+0000","last_peered":"2024-03-30T01:42:24.290107+0000","last_clean":"2024-03-30T01:42:24.290107+0000","last_became_active":"2024-03-30T01:42:10.291199+0000","last_became_peered":"2024-03-30T01:42:10.291199+0000","last_unstale":"2024-03-30T01:42:24.290107+0000","last_undegraded":"2024-03-30T01:42:24.290107+0000","last_fullsized":"2024-03-30T01:42:24.290107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297204+0000","last_change":"2024-03-30T01:42:23.824219+0000","last_active":"2024-03-30T01:42:24.297204+0000","last_peered":"2024-03-30T01:42:24.297204+0000","last_clean":"2024-03-30T01:42:24.297204+0000","last_became_active":"2024-03-30T01:42:23.823949+0000","last_became_peered":"2024-03-30T01:42:23.823949+0000","last_unstale":"2024-03-30T01:42:24.297204+0000","last_undegraded":"2024-03-30T01:42:24.297204+0000","last_fullsized":"2024-03-30T01:42:24.297204+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:34.173290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813275+0000","last_change":"2024-03-30T01:42:24.301996+0000","last_active":"2024-03-30T01:42:25.813275+0000","last_peered":"2024-03-30T01:42:25.813275+0000","last_clean":"2024-03-30T01:42:25.813275+0000","last_became_active":"2024-03-30T01:42:24.301718+0000","last_became_peered":"2024-03-30T01:42:24.301718+0000","last_unstale":"2024-03-30T01:42:25.813275+0000","last_undegraded":"2024-03-30T01:42:25.813275+0000","last_fullsized":"2024-03-30T01:42:25.813275+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:45.153231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.286403+0000","last_change":"2024-03-30T01:42:09.826326+0000","last_active":"2024-03-30T01:42:24.286403+0000","last_peered":"2024-03-30T01:42:24.286403+0000","last_clean":"2024-03-30T01:42:24.286403+0000","last_became_active":"2024-03-30T01:42:09.826050+0000","last_became_peered":"2024-03-30T01:42:09.826050+0000","last_unstale":"2024-03-30T01:42:24.286403+0000","last_undegraded":"2024-03-30T01:42:24.286403+0000","last_fullsized":"2024-03-30T01:42:24.286403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:31.796813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.288773+0000","last_change":"2024-03-30T01:42:10.303359+0000","last_active":"2024-03-30T01:42:24.288773+0000","last_peered":"2024-03-30T01:42:24.288773+0000","last_clean":"2024-03-30T01:42:24.288773+0000","last_became_active":"2024-03-30T01:42:10.303039+0000","last_became_peered":"2024-03-30T01:42:10.303039+0000","last_unstale":"2024-03-30T01:42:24.288773+0000","last_undegraded":"2024-03-30T01:42:24.288773+0000","last_fullsized":"2024-03-30T01:42:24.288773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.300271+0000","last_change":"2024-03-30T01:42:23.830543+0000","last_active":"2024-03-30T01:42:24.300271+0000","last_peered":"2024-03-30T01:42:24.300271+0000","last_clean":"2024-03-30T01:42:24.300271+0000","last_became_active":"2024-03-30T01:42:23.830218+0000","last_became_peered":"2024-03-30T01:42:23.830218+0000","last_unstale":"2024-03-30T01:42:24.300271+0000","last_undegraded":"2024-03-30T01:42:24.300271+0000","last_fullsized":"2024-03-30T01:42:24.300271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:04:36.452658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812429+0000","last_change":"2024-03-30T01:42:24.302598+0000","last_active":"2024-03-30T01:42:25.812429+0000","last_peered":"2024-03-30T01:42:25.812429+0000","last_clean":"2024-03-30T01:42:25.812429+0000","last_became_active":"2024-03-30T01:42:24.302365+0000","last_became_peered":"2024-03-30T01:42:24.302365+0000","last_unstale":"2024-03-30T01:42:25.812429+0000","last_undegraded":"2024-03-30T01:42:25.812429+0000","last_fullsized":"2024-03-30T01:42:25.812429+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:45.153231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282526+0000","last_change":"2024-03-30T01:42:09.831981+0000","last_active":"2024-03-30T01:42:24.282526+0000","last_peered":"2024-03-30T01:42:24.282526+0000","last_clean":"2024-03-30T01:42:24.282526+0000","last_became_active":"2024-03-30T01:42:09.831647+0000","last_became_peered":"2024-03-30T01:42:09.831647+0000","last_unstale":"2024-03-30T01:42:24.282526+0000","last_undegraded":"2024-03-30T01:42:24.282526+0000","last_fullsized":"2024-03-30T01:42:24.282526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:02.883477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277377+0000","last_change":"2024-03-30T01:42:10.289213+0000","last_active":"2024-03-30T01:42:24.277377+0000","last_peered":"2024-03-30T01:42:24.277377+0000","last_clean":"2024-03-30T01:42:24.277377+0000","last_became_active":"2024-03-30T01:42:10.288677+0000","last_became_peered":"2024-03-30T01:42:10.288677+0000","last_unstale":"2024-03-30T01:42:24.277377+0000","last_undegraded":"2024-03-30T01:42:24.277377+0000","last_fullsized":"2024-03-30T01:42:24.277377+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:28:36.125746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278504+0000","last_change":"2024-03-30T01:42:23.829725+0000","last_active":"2024-03-30T01:42:24.278504+0000","last_peered":"2024-03-30T01:42:24.278504+0000","last_clean":"2024-03-30T01:42:24.278504+0000","last_became_active":"2024-03-30T01:42:23.829414+0000","last_became_peered":"2024-03-30T01:42:23.829414+0000","last_unstale":"2024-03-30T01:42:24.278504+0000","last_undegraded":"2024-03-30T01:42:24.278504+0000","last_fullsized":"2024-03-30T01:42:24.278504+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:32.897671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813494+0000","last_change":"2024-03-30T01:42:24.306123+0000","last_active":"2024-03-30T01:42:25.813494+0000","last_peered":"2024-03-30T01:42:25.813494+0000","last_clean":"2024-03-30T01:42:25.813494+0000","last_became_active":"2024-03-30T01:42:24.305878+0000","last_became_peered":"2024-03-30T01:42:24.305878+0000","last_unstale":"2024-03-30T01:42:25.813494+0000","last_undegraded":"2024-03-30T01:42:25.813494+0000","last_fullsized":"2024-03-30T01:42:25.813494+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:22:35.546913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277010+0000","last_change":"2024-03-30T01:42:09.831496+0000","last_active":"2024-03-30T01:42:24.277010+0000","last_peered":"2024-03-30T01:42:24.277010+0000","last_clean":"2024-03-30T01:42:24.277010+0000","last_became_active":"2024-03-30T01:42:09.831197+0000","last_became_peered":"2024-03-30T01:42:09.831197+0000","last_unstale":"2024-03-30T01:42:24.277010+0000","last_undegraded":"2024-03-30T01:42:24.277010+0000","last_fullsized":"2024-03-30T01:42:24.277010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:20.251461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278647+0000","last_change":"2024-03-30T01:42:10.287526+0000","last_active":"2024-03-30T01:42:24.278647+0000","last_peered":"2024-03-30T01:42:24.278647+0000","last_clean":"2024-03-30T01:42:24.278647+0000","last_became_active":"2024-03-30T01:42:10.287255+0000","last_became_peered":"2024-03-30T01:42:10.287255+0000","last_unstale":"2024-03-30T01:42:24.278647+0000","last_undegraded":"2024-03-30T01:42:24.278647+0000","last_fullsized":"2024-03-30T01:42:24.278647+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273224+0000","last_change":"2024-03-30T01:42:23.825156+0000","last_active":"2024-03-30T01:42:24.273224+0000","last_peered":"2024-03-30T01:42:24.273224+0000","last_clean":"2024-03-30T01:42:24.273224+0000","last_became_active":"2024-03-30T01:42:23.824623+0000","last_became_peered":"2024-03-30T01:42:23.824623+0000","last_unstale":"2024-03-30T01:42:24.273224+0000","last_undegraded":"2024-03-30T01:42:24.273224+0000","last_fullsized":"2024-03-30T01:42:24.273224+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:35.849177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812469+0000","last_change":"2024-03-30T01:42:24.299646+0000","last_active":"2024-03-30T01:42:25.812469+0000","last_peered":"2024-03-30T01:42:25.812469+0000","last_clean":"2024-03-30T01:42:25.812469+0000","last_became_active":"2024-03-30T01:42:24.298957+0000","last_became_peered":"2024-03-30T01:42:24.298957+0000","last_unstale":"2024-03-30T01:42:25.812469+0000","last_undegraded":"2024-03-30T01:42:25.812469+0000","last_fullsized":"2024-03-30T01:42:25.812469+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:49:25.818990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.289951+0000","last_change":"2024-03-30T01:42:09.824809+0000","last_active":"2024-03-30T01:42:24.289951+0000","last_peered":"2024-03-30T01:42:24.289951+0000","last_clean":"2024-03-30T01:42:24.289951+0000","last_became_active":"2024-03-30T01:42:09.824493+0000","last_became_peered":"2024-03-30T01:42:09.824493+0000","last_unstale":"2024-03-30T01:42:24.289951+0000","last_undegraded":"2024-03-30T01:42:24.289951+0000","last_fullsized":"2024-03-30T01:42:24.289951+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:43.662893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297494+0000","last_change":"2024-03-30T01:42:10.291979+0000","last_active":"2024-03-30T01:42:24.297494+0000","last_peered":"2024-03-30T01:42:24.297494+0000","last_clean":"2024-03-30T01:42:24.297494+0000","last_became_active":"2024-03-30T01:42:10.291708+0000","last_became_peered":"2024-03-30T01:42:10.291708+0000","last_unstale":"2024-03-30T01:42:24.297494+0000","last_undegraded":"2024-03-30T01:42:24.297494+0000","last_fullsized":"2024-03-30T01:42:24.297494+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:46.378176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.299720+0000","last_change":"2024-03-30T01:42:23.828108+0000","last_active":"2024-03-30T01:42:24.299720+0000","last_peered":"2024-03-30T01:42:24.299720+0000","last_clean":"2024-03-30T01:42:24.299720+0000","last_became_active":"2024-03-30T01:42:23.827597+0000","last_became_peered":"2024-03-30T01:42:23.827597+0000","last_unstale":"2024-03-30T01:42:24.299720+0000","last_undegraded":"2024-03-30T01:42:24.299720+0000","last_fullsized":"2024-03-30T01:42:24.299720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:50.202292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275252+0000","last_change":"2024-03-30T01:42:24.292625+0000","last_active":"2024-03-30T01:42:25.275252+0000","last_peered":"2024-03-30T01:42:25.275252+0000","last_clean":"2024-03-30T01:42:25.275252+0000","last_became_active":"2024-03-30T01:42:24.292322+0000","last_became_peered":"2024-03-30T01:42:24.292322+0000","last_unstale":"2024-03-30T01:42:25.275252+0000","last_undegraded":"2024-03-30T01:42:25.275252+0000","last_fullsized":"2024-03-30T01:42:25.275252+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:44.799982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290658+0000","last_change":"2024-03-30T01:42:09.829349+0000","last_active":"2024-03-30T01:42:24.290658+0000","last_peered":"2024-03-30T01:42:24.290658+0000","last_clean":"2024-03-30T01:42:24.290658+0000","last_became_active":"2024-03-30T01:42:09.828931+0000","last_became_peered":"2024-03-30T01:42:09.828931+0000","last_unstale":"2024-03-30T01:42:24.290658+0000","last_undegraded":"2024-03-30T01:42:24.290658+0000","last_fullsized":"2024-03-30T01:42:24.290658+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:51.819885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283722+0000","last_change":"2024-03-30T01:42:10.300292+0000","last_active":"2024-03-30T01:42:24.283722+0000","last_peered":"2024-03-30T01:42:24.283722+0000","last_clean":"2024-03-30T01:42:24.283722+0000","last_became_active":"2024-03-30T01:42:10.299867+0000","last_became_peered":"2024-03-30T01:42:10.299867+0000","last_unstale":"2024-03-30T01:42:24.283722+0000","last_undegraded":"2024-03-30T01:42:24.283722+0000","last_fullsized":"2024-03-30T01:42:24.283722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:32:26.929411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"42'4","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.298762+0000","last_change":"2024-03-30T01:42:23.822358+0000","last_active":"2024-03-30T01:42:24.298762+0000","last_peered":"2024-03-30T01:42:24.298762+0000","last_clean":"2024-03-30T01:42:24.298762+0000","last_became_active":"2024-03-30T01:42:23.822053+0000","last_became_peered":"2024-03-30T01:42:23.822053+0000","last_unstale":"2024-03-30T01:42:24.298762+0000","last_undegraded":"2024-03-30T01:42:24.298762+0000","last_fullsized":"2024-03-30T01:42:24.298762+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:59:13.222722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812080+0000","last_change":"2024-03-30T01:42:24.292298+0000","last_active":"2024-03-30T01:42:25.812080+0000","last_peered":"2024-03-30T01:42:25.812080+0000","last_clean":"2024-03-30T01:42:25.812080+0000","last_became_active":"2024-03-30T01:42:24.292030+0000","last_became_peered":"2024-03-30T01:42:24.292030+0000","last_unstale":"2024-03-30T01:42:25.812080+0000","last_undegraded":"2024-03-30T01:42:25.812080+0000","last_fullsized":"2024-03-30T01:42:25.812080+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:44.799982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280617+0000","last_change":"2024-03-30T01:42:09.829899+0000","last_active":"2024-03-30T01:42:24.280617+0000","last_peered":"2024-03-30T01:42:24.280617+0000","last_clean":"2024-03-30T01:42:24.280617+0000","last_became_active":"2024-03-30T01:42:09.829601+0000","last_became_peered":"2024-03-30T01:42:09.829601+0000","last_unstale":"2024-03-30T01:42:24.280617+0000","last_undegraded":"2024-03-30T01:42:24.280617+0000","last_fullsized":"2024-03-30T01:42:24.280617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:30.887459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290890+0000","last_change":"2024-03-30T01:42:10.298381+0000","last_active":"2024-03-30T01:42:24.290890+0000","last_peered":"2024-03-30T01:42:24.290890+0000","last_clean":"2024-03-30T01:42:24.290890+0000","last_became_active":"2024-03-30T01:42:10.298109+0000","last_became_peered":"2024-03-30T01:42:10.298109+0000","last_unstale":"2024-03-30T01:42:24.290890+0000","last_undegraded":"2024-03-30T01:42:24.290890+0000","last_fullsized":"2024-03-30T01:42:24.290890+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.295295+0000","last_change":"2024-03-30T01:42:23.840369+0000","last_active":"2024-03-30T01:42:24.295295+0000","last_peered":"2024-03-30T01:42:24.295295+0000","last_clean":"2024-03-30T01:42:24.295295+0000","last_became_active":"2024-03-30T01:42:23.840105+0000","last_became_peered":"2024-03-30T01:42:23.840105+0000","last_unstale":"2024-03-30T01:42:24.295295+0000","last_undegraded":"2024-03-30T01:42:24.295295+0000","last_fullsized":"2024-03-30T01:42:24.295295+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:26.348980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811796+0000","last_change":"2024-03-30T01:42:24.299326+0000","last_active":"2024-03-30T01:42:25.811796+0000","last_peered":"2024-03-30T01:42:25.811796+0000","last_clean":"2024-03-30T01:42:25.811796+0000","last_became_active":"2024-03-30T01:42:24.299030+0000","last_became_peered":"2024-03-30T01:42:24.299030+0000","last_unstale":"2024-03-30T01:42:25.811796+0000","last_undegraded":"2024-03-30T01:42:25.811796+0000","last_fullsized":"2024-03-30T01:42:25.811796+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:57.959269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.274118+0000","last_change":"2024-03-30T01:42:09.824850+0000","last_active":"2024-03-30T01:42:24.274118+0000","last_peered":"2024-03-30T01:42:24.274118+0000","last_clean":"2024-03-30T01:42:24.274118+0000","last_became_active":"2024-03-30T01:42:09.824537+0000","last_became_peered":"2024-03-30T01:42:09.824537+0000","last_unstale":"2024-03-30T01:42:24.274118+0000","last_undegraded":"2024-03-30T01:42:24.274118+0000","last_fullsized":"2024-03-30T01:42:24.274118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:08.800098+0000","last_clean_scrub_stamp":"2024-03-30T01:42:08.800098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:17.759503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282490+0000","last_change":"2024-03-30T01:42:10.285919+0000","last_active":"2024-03-30T01:42:24.282490+0000","last_peered":"2024-03-30T01:42:24.282490+0000","last_clean":"2024-03-30T01:42:24.282490+0000","last_became_active":"2024-03-30T01:42:10.285629+0000","last_became_peered":"2024-03-30T01:42:10.285629+0000","last_unstale":"2024-03-30T01:42:24.282490+0000","last_undegraded":"2024-03-30T01:42:24.282490+0000","last_fullsized":"2024-03-30T01:42:24.282490+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"42'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.300062+0000","last_change":"2024-03-30T01:42:23.831900+0000","last_active":"2024-03-30T01:42:24.300062+0000","last_peered":"2024-03-30T01:42:24.300062+0000","last_clean":"2024-03-30T01:42:24.300062+0000","last_became_active":"2024-03-30T01:42:23.831395+0000","last_became_peered":"2024-03-30T01:42:23.831395+0000","last_unstale":"2024-03-30T01:42:24.300062+0000","last_undegraded":"2024-03-30T01:42:24.300062+0000","last_fullsized":"2024-03-30T01:42:24.300062+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:22.795162+0000","last_clean_scrub_stamp":"2024-03-30T01:42:22.795162+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:57:35.672193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318446+0000","last_change":"2024-03-30T01:42:24.296156+0000","last_active":"2024-03-30T01:42:25.318446+0000","last_peered":"2024-03-30T01:42:25.318446+0000","last_clean":"2024-03-30T01:42:25.318446+0000","last_became_active":"2024-03-30T01:42:24.295854+0000","last_became_peered":"2024-03-30T01:42:24.295854+0000","last_unstale":"2024-03-30T01:42:25.318446+0000","last_undegraded":"2024-03-30T01:42:25.318446+0000","last_fullsized":"2024-03-30T01:42:25.318446+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:13.391171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813761+0000","last_change":"2024-03-30T01:42:24.297566+0000","last_active":"2024-03-30T01:42:25.813761+0000","last_peered":"2024-03-30T01:42:25.813761+0000","last_clean":"2024-03-30T01:42:25.813761+0000","last_became_active":"2024-03-30T01:42:24.297251+0000","last_became_peered":"2024-03-30T01:42:24.297251+0000","last_unstale":"2024-03-30T01:42:25.813761+0000","last_undegraded":"2024-03-30T01:42:25.813761+0000","last_fullsized":"2024-03-30T01:42:25.813761+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:57.853997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.286985+0000","last_change":"2024-03-30T01:42:10.297189+0000","last_active":"2024-03-30T01:42:24.286985+0000","last_peered":"2024-03-30T01:42:24.286985+0000","last_clean":"2024-03-30T01:42:24.286985+0000","last_became_active":"2024-03-30T01:42:10.296821+0000","last_became_peered":"2024-03-30T01:42:10.296821+0000","last_unstale":"2024-03-30T01:42:24.286985+0000","last_undegraded":"2024-03-30T01:42:24.286985+0000","last_fullsized":"2024-03-30T01:42:24.286985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:26:09.506454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606988+0000","last_change":"2024-03-30T01:42:24.296647+0000","last_active":"2024-03-30T01:42:25.606988+0000","last_peered":"2024-03-30T01:42:25.606988+0000","last_clean":"2024-03-30T01:42:25.606988+0000","last_became_active":"2024-03-30T01:42:24.288648+0000","last_became_peered":"2024-03-30T01:42:24.288648+0000","last_unstale":"2024-03-30T01:42:25.606988+0000","last_undegraded":"2024-03-30T01:42:25.606988+0000","last_fullsized":"2024-03-30T01:42:25.606988+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:10.469402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283864+0000","last_change":"2024-03-30T01:42:10.274200+0000","last_active":"2024-03-30T01:42:24.283864+0000","last_peered":"2024-03-30T01:42:24.283864+0000","last_clean":"2024-03-30T01:42:24.283864+0000","last_became_active":"2024-03-30T01:42:10.273923+0000","last_became_peered":"2024-03-30T01:42:10.273923+0000","last_unstale":"2024-03-30T01:42:24.283864+0000","last_undegraded":"2024-03-30T01:42:24.283864+0000","last_fullsized":"2024-03-30T01:42:24.283864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:44.766475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318538+0000","last_change":"2024-03-30T01:42:24.298867+0000","last_active":"2024-03-30T01:42:25.318538+0000","last_peered":"2024-03-30T01:42:25.318538+0000","last_clean":"2024-03-30T01:42:25.318538+0000","last_became_active":"2024-03-30T01:42:24.298557+0000","last_became_peered":"2024-03-30T01:42:24.298557+0000","last_unstale":"2024-03-30T01:42:25.318538+0000","last_undegraded":"2024-03-30T01:42:25.318538+0000","last_fullsized":"2024-03-30T01:42:25.318538+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:02:13.092965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273427+0000","last_change":"2024-03-30T01:42:10.290078+0000","last_active":"2024-03-30T01:42:24.273427+0000","last_peered":"2024-03-30T01:42:24.273427+0000","last_clean":"2024-03-30T01:42:24.273427+0000","last_became_active":"2024-03-30T01:42:10.289127+0000","last_became_peered":"2024-03-30T01:42:10.289127+0000","last_unstale":"2024-03-30T01:42:24.273427+0000","last_undegraded":"2024-03-30T01:42:24.273427+0000","last_fullsized":"2024-03-30T01:42:24.273427+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:36.303285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606622+0000","last_change":"2024-03-30T01:42:24.288253+0000","last_active":"2024-03-30T01:42:25.606622+0000","last_peered":"2024-03-30T01:42:25.606622+0000","last_clean":"2024-03-30T01:42:25.606622+0000","last_became_active":"2024-03-30T01:42:24.287967+0000","last_became_peered":"2024-03-30T01:42:24.287967+0000","last_unstale":"2024-03-30T01:42:25.606622+0000","last_undegraded":"2024-03-30T01:42:25.606622+0000","last_fullsized":"2024-03-30T01:42:25.606622+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:01.354401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275460+0000","last_change":"2024-03-30T01:42:10.291099+0000","last_active":"2024-03-30T01:42:24.275460+0000","last_peered":"2024-03-30T01:42:24.275460+0000","last_clean":"2024-03-30T01:42:24.275460+0000","last_became_active":"2024-03-30T01:42:10.290802+0000","last_became_peered":"2024-03-30T01:42:10.290802+0000","last_unstale":"2024-03-30T01:42:24.275460+0000","last_undegraded":"2024-03-30T01:42:24.275460+0000","last_fullsized":"2024-03-30T01:42:24.275460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:37:13.070677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811731+0000","last_change":"2024-03-30T01:42:24.302481+0000","last_active":"2024-03-30T01:42:25.811731+0000","last_peered":"2024-03-30T01:42:25.811731+0000","last_clean":"2024-03-30T01:42:25.811731+0000","last_became_active":"2024-03-30T01:42:24.302183+0000","last_became_peered":"2024-03-30T01:42:24.302183+0000","last_unstale":"2024-03-30T01:42:25.811731+0000","last_undegraded":"2024-03-30T01:42:25.811731+0000","last_fullsized":"2024-03-30T01:42:25.811731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:34.136863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277627+0000","last_change":"2024-03-30T01:42:10.288633+0000","last_active":"2024-03-30T01:42:24.277627+0000","last_peered":"2024-03-30T01:42:24.277627+0000","last_clean":"2024-03-30T01:42:24.277627+0000","last_became_active":"2024-03-30T01:42:10.288326+0000","last_became_peered":"2024-03-30T01:42:10.288326+0000","last_unstale":"2024-03-30T01:42:24.277627+0000","last_undegraded":"2024-03-30T01:42:24.277627+0000","last_fullsized":"2024-03-30T01:42:24.277627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:16:48.756729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318329+0000","last_change":"2024-03-30T01:42:24.295937+0000","last_active":"2024-03-30T01:42:25.318329+0000","last_peered":"2024-03-30T01:42:25.318329+0000","last_clean":"2024-03-30T01:42:25.318329+0000","last_became_active":"2024-03-30T01:42:24.295604+0000","last_became_peered":"2024-03-30T01:42:24.295604+0000","last_unstale":"2024-03-30T01:42:25.318329+0000","last_undegraded":"2024-03-30T01:42:25.318329+0000","last_fullsized":"2024-03-30T01:42:25.318329+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:00.017558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.274341+0000","last_change":"2024-03-30T01:42:10.274845+0000","last_active":"2024-03-30T01:42:24.274341+0000","last_peered":"2024-03-30T01:42:24.274341+0000","last_clean":"2024-03-30T01:42:24.274341+0000","last_became_active":"2024-03-30T01:42:10.274566+0000","last_became_peered":"2024-03-30T01:42:10.274566+0000","last_unstale":"2024-03-30T01:42:24.274341+0000","last_undegraded":"2024-03-30T01:42:24.274341+0000","last_fullsized":"2024-03-30T01:42:24.274341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:32.801396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606081+0000","last_change":"2024-03-30T01:42:24.295712+0000","last_active":"2024-03-30T01:42:25.606081+0000","last_peered":"2024-03-30T01:42:25.606081+0000","last_clean":"2024-03-30T01:42:25.606081+0000","last_became_active":"2024-03-30T01:42:24.295459+0000","last_became_peered":"2024-03-30T01:42:24.295459+0000","last_unstale":"2024-03-30T01:42:25.606081+0000","last_undegraded":"2024-03-30T01:42:25.606081+0000","last_fullsized":"2024-03-30T01:42:25.606081+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:01:51.251893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.290183+0000","last_change":"2024-03-30T01:42:10.290378+0000","last_active":"2024-03-30T01:42:24.290183+0000","last_peered":"2024-03-30T01:42:24.290183+0000","last_clean":"2024-03-30T01:42:24.290183+0000","last_became_active":"2024-03-30T01:42:10.290022+0000","last_became_peered":"2024-03-30T01:42:10.290022+0000","last_unstale":"2024-03-30T01:42:24.290183+0000","last_undegraded":"2024-03-30T01:42:24.290183+0000","last_fullsized":"2024-03-30T01:42:24.290183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:39.797345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275857+0000","last_change":"2024-03-30T01:42:24.292681+0000","last_active":"2024-03-30T01:42:25.275857+0000","last_peered":"2024-03-30T01:42:25.275857+0000","last_clean":"2024-03-30T01:42:25.275857+0000","last_became_active":"2024-03-30T01:42:24.292407+0000","last_became_peered":"2024-03-30T01:42:24.292407+0000","last_unstale":"2024-03-30T01:42:25.275857+0000","last_undegraded":"2024-03-30T01:42:25.275857+0000","last_fullsized":"2024-03-30T01:42:25.275857+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:33:13.335957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273835+0000","last_change":"2024-03-30T01:42:10.285378+0000","last_active":"2024-03-30T01:42:24.273835+0000","last_peered":"2024-03-30T01:42:24.273835+0000","last_clean":"2024-03-30T01:42:24.273835+0000","last_became_active":"2024-03-30T01:42:10.285105+0000","last_became_peered":"2024-03-30T01:42:10.285105+0000","last_unstale":"2024-03-30T01:42:24.273835+0000","last_undegraded":"2024-03-30T01:42:24.273835+0000","last_fullsized":"2024-03-30T01:42:24.273835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:39:52.824468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812316+0000","last_change":"2024-03-30T01:42:24.299833+0000","last_active":"2024-03-30T01:42:25.812316+0000","last_peered":"2024-03-30T01:42:25.812316+0000","last_clean":"2024-03-30T01:42:25.812316+0000","last_became_active":"2024-03-30T01:42:24.299327+0000","last_became_peered":"2024-03-30T01:42:24.299327+0000","last_unstale":"2024-03-30T01:42:25.812316+0000","last_undegraded":"2024-03-30T01:42:25.812316+0000","last_fullsized":"2024-03-30T01:42:25.812316+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:11.669273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.274758+0000","last_change":"2024-03-30T01:42:10.286887+0000","last_active":"2024-03-30T01:42:24.274758+0000","last_peered":"2024-03-30T01:42:24.274758+0000","last_clean":"2024-03-30T01:42:24.274758+0000","last_became_active":"2024-03-30T01:42:10.278107+0000","last_became_peered":"2024-03-30T01:42:10.278107+0000","last_unstale":"2024-03-30T01:42:24.274758+0000","last_undegraded":"2024-03-30T01:42:24.274758+0000","last_fullsized":"2024-03-30T01:42:24.274758+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.607019+0000","last_change":"2024-03-30T01:42:24.304463+0000","last_active":"2024-03-30T01:42:25.607019+0000","last_peered":"2024-03-30T01:42:25.607019+0000","last_clean":"2024-03-30T01:42:25.607019+0000","last_became_active":"2024-03-30T01:42:24.304127+0000","last_became_peered":"2024-03-30T01:42:24.304127+0000","last_unstale":"2024-03-30T01:42:25.607019+0000","last_undegraded":"2024-03-30T01:42:25.607019+0000","last_fullsized":"2024-03-30T01:42:25.607019+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:28:30.527405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.284166+0000","last_change":"2024-03-30T01:42:10.281331+0000","last_active":"2024-03-30T01:42:24.284166+0000","last_peered":"2024-03-30T01:42:24.284166+0000","last_clean":"2024-03-30T01:42:24.284166+0000","last_became_active":"2024-03-30T01:42:10.281011+0000","last_became_peered":"2024-03-30T01:42:10.281011+0000","last_unstale":"2024-03-30T01:42:24.284166+0000","last_undegraded":"2024-03-30T01:42:24.284166+0000","last_fullsized":"2024-03-30T01:42:24.284166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.288897+0000","last_change":"2024-03-30T01:42:10.299194+0000","last_active":"2024-03-30T01:42:24.288897+0000","last_peered":"2024-03-30T01:42:24.288897+0000","last_clean":"2024-03-30T01:42:24.288897+0000","last_became_active":"2024-03-30T01:42:10.298900+0000","last_became_peered":"2024-03-30T01:42:10.298900+0000","last_unstale":"2024-03-30T01:42:24.288897+0000","last_undegraded":"2024-03-30T01:42:24.288897+0000","last_fullsized":"2024-03-30T01:42:24.288897+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.318379+0000","last_change":"2024-03-30T01:42:24.278409+0000","last_active":"2024-03-30T01:42:25.318379+0000","last_peered":"2024-03-30T01:42:25.318379+0000","last_clean":"2024-03-30T01:42:25.318379+0000","last_became_active":"2024-03-30T01:42:24.278120+0000","last_became_peered":"2024-03-30T01:42:24.278120+0000","last_unstale":"2024-03-30T01:42:25.318379+0000","last_undegraded":"2024-03-30T01:42:25.318379+0000","last_fullsized":"2024-03-30T01:42:25.318379+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:51.209327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.275167+0000","last_change":"2024-03-30T01:42:24.307026+0000","last_active":"2024-03-30T01:42:25.275167+0000","last_peered":"2024-03-30T01:42:25.275167+0000","last_clean":"2024-03-30T01:42:25.275167+0000","last_became_active":"2024-03-30T01:42:24.306782+0000","last_became_peered":"2024-03-30T01:42:24.306782+0000","last_unstale":"2024-03-30T01:42:25.275167+0000","last_undegraded":"2024-03-30T01:42:25.275167+0000","last_fullsized":"2024-03-30T01:42:25.275167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:22:35.546913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.276861+0000","last_change":"2024-03-30T01:42:10.304038+0000","last_active":"2024-03-30T01:42:24.276861+0000","last_peered":"2024-03-30T01:42:24.276861+0000","last_clean":"2024-03-30T01:42:24.276861+0000","last_became_active":"2024-03-30T01:42:10.303747+0000","last_became_peered":"2024-03-30T01:42:10.303747+0000","last_unstale":"2024-03-30T01:42:24.276861+0000","last_undegraded":"2024-03-30T01:42:24.276861+0000","last_fullsized":"2024-03-30T01:42:24.276861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:00:28.775316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.274030+0000","last_change":"2024-03-30T01:42:10.300160+0000","last_active":"2024-03-30T01:42:24.274030+0000","last_peered":"2024-03-30T01:42:24.274030+0000","last_clean":"2024-03-30T01:42:24.274030+0000","last_became_active":"2024-03-30T01:42:10.299728+0000","last_became_peered":"2024-03-30T01:42:10.299728+0000","last_unstale":"2024-03-30T01:42:24.274030+0000","last_undegraded":"2024-03-30T01:42:24.274030+0000","last_fullsized":"2024-03-30T01:42:24.274030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:20.057132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812787+0000","last_change":"2024-03-30T01:42:24.307796+0000","last_active":"2024-03-30T01:42:25.812787+0000","last_peered":"2024-03-30T01:42:25.812787+0000","last_clean":"2024-03-30T01:42:25.812787+0000","last_became_active":"2024-03-30T01:42:24.307592+0000","last_became_peered":"2024-03-30T01:42:24.307592+0000","last_unstale":"2024-03-30T01:42:25.812787+0000","last_undegraded":"2024-03-30T01:42:25.812787+0000","last_fullsized":"2024-03-30T01:42:25.812787+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:21.224256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278021+0000","last_change":"2024-03-30T01:42:10.292351+0000","last_active":"2024-03-30T01:42:24.278021+0000","last_peered":"2024-03-30T01:42:24.278021+0000","last_clean":"2024-03-30T01:42:24.278021+0000","last_became_active":"2024-03-30T01:42:10.292040+0000","last_became_peered":"2024-03-30T01:42:10.292040+0000","last_unstale":"2024-03-30T01:42:24.278021+0000","last_undegraded":"2024-03-30T01:42:24.278021+0000","last_fullsized":"2024-03-30T01:42:24.278021+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:40:09.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606379+0000","last_change":"2024-03-30T01:42:24.300935+0000","last_active":"2024-03-30T01:42:25.606379+0000","last_peered":"2024-03-30T01:42:25.606379+0000","last_clean":"2024-03-30T01:42:25.606379+0000","last_became_active":"2024-03-30T01:42:24.300663+0000","last_became_peered":"2024-03-30T01:42:24.300663+0000","last_unstale":"2024-03-30T01:42:25.606379+0000","last_undegraded":"2024-03-30T01:42:25.606379+0000","last_fullsized":"2024-03-30T01:42:25.606379+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:12:17.200114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.273912+0000","last_change":"2024-03-30T01:42:10.300393+0000","last_active":"2024-03-30T01:42:24.273912+0000","last_peered":"2024-03-30T01:42:24.273912+0000","last_clean":"2024-03-30T01:42:24.273912+0000","last_became_active":"2024-03-30T01:42:10.299795+0000","last_became_peered":"2024-03-30T01:42:10.299795+0000","last_unstale":"2024-03-30T01:42:24.273912+0000","last_undegraded":"2024-03-30T01:42:24.273912+0000","last_fullsized":"2024-03-30T01:42:24.273912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:18:21.256043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274563+0000","last_change":"2024-03-30T01:42:24.295808+0000","last_active":"2024-03-30T01:42:25.274563+0000","last_peered":"2024-03-30T01:42:25.274563+0000","last_clean":"2024-03-30T01:42:25.274563+0000","last_became_active":"2024-03-30T01:42:24.295490+0000","last_became_peered":"2024-03-30T01:42:24.295490+0000","last_unstale":"2024-03-30T01:42:25.274563+0000","last_undegraded":"2024-03-30T01:42:25.274563+0000","last_fullsized":"2024-03-30T01:42:25.274563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:33.904706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283022+0000","last_change":"2024-03-30T01:42:10.285733+0000","last_active":"2024-03-30T01:42:24.283022+0000","last_peered":"2024-03-30T01:42:24.283022+0000","last_clean":"2024-03-30T01:42:24.283022+0000","last_became_active":"2024-03-30T01:42:10.285428+0000","last_became_peered":"2024-03-30T01:42:10.285428+0000","last_unstale":"2024-03-30T01:42:24.283022+0000","last_undegraded":"2024-03-30T01:42:24.283022+0000","last_fullsized":"2024-03-30T01:42:24.283022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:44:31.892041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606152+0000","last_change":"2024-03-30T01:42:24.296607+0000","last_active":"2024-03-30T01:42:25.606152+0000","last_peered":"2024-03-30T01:42:25.606152+0000","last_clean":"2024-03-30T01:42:25.606152+0000","last_became_active":"2024-03-30T01:42:24.288603+0000","last_became_peered":"2024-03-30T01:42:24.288603+0000","last_unstale":"2024-03-30T01:42:25.606152+0000","last_undegraded":"2024-03-30T01:42:25.606152+0000","last_fullsized":"2024-03-30T01:42:25.606152+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:53:18.158278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813829+0000","last_change":"2024-03-30T01:42:24.297299+0000","last_active":"2024-03-30T01:42:25.813829+0000","last_peered":"2024-03-30T01:42:25.813829+0000","last_clean":"2024-03-30T01:42:25.813829+0000","last_became_active":"2024-03-30T01:42:24.297027+0000","last_became_peered":"2024-03-30T01:42:24.297027+0000","last_unstale":"2024-03-30T01:42:25.813829+0000","last_undegraded":"2024-03-30T01:42:25.813829+0000","last_fullsized":"2024-03-30T01:42:25.813829+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:58.973438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.282495+0000","last_change":"2024-03-30T01:42:10.284419+0000","last_active":"2024-03-30T01:42:24.282495+0000","last_peered":"2024-03-30T01:42:24.282495+0000","last_clean":"2024-03-30T01:42:24.282495+0000","last_became_active":"2024-03-30T01:42:10.284098+0000","last_became_peered":"2024-03-30T01:42:10.284098+0000","last_unstale":"2024-03-30T01:42:24.282495+0000","last_undegraded":"2024-03-30T01:42:24.282495+0000","last_fullsized":"2024-03-30T01:42:24.282495+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:47.438099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274959+0000","last_change":"2024-03-30T01:42:24.299819+0000","last_active":"2024-03-30T01:42:25.274959+0000","last_peered":"2024-03-30T01:42:25.274959+0000","last_clean":"2024-03-30T01:42:25.274959+0000","last_became_active":"2024-03-30T01:42:24.299346+0000","last_became_peered":"2024-03-30T01:42:24.299346+0000","last_unstale":"2024-03-30T01:42:25.274959+0000","last_undegraded":"2024-03-30T01:42:25.274959+0000","last_fullsized":"2024-03-30T01:42:25.274959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:45.153231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277529+0000","last_change":"2024-03-30T01:42:10.293953+0000","last_active":"2024-03-30T01:42:24.277529+0000","last_peered":"2024-03-30T01:42:24.277529+0000","last_clean":"2024-03-30T01:42:24.277529+0000","last_became_active":"2024-03-30T01:42:10.293641+0000","last_became_peered":"2024-03-30T01:42:10.293641+0000","last_unstale":"2024-03-30T01:42:24.277529+0000","last_undegraded":"2024-03-30T01:42:24.277529+0000","last_fullsized":"2024-03-30T01:42:24.277529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:41.932828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.813811+0000","last_change":"2024-03-30T01:42:24.291685+0000","last_active":"2024-03-30T01:42:25.813811+0000","last_peered":"2024-03-30T01:42:25.813811+0000","last_clean":"2024-03-30T01:42:25.813811+0000","last_became_active":"2024-03-30T01:42:24.291392+0000","last_became_peered":"2024-03-30T01:42:24.291392+0000","last_unstale":"2024-03-30T01:42:25.813811+0000","last_undegraded":"2024-03-30T01:42:25.813811+0000","last_fullsized":"2024-03-30T01:42:25.813811+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:24:32.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.283414+0000","last_change":"2024-03-30T01:42:10.294667+0000","last_active":"2024-03-30T01:42:24.283414+0000","last_peered":"2024-03-30T01:42:24.283414+0000","last_clean":"2024-03-30T01:42:24.283414+0000","last_became_active":"2024-03-30T01:42:10.294330+0000","last_became_peered":"2024-03-30T01:42:10.294330+0000","last_unstale":"2024-03-30T01:42:24.283414+0000","last_undegraded":"2024-03-30T01:42:24.283414+0000","last_fullsized":"2024-03-30T01:42:24.283414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606681+0000","last_change":"2024-03-30T01:42:24.296243+0000","last_active":"2024-03-30T01:42:25.606681+0000","last_peered":"2024-03-30T01:42:25.606681+0000","last_clean":"2024-03-30T01:42:25.606681+0000","last_became_active":"2024-03-30T01:42:24.295980+0000","last_became_peered":"2024-03-30T01:42:24.295980+0000","last_unstale":"2024-03-30T01:42:25.606681+0000","last_undegraded":"2024-03-30T01:42:25.606681+0000","last_fullsized":"2024-03-30T01:42:25.606681+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:20:21.487199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.276735+0000","last_change":"2024-03-30T01:42:10.295808+0000","last_active":"2024-03-30T01:42:24.276735+0000","last_peered":"2024-03-30T01:42:24.276735+0000","last_clean":"2024-03-30T01:42:24.276735+0000","last_became_active":"2024-03-30T01:42:10.295460+0000","last_became_peered":"2024-03-30T01:42:10.295460+0000","last_unstale":"2024-03-30T01:42:24.276735+0000","last_undegraded":"2024-03-30T01:42:24.276735+0000","last_fullsized":"2024-03-30T01:42:24.276735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:29.325984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.273874+0000","last_change":"2024-03-30T01:42:24.295697+0000","last_active":"2024-03-30T01:42:25.273874+0000","last_peered":"2024-03-30T01:42:25.273874+0000","last_clean":"2024-03-30T01:42:25.273874+0000","last_became_active":"2024-03-30T01:42:24.295417+0000","last_became_peered":"2024-03-30T01:42:24.295417+0000","last_unstale":"2024-03-30T01:42:25.273874+0000","last_undegraded":"2024-03-30T01:42:25.273874+0000","last_fullsized":"2024-03-30T01:42:25.273874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:44.799982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.278835+0000","last_change":"2024-03-30T01:42:10.294103+0000","last_active":"2024-03-30T01:42:24.278835+0000","last_peered":"2024-03-30T01:42:24.278835+0000","last_clean":"2024-03-30T01:42:24.278835+0000","last_became_active":"2024-03-30T01:42:10.293833+0000","last_became_peered":"2024-03-30T01:42:10.293833+0000","last_unstale":"2024-03-30T01:42:24.278835+0000","last_undegraded":"2024-03-30T01:42:24.278835+0000","last_fullsized":"2024-03-30T01:42:24.278835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:52:22.858695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812622+0000","last_change":"2024-03-30T01:42:24.292418+0000","last_active":"2024-03-30T01:42:25.812622+0000","last_peered":"2024-03-30T01:42:25.812622+0000","last_clean":"2024-03-30T01:42:25.812622+0000","last_became_active":"2024-03-30T01:42:24.292134+0000","last_became_peered":"2024-03-30T01:42:24.292134+0000","last_unstale":"2024-03-30T01:42:25.812622+0000","last_undegraded":"2024-03-30T01:42:25.812622+0000","last_fullsized":"2024-03-30T01:42:25.812622+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:11:21.224256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277244+0000","last_change":"2024-03-30T01:42:10.296160+0000","last_active":"2024-03-30T01:42:24.277244+0000","last_peered":"2024-03-30T01:42:24.277244+0000","last_clean":"2024-03-30T01:42:24.277244+0000","last_became_active":"2024-03-30T01:42:10.295867+0000","last_became_peered":"2024-03-30T01:42:10.295867+0000","last_unstale":"2024-03-30T01:42:24.277244+0000","last_undegraded":"2024-03-30T01:42:24.277244+0000","last_fullsized":"2024-03-30T01:42:24.277244+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:06.595754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.811563+0000","last_change":"2024-03-30T01:42:24.302182+0000","last_active":"2024-03-30T01:42:25.811563+0000","last_peered":"2024-03-30T01:42:25.811563+0000","last_clean":"2024-03-30T01:42:25.811563+0000","last_became_active":"2024-03-30T01:42:24.301909+0000","last_became_peered":"2024-03-30T01:42:24.301909+0000","last_unstale":"2024-03-30T01:42:25.811563+0000","last_undegraded":"2024-03-30T01:42:25.811563+0000","last_fullsized":"2024-03-30T01:42:25.811563+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:11:54.175140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277512+0000","last_change":"2024-03-30T01:42:10.295963+0000","last_active":"2024-03-30T01:42:24.277512+0000","last_peered":"2024-03-30T01:42:24.277512+0000","last_clean":"2024-03-30T01:42:24.277512+0000","last_became_active":"2024-03-30T01:42:10.295545+0000","last_became_peered":"2024-03-30T01:42:10.295545+0000","last_unstale":"2024-03-30T01:42:24.277512+0000","last_undegraded":"2024-03-30T01:42:24.277512+0000","last_fullsized":"2024-03-30T01:42:24.277512+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606146+0000","last_change":"2024-03-30T01:42:24.288825+0000","last_active":"2024-03-30T01:42:25.606146+0000","last_peered":"2024-03-30T01:42:25.606146+0000","last_clean":"2024-03-30T01:42:25.606146+0000","last_became_active":"2024-03-30T01:42:24.288324+0000","last_became_peered":"2024-03-30T01:42:24.288324+0000","last_unstale":"2024-03-30T01:42:25.606146+0000","last_undegraded":"2024-03-30T01:42:25.606146+0000","last_fullsized":"2024-03-30T01:42:25.606146+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:34.136863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280197+0000","last_change":"2024-03-30T01:42:10.285717+0000","last_active":"2024-03-30T01:42:24.280197+0000","last_peered":"2024-03-30T01:42:24.280197+0000","last_clean":"2024-03-30T01:42:24.280197+0000","last_became_active":"2024-03-30T01:42:10.285437+0000","last_became_peered":"2024-03-30T01:42:10.285437+0000","last_unstale":"2024-03-30T01:42:24.280197+0000","last_undegraded":"2024-03-30T01:42:24.280197+0000","last_fullsized":"2024-03-30T01:42:24.280197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:18:45.855390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274662+0000","last_change":"2024-03-30T01:42:24.306285+0000","last_active":"2024-03-30T01:42:25.274662+0000","last_peered":"2024-03-30T01:42:25.274662+0000","last_clean":"2024-03-30T01:42:25.274662+0000","last_became_active":"2024-03-30T01:42:24.305317+0000","last_became_peered":"2024-03-30T01:42:24.305317+0000","last_unstale":"2024-03-30T01:42:25.274662+0000","last_undegraded":"2024-03-30T01:42:25.274662+0000","last_fullsized":"2024-03-30T01:42:25.274662+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:24:32.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291778+0000","last_change":"2024-03-30T01:42:10.298731+0000","last_active":"2024-03-30T01:42:24.291778+0000","last_peered":"2024-03-30T01:42:24.291778+0000","last_clean":"2024-03-30T01:42:24.291778+0000","last_became_active":"2024-03-30T01:42:10.298525+0000","last_became_peered":"2024-03-30T01:42:10.298525+0000","last_unstale":"2024-03-30T01:42:24.291778+0000","last_undegraded":"2024-03-30T01:42:24.291778+0000","last_fullsized":"2024-03-30T01:42:24.291778+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:32:59.338440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812533+0000","last_change":"2024-03-30T01:42:24.306384+0000","last_active":"2024-03-30T01:42:25.812533+0000","last_peered":"2024-03-30T01:42:25.812533+0000","last_clean":"2024-03-30T01:42:25.812533+0000","last_became_active":"2024-03-30T01:42:24.305978+0000","last_became_peered":"2024-03-30T01:42:24.305978+0000","last_unstale":"2024-03-30T01:42:25.812533+0000","last_undegraded":"2024-03-30T01:42:25.812533+0000","last_fullsized":"2024-03-30T01:42:25.812533+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:39:42.096508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.277222+0000","last_change":"2024-03-30T01:42:10.289068+0000","last_active":"2024-03-30T01:42:24.277222+0000","last_peered":"2024-03-30T01:42:24.277222+0000","last_clean":"2024-03-30T01:42:24.277222+0000","last_became_active":"2024-03-30T01:42:10.288282+0000","last_became_peered":"2024-03-30T01:42:10.288282+0000","last_unstale":"2024-03-30T01:42:24.277222+0000","last_undegraded":"2024-03-30T01:42:24.277222+0000","last_fullsized":"2024-03-30T01:42:24.277222+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:31:40.119660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812132+0000","last_change":"2024-03-30T01:42:24.289951+0000","last_active":"2024-03-30T01:42:25.812132+0000","last_peered":"2024-03-30T01:42:25.812132+0000","last_clean":"2024-03-30T01:42:25.812132+0000","last_became_active":"2024-03-30T01:42:24.289446+0000","last_became_peered":"2024-03-30T01:42:24.289446+0000","last_unstale":"2024-03-30T01:42:25.812132+0000","last_undegraded":"2024-03-30T01:42:25.812132+0000","last_fullsized":"2024-03-30T01:42:25.812132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:45:08.115455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.285326+0000","last_change":"2024-03-30T01:42:10.287001+0000","last_active":"2024-03-30T01:42:24.285326+0000","last_peered":"2024-03-30T01:42:24.285326+0000","last_clean":"2024-03-30T01:42:24.285326+0000","last_became_active":"2024-03-30T01:42:10.286700+0000","last_became_peered":"2024-03-30T01:42:10.286700+0000","last_unstale":"2024-03-30T01:42:24.285326+0000","last_undegraded":"2024-03-30T01:42:24.285326+0000","last_fullsized":"2024-03-30T01:42:24.285326+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:07.150495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.276236+0000","last_change":"2024-03-30T01:42:24.292271+0000","last_active":"2024-03-30T01:42:25.276236+0000","last_peered":"2024-03-30T01:42:25.276236+0000","last_clean":"2024-03-30T01:42:25.276236+0000","last_became_active":"2024-03-30T01:42:24.292005+0000","last_became_peered":"2024-03-30T01:42:24.292005+0000","last_unstale":"2024-03-30T01:42:25.276236+0000","last_undegraded":"2024-03-30T01:42:25.276236+0000","last_fullsized":"2024-03-30T01:42:25.276236+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:02:43.323501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275640+0000","last_change":"2024-03-30T01:42:10.299868+0000","last_active":"2024-03-30T01:42:24.275640+0000","last_peered":"2024-03-30T01:42:24.275640+0000","last_clean":"2024-03-30T01:42:24.275640+0000","last_became_active":"2024-03-30T01:42:10.299612+0000","last_became_peered":"2024-03-30T01:42:10.299612+0000","last_unstale":"2024-03-30T01:42:24.275640+0000","last_undegraded":"2024-03-30T01:42:24.275640+0000","last_fullsized":"2024-03-30T01:42:24.275640+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:54:49.471045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.606566+0000","last_change":"2024-03-30T01:42:24.301867+0000","last_active":"2024-03-30T01:42:25.606566+0000","last_peered":"2024-03-30T01:42:25.606566+0000","last_clean":"2024-03-30T01:42:25.606566+0000","last_became_active":"2024-03-30T01:42:24.301239+0000","last_became_peered":"2024-03-30T01:42:24.301239+0000","last_unstale":"2024-03-30T01:42:25.606566+0000","last_undegraded":"2024-03-30T01:42:25.606566+0000","last_fullsized":"2024-03-30T01:42:25.606566+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:00.869560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.275157+0000","last_change":"2024-03-30T01:42:10.294277+0000","last_active":"2024-03-30T01:42:24.275157+0000","last_peered":"2024-03-30T01:42:24.275157+0000","last_clean":"2024-03-30T01:42:24.275157+0000","last_became_active":"2024-03-30T01:42:10.293995+0000","last_became_peered":"2024-03-30T01:42:10.293995+0000","last_unstale":"2024-03-30T01:42:24.275157+0000","last_undegraded":"2024-03-30T01:42:24.275157+0000","last_fullsized":"2024-03-30T01:42:24.275157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:03.888060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.274283+0000","last_change":"2024-03-30T01:42:24.306793+0000","last_active":"2024-03-30T01:42:25.274283+0000","last_peered":"2024-03-30T01:42:25.274283+0000","last_clean":"2024-03-30T01:42:25.274283+0000","last_became_active":"2024-03-30T01:42:24.306099+0000","last_became_peered":"2024-03-30T01:42:24.306099+0000","last_unstale":"2024-03-30T01:42:25.274283+0000","last_undegraded":"2024-03-30T01:42:25.274283+0000","last_fullsized":"2024-03-30T01:42:25.274283+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:43:06.308084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.280198+0000","last_change":"2024-03-30T01:42:10.285050+0000","last_active":"2024-03-30T01:42:24.280198+0000","last_peered":"2024-03-30T01:42:24.280198+0000","last_clean":"2024-03-30T01:42:24.280198+0000","last_became_active":"2024-03-30T01:42:10.284771+0000","last_became_peered":"2024-03-30T01:42:10.284771+0000","last_unstale":"2024-03-30T01:42:24.280198+0000","last_undegraded":"2024-03-30T01:42:24.280198+0000","last_fullsized":"2024-03-30T01:42:24.280198+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.273680+0000","last_change":"2024-03-30T01:42:24.305711+0000","last_active":"2024-03-30T01:42:25.273680+0000","last_peered":"2024-03-30T01:42:25.273680+0000","last_clean":"2024-03-30T01:42:25.273680+0000","last_became_active":"2024-03-30T01:42:24.305425+0000","last_became_peered":"2024-03-30T01:42:24.305425+0000","last_unstale":"2024-03-30T01:42:25.273680+0000","last_undegraded":"2024-03-30T01:42:25.273680+0000","last_fullsized":"2024-03-30T01:42:25.273680+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:03:25.829367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.291583+0000","last_change":"2024-03-30T01:42:10.294965+0000","last_active":"2024-03-30T01:42:24.291583+0000","last_peered":"2024-03-30T01:42:24.291583+0000","last_clean":"2024-03-30T01:42:24.291583+0000","last_became_active":"2024-03-30T01:42:10.294685+0000","last_became_peered":"2024-03-30T01:42:10.294685+0000","last_unstale":"2024-03-30T01:42:24.291583+0000","last_undegraded":"2024-03-30T01:42:24.291583+0000","last_fullsized":"2024-03-30T01:42:24.291583+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:33.733657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-30T01:42:25.812681+0000","last_change":"2024-03-30T01:42:24.291578+0000","last_active":"2024-03-30T01:42:25.812681+0000","last_peered":"2024-03-30T01:42:25.812681+0000","last_clean":"2024-03-30T01:42:25.812681+0000","last_became_active":"2024-03-30T01:42:24.291309+0000","last_became_peered":"2024-03-30T01:42:24.291309+0000","last_unstale":"2024-03-30T01:42:25.812681+0000","last_undegraded":"2024-03-30T01:42:25.812681+0000","last_fullsized":"2024-03-30T01:42:25.812681+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:23.802198+0000","last_clean_scrub_stamp":"2024-03-30T01:42:23.802198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:03:25.829367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-03-30T01:42:24.297193+0000","last_change":"2024-03-30T01:42:10.288258+0000","last_active":"2024-03-30T01:42:24.297193+0000","last_peered":"2024-03-30T01:42:24.297193+0000","last_clean":"2024-03-30T01:42:24.297193+0000","last_became_active":"2024-03-30T01:42:10.287993+0000","last_became_peered":"2024-03-30T01:42:10.287993+0000","last_unstale":"2024-03-30T01:42:24.297193+0000","last_undegraded":"2024-03-30T01:42:24.297193+0000","last_fullsized":"2024-03-30T01:42:24.297193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:42:09.804681+0000","last_clean_scrub_stamp":"2024-03-30T01:42:09.804681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:48:49.644172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1118208,"data_stored":817623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46600000000000003}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.623},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.41499999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.2009999999999996,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.442}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.63600000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":225220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.501}]},{"osd":1,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.3250000000000002,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.45700000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.38600000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.40100000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56399999999999995}]},{"osd":7,"last update":"Sat Mar 30 01:41:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":224236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":15.038,"5min":15.038,"15min":15.038},"last":0.46600000000000003}]},{"osd":2,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":11.858000000000001,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.42899999999999999}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.372},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1289999999999996,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":11.99,"5min":11.99,"15min":11.99},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.624}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":15.006,"5min":15.006,"15min":15.006},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":11.926,"5min":11.926,"15min":11.926},"last":0.71499999999999997}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":14.974,"5min":14.974,"15min":14.974},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.960000000000001,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":14.923999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.66500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1101824,"data_stored":813475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.312},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.946999999999999,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.436}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":12.983000000000001,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.2279999999999998,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.5060000000000002,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.61199999999999999}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.3869999999999996,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":5.468,"5min":5.468,"15min":5.468},"last":0.57099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":225620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":12.478999999999999,"5min":12.478999999999999,"15min":12.478999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":8.6229999999999993,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.40799999999999997}]},{"osd":1,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":12.425000000000001,"5min":12.425000000000001,"15min":12.425000000000001},"last":0.32000000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.434},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":13.217000000000001,"5min":13.217000000000001,"15min":13.217000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":8.7400000000000002,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":8.5640000000000001,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":8.7759999999999998,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":8.5190000000000001,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.66800000000000004}]},{"osd":7,"last update":"Sat Mar 30 01:41:47 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.621},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":8.5939999999999994,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":0.68300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":228681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.379},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.34399999999999997}]},{"osd":1,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.30499999999999999}]},{"osd":2,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.44800000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.64600000000000002}]},{"osd":5,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.751}]},{"osd":6,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.73599999999999999}]},{"osd":7,"last update":"Sat Mar 30 01:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.77300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":225172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.58899999999999997}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":9.3970000000000002,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.45100000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":5.5289999999999999,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":9.452,"5min":9.452,"15min":9.452},"last":0.621}]},{"osd":5,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.54900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":224098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":6.6760000000000002,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":6.7480000000000002,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.7709999999999999,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.8410000000000002,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.747}]},{"osd":3,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64400000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.629},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.76500000000000001}]},{"osd":7,"last update":"Sat Mar 30 01:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:42:33.710 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T01:42:33.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mon 'allow r' osd 'allow rw pool=cephfs2_data' mds allow 2024-03-30T01:42:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.885+0000 7f51445c4700 1 Processor -- start 2024-03-30T01:42:33.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.885+0000 7f51445c4700 1 -- start start 2024-03-30T01:42:33.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f51445c4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c14fb40 0x7f513c0c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f51445c4700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f513c0a7600 0x7f513c152f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f51445c4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c0c4620 con 0x7f513c14ff20 2024-03-30T01:42:33.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f51445c4700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513c152570 con 0x7f513c0a7600 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f51445c4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c152ad0 con 0x7f513c14fb40 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f5141b5f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f513c0a7600 0x7f513c152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f5141b5f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f513c0a7600 0x7f513c152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44758/0 (socket says 172.21.15.31:44758) 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f5141b5f700 1 -- 172.21.15.31:0/2148298801 learned_addr learned my addr 172.21.15.31:0/2148298801 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f5142360700 1 --2- 172.21.15.31:0/2148298801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c14fb40 0x7f513c0c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f5141b5f700 1 -- 172.21.15.31:0/2148298801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c14fb40 msgr2=0x7f513c0c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f5141b5f700 1 --2- 172.21.15.31:0/2148298801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c14fb40 0x7f513c0c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f5141b5f700 1 -- 172.21.15.31:0/2148298801 >> v1:172.21.15.59:6789/0 conn(0x7f513c14ff20 legacy=0x7f513c0c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f5141b5f700 1 -- 172.21.15.31:0/2148298801 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c0c2ca0 con 0x7f513c0a7600 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.886+0000 7f5141b5f700 1 --2- 172.21.15.31:0/2148298801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c0a7600 0x7f513c152f60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f512c0098f0 tx=0x7f512c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be24fe3a64cc3ca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:33.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f514135e700 1 -- 172.21.15.31:0/2148298801 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c004230 con 0x7f513c0a7600 2024-03-30T01:42:33.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f514135e700 1 -- 172.21.15.31:0/2148298801 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c004390 con 0x7f513c0a7600 2024-03-30T01:42:33.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f514135e700 1 -- 172.21.15.31:0/2148298801 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0036a0 con 0x7f513c0a7600 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f51445c4700 1 -- 172.21.15.31:0/2148298801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c0a7600 msgr2=0x7f513c152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f51445c4700 1 --2- 172.21.15.31:0/2148298801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c0a7600 0x7f513c152f60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f512c0098f0 tx=0x7f512c009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f51445c4700 1 -- 172.21.15.31:0/2148298801 shutdown_connections 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f51445c4700 1 --2- 172.21.15.31:0/2148298801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c14fb40 0x7f513c0c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f51445c4700 1 --2- 172.21.15.31:0/2148298801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c0a7600 0x7f513c152f60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f51445c4700 1 -- 172.21.15.31:0/2148298801 >> 172.21.15.31:0/2148298801 conn(0x7f513c0c7d60 msgr2=0x7f513c0c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f51445c4700 1 -- 172.21.15.31:0/2148298801 shutdown_connections 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.887+0000 7f51445c4700 1 -- 172.21.15.31:0/2148298801 wait complete. 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.889+0000 7f51445c4700 1 Processor -- start 2024-03-30T01:42:33.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.889+0000 7f51445c4700 1 -- start start 2024-03-30T01:42:33.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f51445c4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c0a7600 0x7f513c1eb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f51445c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c14fb40 0x7f513c0bd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f51445c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f513c14ff20 0x7f513c150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f51445c4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1eba80 con 0x7f513c14fb40 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f51445c4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0bd3d0 con 0x7f513c14ff20 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f51445c4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0bdda0 con 0x7f513c0a7600 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f5141b5f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c14fb40 0x7f513c0bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f5141b5f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c14fb40 0x7f513c0bd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44774/0 (socket says 172.21.15.31:44774) 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f5141b5f700 1 -- 172.21.15.31:0/758355760 learned_addr learned my addr 172.21.15.31:0/758355760 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f5142360700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c0a7600 0x7f513c1eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.890+0000 7f5142b61700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f513c14ff20 0x7f513c150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f5141b5f700 1 -- 172.21.15.31:0/758355760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c0a7600 msgr2=0x7f513c1eb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f5141b5f700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c0a7600 0x7f513c1eb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f5141b5f700 1 -- 172.21.15.31:0/758355760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f513c14ff20 msgr2=0x7f513c150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f5141b5f700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f513c14ff20 0x7f513c150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f5141b5f700 1 -- 172.21.15.31:0/758355760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c009550 con 0x7f513c14fb40 2024-03-30T01:42:33.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f5141b5f700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c14fb40 0x7f513c0bd860 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f512c009130 tx=0x7f512c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:33.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f5132ffd700 1 -- 172.21.15.31:0/758355760 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c003ae0 con 0x7f513c14fb40 2024-03-30T01:42:33.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f5132ffd700 1 -- 172.21.15.31:0/758355760 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c01f070 con 0x7f513c14fb40 2024-03-30T01:42:33.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c0be800 con 0x7f513c14fb40 2024-03-30T01:42:33.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f5132ffd700 1 -- 172.21.15.31:0/758355760 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c003e80 con 0x7f513c14fb40 2024-03-30T01:42:33.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.891+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c0bcd80 con 0x7f513c14fb40 2024-03-30T01:42:33.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.892+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c0c2190 con 0x7f513c14fb40 2024-03-30T01:42:33.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.892+0000 7f5132ffd700 1 -- 172.21.15.31:0/758355760 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f512c021030 con 0x7f513c14fb40 2024-03-30T01:42:33.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.893+0000 7f5132ffd700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512006ac90 0x7f512006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:33.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.893+0000 7f5132ffd700 1 -- 172.21.15.31:0/758355760 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f512c08ab50 con 0x7f513c14fb40 2024-03-30T01:42:33.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.893+0000 7f5142360700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512006ac90 0x7f512006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:33.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.893+0000 7f5142360700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512006ac90 0x7f512006d150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5138006b20 tx=0x7f5138008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:33.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:33.898+0000 7f5132ffd700 1 -- 172.21.15.31:0/758355760 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f512c055920 con 0x7f513c14fb40 2024-03-30T01:42:34.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.206+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]} v 0) v1 -- 0x7f513c0c4620 con 0x7f513c14fb40 2024-03-30T01:42:34.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.212+0000 7f5132ffd700 1 -- 172.21.15.31:0/758355760 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]}]=0 updated caps for client.1 v14) v1 ==== 185+0+0 (secure 0 0 0) 0x7f513c0c4620 con 0x7f513c14fb40 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512006ac90 msgr2=0x7f512006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512006ac90 0x7f512006d150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5138006b20 tx=0x7f5138008040 comp rx=0 tx=0).stop 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c14fb40 msgr2=0x7f513c0bd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c14fb40 0x7f513c0bd860 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f512c009130 tx=0x7f512c004750 comp rx=0 tx=0).stop 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 shutdown_connections 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f512006ac90 0x7f512006d150 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f513c14ff20 0x7f513c150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f513c0a7600 0x7f513c1eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 --2- 172.21.15.31:0/758355760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f513c14fb40 0x7f513c0bd860 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 >> 172.21.15.31:0/758355760 conn(0x7f513c0c7d60 msgr2=0x7f513c0c47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:34.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 shutdown_connections 2024-03-30T01:42:34.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.214+0000 7f51445c4700 1 -- 172.21.15.31:0/758355760 wait complete. 2024-03-30T01:42:34.221 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.1 2024-03-30T01:42:34.233 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:42:34.233 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:34.256 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:42:34.257 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:34.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:34.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.456+0000 7f70f586a700 1 Processor -- start 2024-03-30T01:42:34.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.456+0000 7f70f586a700 1 -- start start 2024-03-30T01:42:34.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.456+0000 7f70f586a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 0x7f70f0155060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.456+0000 7f70f586a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70f006e7e0 0x7f70f006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.456+0000 7f70f586a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f00c3610 con 0x7f70f0071880 2024-03-30T01:42:34.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.456+0000 7f70f586a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f01555a0 con 0x7f70f006e7e0 2024-03-30T01:42:34.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.456+0000 7f70f586a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0157da0 con 0x7f70f014e680 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.457+0000 7f70ef7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70f006e7e0 0x7f70f006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.457+0000 7f70eeffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 0x7f70f0155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.457+0000 7f70ee7fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f70f0071880 0x7f70f006a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:48456/0 (socket says 172.21.15.31:48456) 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.457+0000 7f70ee7fc700 1 -- 172.21.15.31:0/3589105885 learned_addr learned my addr 172.21.15.31:0/3589105885 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.457+0000 7f70eeffd700 1 -- 172.21.15.31:0/3589105885 >> v2:172.21.15.59:3300/0 conn(0x7f70f006e7e0 msgr2=0x7f70f006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.457+0000 7f70eeffd700 1 --2- 172.21.15.31:0/3589105885 >> v2:172.21.15.59:3300/0 conn(0x7f70f006e7e0 0x7f70f006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.457+0000 7f70eeffd700 1 -- 172.21.15.31:0/3589105885 >> v1:172.21.15.59:6789/0 conn(0x7f70f0071880 legacy=0x7f70f006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.457+0000 7f70eeffd700 1 -- 172.21.15.31:0/3589105885 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f0158080 con 0x7f70f014e680 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.457+0000 7f70ef7fe700 1 --2- 172.21.15.31:0/3589105885 >> v2:172.21.15.59:3300/0 conn(0x7f70f006e7e0 0x7f70f006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70eeffd700 1 --2- 172.21.15.31:0/3589105885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 0x7f70f0155060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009840 tx=0x7f70e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9890fc4fcb08472a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70edffb700 1 -- 172.21.15.31:0/3589105885 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0011830 con 0x7f70f014e680 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70edffb700 1 -- 172.21.15.31:0/3589105885 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e0011990 con 0x7f70f014e680 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70edffb700 1 -- 172.21.15.31:0/3589105885 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0011d20 con 0x7f70f014e680 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70f586a700 1 -- 172.21.15.31:0/3589105885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 msgr2=0x7f70f0155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70f586a700 1 --2- 172.21.15.31:0/3589105885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 0x7f70f0155060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009840 tx=0x7f70e0009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70f586a700 1 -- 172.21.15.31:0/3589105885 shutdown_connections 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70f586a700 1 --2- 172.21.15.31:0/3589105885 >> v2:172.21.15.59:3300/0 conn(0x7f70f006e7e0 0x7f70f006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70f586a700 1 --2- 172.21.15.31:0/3589105885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 0x7f70f0155060 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70f586a700 1 -- 172.21.15.31:0/3589105885 >> 172.21.15.31:0/3589105885 conn(0x7f70f006c230 msgr2=0x7f70f00c6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70f586a700 1 -- 172.21.15.31:0/3589105885 shutdown_connections 2024-03-30T01:42:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.458+0000 7f70f586a700 1 -- 172.21.15.31:0/3589105885 wait complete. 2024-03-30T01:42:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.460+0000 7f70f586a700 1 Processor -- start 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.460+0000 7f70f586a700 1 -- start start 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.460+0000 7f70f586a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70f006e7e0 0x7f70f00ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.460+0000 7f70f586a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f0071880 0x7f70f01ea260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.460+0000 7f70f586a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 0x7f70f0154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.460+0000 7f70f586a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0151040 con 0x7f70f006e7e0 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.460+0000 7f70f586a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f01ea7a0 con 0x7f70f0071880 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70f586a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00abbc0 con 0x7f70f014e680 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70eeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70f006e7e0 0x7f70f00ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70eeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70f006e7e0 0x7f70f00ab4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44782/0 (socket says 172.21.15.31:44782) 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70eeffd700 1 -- 172.21.15.31:0/1203129890 learned_addr learned my addr 172.21.15.31:0/1203129890 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70ef7fe700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 0x7f70f0154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70ee7fc700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f0071880 0x7f70f01ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70eeffd700 1 -- 172.21.15.31:0/1203129890 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 msgr2=0x7f70f0154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70eeffd700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 0x7f70f0154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70eeffd700 1 -- 172.21.15.31:0/1203129890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f0071880 msgr2=0x7f70f01ea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70eeffd700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f0071880 0x7f70f01ea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70eeffd700 1 -- 172.21.15.31:0/1203129890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e8009a70 con 0x7f70f006e7e0 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70eeffd700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70f006e7e0 0x7f70f00ab4c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f70e00090b0 tx=0x7f70e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70d7fff700 1 -- 172.21.15.31:0/1203129890 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e00036a0 con 0x7f70f006e7e0 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.461+0000 7f70d7fff700 1 -- 172.21.15.31:0/1203129890 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e001f070 con 0x7f70f006e7e0 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.462+0000 7f70d7fff700 1 -- 172.21.15.31:0/1203129890 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0003a00 con 0x7f70f006e7e0 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.462+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e00094a0 con 0x7f70f006e7e0 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.462+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f0151250 con 0x7f70f006e7e0 2024-03-30T01:42:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.462+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f0067bb0 con 0x7f70f006e7e0 2024-03-30T01:42:34.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.463+0000 7f70d7fff700 1 -- 172.21.15.31:0/1203129890 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f70e0003b60 con 0x7f70f006e7e0 2024-03-30T01:42:34.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.463+0000 7f70d7fff700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70dc06ac90 0x7f70dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.464+0000 7f70d7fff700 1 -- 172.21.15.31:0/1203129890 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f70e0021070 con 0x7f70f006e7e0 2024-03-30T01:42:34.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.464+0000 7f70ee7fc700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70dc06ac90 0x7f70dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.464+0000 7f70ee7fc700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70dc06ac90 0x7f70dc06d150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f70f0153ea0 tx=0x7f70e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:34.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.468+0000 7f70d7fff700 1 -- 172.21.15.31:0/1203129890 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f70e0055df0 con 0x7f70f006e7e0 2024-03-30T01:42:34.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.683+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70f00ac0e0 con 0x7f70f006e7e0 2024-03-30T01:42:34.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.684+0000 7f70d7fff700 1 -- 172.21.15.31:0/1203129890 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+19974 (secure 0 0 0) 0x7f70e0015090 con 0x7f70f006e7e0 2024-03-30T01:42:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:34.690 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":43,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:25.270018+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:34.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70dc06ac90 msgr2=0x7f70dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70dc06ac90 0x7f70dc06d150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f70f0153ea0 tx=0x7f70e4009040 comp rx=0 tx=0).stop 2024-03-30T01:42:34.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70f006e7e0 msgr2=0x7f70f00ab4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70f006e7e0 0x7f70f00ab4c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f70e00090b0 tx=0x7f70e0004750 comp rx=0 tx=0).stop 2024-03-30T01:42:34.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 shutdown_connections 2024-03-30T01:42:34.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70f006e7e0 0x7f70f00ab4c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70dc06ac90 0x7f70dc06d150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f014e680 0x7f70f0154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 --2- 172.21.15.31:0/1203129890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f0071880 0x7f70f01ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 >> 172.21.15.31:0/1203129890 conn(0x7f70f006c230 msgr2=0x7f70f00c1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:34.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 shutdown_connections 2024-03-30T01:42:34.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.687+0000 7f70f586a700 1 -- 172.21.15.31:0/1203129890 wait complete. 2024-03-30T01:42:34.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:42:07.881738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:42:09.236870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-30T01:42:21.874769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-30T01:42:23.236623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:42:34.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:42:34.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.842+0000 7f2e7bd54700 1 Processor -- start 2024-03-30T01:42:34.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.842+0000 7f2e7bd54700 1 -- start start 2024-03-30T01:42:34.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e7bd54700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e7406d510 0x7f2e740c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e7bd54700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e74159b00 con 0x7f2e740c4a60 2024-03-30T01:42:34.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e7bd54700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e7415c5e0 con 0x7f2e740c3aa0 2024-03-30T01:42:34.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e7bd54700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e740c45d0 con 0x7f2e7406d510 2024-03-30T01:42:34.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e79af0700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2e740c4a60 0x7f2e7415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47430/0 (socket says 172.21.15.31:47430) 2024-03-30T01:42:34.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e79af0700 1 -- 172.21.15.31:0/1799124978 learned_addr learned my addr 172.21.15.31:0/1799124978 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:34.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e7a2f1700 1 --2- 172.21.15.31:0/1799124978 >> v2:172.21.15.59:3300/0 conn(0x7f2e7406d510 0x7f2e740c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e7a2f1700 1 -- 172.21.15.31:0/1799124978 >> v1:172.21.15.59:6789/0 conn(0x7f2e740c3aa0 legacy=0x7f2e74158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e7a2f1700 1 -- 172.21.15.31:0/1799124978 >> v1:172.21.15.31:6789/0 conn(0x7f2e740c4a60 legacy=0x7f2e7415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.843+0000 7f2e7a2f1700 1 -- 172.21.15.31:0/1799124978 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740c4010 con 0x7f2e7406d510 2024-03-30T01:42:34.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.844+0000 7f2e7a2f1700 1 --2- 172.21.15.31:0/1799124978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 0x7f2e740c6c70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009d90 tx=0x7f2e6c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f408bf7d84885f3b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:34.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.845+0000 7f2e78aee700 1 -- 172.21.15.31:0/1799124978 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00e070 con 0x7f2e7406d510 2024-03-30T01:42:34.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.846+0000 7f2e78aee700 1 -- 172.21.15.31:0/1799124978 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c00bbd0 con 0x7f2e7406d510 2024-03-30T01:42:34.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.846+0000 7f2e78aee700 1 -- 172.21.15.31:0/1799124978 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c012430 con 0x7f2e7406d510 2024-03-30T01:42:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.846+0000 7f2e7bd54700 1 -- 172.21.15.31:0/1799124978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 msgr2=0x7f2e740c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.846+0000 7f2e7bd54700 1 --2- 172.21.15.31:0/1799124978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 0x7f2e740c6c70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009d90 tx=0x7f2e6c00b330 comp rx=0 tx=0).stop 2024-03-30T01:42:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.846+0000 7f2e7bd54700 1 -- 172.21.15.31:0/1799124978 shutdown_connections 2024-03-30T01:42:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.846+0000 7f2e7bd54700 1 --2- 172.21.15.31:0/1799124978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 0x7f2e740c6c70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.846+0000 7f2e7bd54700 1 -- 172.21.15.31:0/1799124978 >> 172.21.15.31:0/1799124978 conn(0x7f2e740c7d60 msgr2=0x7f2e740c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.846+0000 7f2e7bd54700 1 -- 172.21.15.31:0/1799124978 shutdown_connections 2024-03-30T01:42:34.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.846+0000 7f2e7bd54700 1 -- 172.21.15.31:0/1799124978 wait complete. 2024-03-30T01:42:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.848+0000 7f2e7bd54700 1 Processor -- start 2024-03-30T01:42:34.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.848+0000 7f2e7bd54700 1 -- start start 2024-03-30T01:42:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.848+0000 7f2e7bd54700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 0x7f2e741eea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.848+0000 7f2e7bd54700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e740c3aa0 0x7f2e740a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.848+0000 7f2e7bd54700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2e740c4a60 0x7f2e7406ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.848+0000 7f2e7bd54700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e741eef80 con 0x7f2e740c3aa0 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.848+0000 7f2e7bd54700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406e770 con 0x7f2e7406d510 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.848+0000 7f2e7bd54700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740a7fc0 con 0x7f2e740c4a60 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e792ef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e740c3aa0 0x7f2e740a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e792ef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e740c3aa0 0x7f2e740a7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44784/0 (socket says 172.21.15.31:44784) 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e792ef700 1 -- 172.21.15.31:0/771297804 learned_addr learned my addr 172.21.15.31:0/771297804 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e79af0700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 0x7f2e741eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e7a2f1700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2e740c4a60 0x7f2e7406ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e792ef700 1 -- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2e740c4a60 msgr2=0x7f2e7406ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e792ef700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2e740c4a60 0x7f2e7406ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e792ef700 1 -- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 msgr2=0x7f2e741eea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e792ef700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 0x7f2e741eea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e792ef700 1 -- 172.21.15.31:0/771297804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c009a70 con 0x7f2e740c3aa0 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e7a2f1700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2e740c4a60 0x7f2e7406ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e79af0700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 0x7f2e741eea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e792ef700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e740c3aa0 0x7f2e740a7a80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400c730 tx=0x7f2e6400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e6a7fc700 1 -- 172.21.15.31:0/771297804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e640052d0 con 0x7f2e740c3aa0 2024-03-30T01:42:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e6a7fc700 1 -- 172.21.15.31:0/771297804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e64005430 con 0x7f2e740c3aa0 2024-03-30T01:42:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7406fe50 con 0x7f2e740c3aa0 2024-03-30T01:42:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e6a7fc700 1 -- 172.21.15.31:0/771297804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e640038a0 con 0x7f2e740c3aa0 2024-03-30T01:42:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.849+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7406fb70 con 0x7f2e740c3aa0 2024-03-30T01:42:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.850+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e740747d0 con 0x7f2e740c3aa0 2024-03-30T01:42:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.851+0000 7f2e6a7fc700 1 -- 172.21.15.31:0/771297804 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2e64003a00 con 0x7f2e740c3aa0 2024-03-30T01:42:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.851+0000 7f2e6a7fc700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2e5806ac90 0x7f2e5806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.851+0000 7f2e6a7fc700 1 -- 172.21.15.31:0/771297804 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f2e64088c00 con 0x7f2e740c3aa0 2024-03-30T01:42:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.851+0000 7f2e79af0700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2e5806ac90 0x7f2e5806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.852+0000 7f2e79af0700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2e5806ac90 0x7f2e5806d150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2e70007fd0 tx=0x7f2e70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:34.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:34.856+0000 7f2e6a7fc700 1 -- 172.21.15.31:0/771297804 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2e6408f080 con 0x7f2e740c3aa0 2024-03-30T01:42:35.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.069+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e7406e8b0 con 0x7f2e740c3aa0 2024-03-30T01:42:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.069+0000 7f2e6a7fc700 1 -- 172.21.15.31:0/771297804 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e640170a0 con 0x7f2e740c3aa0 2024-03-30T01:42:35.075 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:42:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2e5806ac90 msgr2=0x7f2e5806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2e5806ac90 0x7f2e5806d150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2e70007fd0 tx=0x7f2e70009040 comp rx=0 tx=0).stop 2024-03-30T01:42:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e740c3aa0 msgr2=0x7f2e740a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e740c3aa0 0x7f2e740a7a80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400c730 tx=0x7f2e6400caf0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 shutdown_connections 2024-03-30T01:42:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f2e5806ac90 0x7f2e5806d150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2e740c4a60 0x7f2e7406ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e7406d510 0x7f2e741eea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 --2- 172.21.15.31:0/771297804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e740c3aa0 0x7f2e740a7a80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 >> 172.21.15.31:0/771297804 conn(0x7f2e740c7d60 msgr2=0x7f2e740ac420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 shutdown_connections 2024-03-30T01:42:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.072+0000 7f2e7bd54700 1 -- 172.21.15.31:0/771297804 wait complete. 2024-03-30T01:42:35.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:42:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.225+0000 7f88ce829700 1 Processor -- start 2024-03-30T01:42:35.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.225+0000 7f88ce829700 1 -- start start 2024-03-30T01:42:35.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88ce829700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f88c8072710 0x7f88c80c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88ce829700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 0x7f88c8155150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88ce829700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c8155690 con 0x7f88c8072af0 2024-03-30T01:42:35.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88ce829700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c8158c50 con 0x7f88c8072710 2024-03-30T01:42:35.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88ce829700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c806add0 con 0x7f88c806ba90 2024-03-30T01:42:35.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88c7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f88c8072710 0x7f88c80c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88ccdc6700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f88c8072af0 0x7f88c8158540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47436/0 (socket says 172.21.15.31:47436) 2024-03-30T01:42:35.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88ccdc6700 1 -- 172.21.15.31:0/3316237843 learned_addr learned my addr 172.21.15.31:0/3316237843 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:35.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88c77fe700 1 --2- 172.21.15.31:0/3316237843 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 0x7f88c8155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88c7fff700 1 -- 172.21.15.31:0/3316237843 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 msgr2=0x7f88c8155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:35.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88c7fff700 1 --2- 172.21.15.31:0/3316237843 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 0x7f88c8155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88c7fff700 1 -- 172.21.15.31:0/3316237843 >> v1:172.21.15.31:6789/0 conn(0x7f88c8072af0 legacy=0x7f88c8158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.227+0000 7f88c7fff700 1 -- 172.21.15.31:0/3316237843 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c806a810 con 0x7f88c8072710 2024-03-30T01:42:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.226+0000 7f88c6ffd700 1 -- 172.21.15.31:0/3316237843 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563737453 0 0) 0x7f88c8155690 con 0x7f88c8072af0 2024-03-30T01:42:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.227+0000 7f88c77fe700 1 --2- 172.21.15.31:0/3316237843 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 0x7f88c8155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.227+0000 7f88c7fff700 1 --2- 172.21.15.31:0/3316237843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072710 0x7f88c80c0fc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009840 tx=0x7f88b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba9a0000bea0717 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.227+0000 7f88c6ffd700 1 -- 172.21.15.31:0/3316237843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8011830 con 0x7f88c8072710 2024-03-30T01:42:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.227+0000 7f88c6ffd700 1 -- 172.21.15.31:0/3316237843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b8011990 con 0x7f88c8072710 2024-03-30T01:42:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.227+0000 7f88c6ffd700 1 -- 172.21.15.31:0/3316237843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8011d20 con 0x7f88c8072710 2024-03-30T01:42:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.227+0000 7f88ce829700 1 -- 172.21.15.31:0/3316237843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072710 msgr2=0x7f88c80c0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.227+0000 7f88ce829700 1 --2- 172.21.15.31:0/3316237843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072710 0x7f88c80c0fc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009840 tx=0x7f88b8009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.228+0000 7f88ce829700 1 -- 172.21.15.31:0/3316237843 shutdown_connections 2024-03-30T01:42:35.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.228+0000 7f88ce829700 1 --2- 172.21.15.31:0/3316237843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072710 0x7f88c80c0fc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.228+0000 7f88ce829700 1 --2- 172.21.15.31:0/3316237843 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 0x7f88c8155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.228+0000 7f88ce829700 1 -- 172.21.15.31:0/3316237843 >> 172.21.15.31:0/3316237843 conn(0x7f88c80c65c0 msgr2=0x7f88c80c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.228+0000 7f88ce829700 1 -- 172.21.15.31:0/3316237843 shutdown_connections 2024-03-30T01:42:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.228+0000 7f88ce829700 1 -- 172.21.15.31:0/3316237843 wait complete. 2024-03-30T01:42:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.229+0000 7f88ce829700 1 Processor -- start 2024-03-30T01:42:35.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.229+0000 7f88ce829700 1 -- start start 2024-03-30T01:42:35.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ce829700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 0x7f88c80c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ce829700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c8072710 0x7f88c80ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ce829700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072af0 0x7f88c81ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ce829700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c81ee530 con 0x7f88c8072af0 2024-03-30T01:42:35.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ce829700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80bec70 con 0x7f88c8072710 2024-03-30T01:42:35.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ce829700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c81ee190 con 0x7f88c806ba90 2024-03-30T01:42:35.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ccdc6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072af0 0x7f88c81ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ccdc6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072af0 0x7f88c81ee950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44810/0 (socket says 172.21.15.31:44810) 2024-03-30T01:42:35.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ccdc6700 1 -- 172.21.15.31:0/2795476282 learned_addr learned my addr 172.21.15.31:0/2795476282 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:35.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88c7fff700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 0x7f88c80c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88c77fe700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c8072710 0x7f88c80ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ccdc6700 1 -- 172.21.15.31:0/2795476282 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 msgr2=0x7f88c80c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ccdc6700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 0x7f88c80c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ccdc6700 1 -- 172.21.15.31:0/2795476282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c8072710 msgr2=0x7f88c80ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ccdc6700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c8072710 0x7f88c80ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88ccdc6700 1 -- 172.21.15.31:0/2795476282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80094a0 con 0x7f88c8072af0 2024-03-30T01:42:35.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.230+0000 7f88c77fe700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c8072710 0x7f88c80ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.231+0000 7f88ccdc6700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072af0 0x7f88c81ee950 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f88c000cba0 tx=0x7f88c000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.231+0000 7f88c4ff9700 1 -- 172.21.15.31:0/2795476282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c000aad0 con 0x7f88c8072af0 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.231+0000 7f88c4ff9700 1 -- 172.21.15.31:0/2795476282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c000ac50 con 0x7f88c8072af0 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.231+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c80a93d0 con 0x7f88c8072af0 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.231+0000 7f88c4ff9700 1 -- 172.21.15.31:0/2795476282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c00070d0 con 0x7f88c8072af0 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.231+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c80a9990 con 0x7f88c8072af0 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.232+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c81ee670 con 0x7f88c8072af0 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.232+0000 7f88c4ff9700 1 -- 172.21.15.31:0/2795476282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f88c0007230 con 0x7f88c8072af0 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.233+0000 7f88c4ff9700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88b406ac90 0x7f88b406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.233+0000 7f88c4ff9700 1 -- 172.21.15.31:0/2795476282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f88c0089250 con 0x7f88c8072af0 2024-03-30T01:42:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.233+0000 7f88c7fff700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88b406ac90 0x7f88b406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.233+0000 7f88c7fff700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88b406ac90 0x7f88b406d150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f88b80090b0 tx=0x7f88b801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:35.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.238+0000 7f88c4ff9700 1 -- 172.21.15.31:0/2795476282 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f88c81ee670 con 0x7f88c8072af0 2024-03-30T01:42:35.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.451+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88c81ee670 con 0x7f88c8072af0 2024-03-30T01:42:35.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.451+0000 7f88c4ff9700 1 -- 172.21.15.31:0/2795476282 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88c81ee670 con 0x7f88c8072af0 2024-03-30T01:42:35.457 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:35.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.453+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88b406ac90 msgr2=0x7f88b406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88b406ac90 0x7f88b406d150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f88b80090b0 tx=0x7f88b801c040 comp rx=0 tx=0).stop 2024-03-30T01:42:35.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072af0 msgr2=0x7f88c81ee950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072af0 0x7f88c81ee950 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f88c000cba0 tx=0x7f88c000ceb0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 shutdown_connections 2024-03-30T01:42:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c8072af0 0x7f88c81ee950 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f88b406ac90 0x7f88b406d150 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88c806ba90 0x7f88c80c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 --2- 172.21.15.31:0/2795476282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88c8072710 0x7f88c80ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 >> 172.21.15.31:0/2795476282 conn(0x7f88c80c65c0 msgr2=0x7f88c80c3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 shutdown_connections 2024-03-30T01:42:35.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.454+0000 7f88ce829700 1 -- 172.21.15.31:0/2795476282 wait complete. 2024-03-30T01:42:35.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:42:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.607+0000 7f7fb9b5c700 1 Processor -- start 2024-03-30T01:42:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.607+0000 7f7fb9b5c700 1 -- start start 2024-03-30T01:42:35.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb9b5c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151b60 0x7f7fb4070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb9b5c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7fb40c5d40 0x7f7fb40c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb9b5c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7fb4151f40 0x7f7fb40c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb9b5c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb4155610 con 0x7f7fb40c5d40 2024-03-30T01:42:35.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb9b5c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb41547a0 con 0x7f7fb4151f40 2024-03-30T01:42:35.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb9b5c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb4154990 con 0x7f7fb4151b60 2024-03-30T01:42:35.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7fb40c5d40 0x7f7fb40c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb2ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7fb40c5d40 0x7f7fb40c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34814/0 (socket says 172.21.15.31:34814) 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb2ffd700 1 -- 172.21.15.31:0/1673578986 learned_addr learned my addr 172.21.15.31:0/1673578986 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb3fff700 1 --2- 172.21.15.31:0/1673578986 >> v2:172.21.15.59:3300/0 conn(0x7f7fb4151f40 0x7f7fb40c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb37fe700 1 --2- 172.21.15.31:0/1673578986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151b60 0x7f7fb4070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb2ffd700 1 -- 172.21.15.31:0/1673578986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151b60 msgr2=0x7f7fb4070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb2ffd700 1 --2- 172.21.15.31:0/1673578986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151b60 0x7f7fb4070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb2ffd700 1 -- 172.21.15.31:0/1673578986 >> v2:172.21.15.59:3300/0 conn(0x7f7fb4151f40 msgr2=0x7f7fb40c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb2ffd700 1 --2- 172.21.15.31:0/1673578986 >> v2:172.21.15.59:3300/0 conn(0x7f7fb4151f40 0x7f7fb40c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.608+0000 7f7fb2ffd700 1 -- 172.21.15.31:0/1673578986 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4154c70 con 0x7f7fb40c5d40 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb2ffd700 1 --2- 172.21.15.31:0/1673578986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 0x7f7fb40c2760 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40a7a10 tx=0x7f7fa8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dd548d403f91e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb27fc700 1 -- 172.21.15.31:0/1673578986 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8011830 con 0x7f7fb40c5d40 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb27fc700 1 -- 172.21.15.31:0/1673578986 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa8011990 con 0x7f7fb40c5d40 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb27fc700 1 -- 172.21.15.31:0/1673578986 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8011d20 con 0x7f7fb40c5d40 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/1673578986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 msgr2=0x7f7fb40c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/1673578986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 0x7f7fb40c2760 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40a7a10 tx=0x7f7fa8009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/1673578986 shutdown_connections 2024-03-30T01:42:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/1673578986 >> v2:172.21.15.59:3300/0 conn(0x7f7fb4151f40 0x7f7fb40c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/1673578986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151b60 0x7f7fb4070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/1673578986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 0x7f7fb40c2760 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.609+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/1673578986 >> 172.21.15.31:0/1673578986 conn(0x7f7fb40c7d60 msgr2=0x7f7fb40c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.610+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/1673578986 shutdown_connections 2024-03-30T01:42:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.610+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/1673578986 wait complete. 2024-03-30T01:42:35.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.611+0000 7f7fb9b5c700 1 Processor -- start 2024-03-30T01:42:35.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.611+0000 7f7fb9b5c700 1 -- start start 2024-03-30T01:42:35.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb9b5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 0x7f7fb4153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb9b5c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb4151b60 0x7f7fb4149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb9b5c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151f40 0x7f7fb414a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb9b5c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb41ea380 con 0x7f7fb40c5d40 2024-03-30T01:42:35.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb9b5c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb41e99e0 con 0x7f7fb4151b60 2024-03-30T01:42:35.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb9b5c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb41e9cc0 con 0x7f7fb4151f40 2024-03-30T01:42:35.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 0x7f7fb4153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 0x7f7fb4153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34828/0 (socket says 172.21.15.31:34828) 2024-03-30T01:42:35.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb37fe700 1 -- 172.21.15.31:0/3438738014 learned_addr learned my addr 172.21.15.31:0/3438738014 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:35.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb2ffd700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb4151b60 0x7f7fb4149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb3fff700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151f40 0x7f7fb414a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb37fe700 1 -- 172.21.15.31:0/3438738014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151f40 msgr2=0x7f7fb414a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb37fe700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151f40 0x7f7fb414a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb37fe700 1 -- 172.21.15.31:0/3438738014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb4151b60 msgr2=0x7f7fb4149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb37fe700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb4151b60 0x7f7fb4149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb37fe700 1 -- 172.21.15.31:0/3438738014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8009550 con 0x7f7fb40c5d40 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.612+0000 7f7fb37fe700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 0x7f7fb4153660 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400e9b0 tx=0x7f7fa400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.613+0000 7f7f9bfff700 1 -- 172.21.15.31:0/3438738014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4009710 con 0x7f7fb40c5d40 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.613+0000 7f7f9bfff700 1 -- 172.21.15.31:0/3438738014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa400c850 con 0x7f7fb40c5d40 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.613+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb41e9fa0 con 0x7f7fb40c5d40 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.613+0000 7f7f9bfff700 1 -- 172.21.15.31:0/3438738014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400cbc0 con 0x7f7fb40c5d40 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.613+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb414d660 con 0x7f7fb40c5d40 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.614+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb41ea5f0 con 0x7f7fb40c5d40 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.614+0000 7f7f9bfff700 1 -- 172.21.15.31:0/3438738014 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7fa400cd20 con 0x7f7fb40c5d40 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.615+0000 7f7f9bfff700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fa006ac90 0x7f7fa006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.615+0000 7f7f9bfff700 1 -- 172.21.15.31:0/3438738014 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f7fa40898a0 con 0x7f7fb40c5d40 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.615+0000 7f7fb2ffd700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fa006ac90 0x7f7fa006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.615+0000 7f7fb2ffd700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fa006ac90 0x7f7fa006d150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8014040 tx=0x7f7fa8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:35.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.619+0000 7f7f9bfff700 1 -- 172.21.15.31:0/3438738014 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7fa4054670 con 0x7f7fb40c5d40 2024-03-30T01:42:35.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.831+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fb40747d0 con 0x7f7fb40c5d40 2024-03-30T01:42:35.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.831+0000 7f7f9bfff700 1 -- 172.21.15.31:0/3438738014 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7fa4057c90 con 0x7f7fb40c5d40 2024-03-30T01:42:35.837 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:35.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fa006ac90 msgr2=0x7f7fa006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fa006ac90 0x7f7fa006d150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8014040 tx=0x7f7fa8008030 comp rx=0 tx=0).stop 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 msgr2=0x7f7fb4153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 0x7f7fb4153660 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400e9b0 tx=0x7f7fa400ed70 comp rx=0 tx=0).stop 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 shutdown_connections 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb40c5d40 0x7f7fb4153660 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fa006ac90 0x7f7fa006d150 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fb4151f40 0x7f7fb414a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 --2- 172.21.15.31:0/3438738014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb4151b60 0x7f7fb4149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 >> 172.21.15.31:0/3438738014 conn(0x7f7fb40c7d60 msgr2=0x7f7fb40c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 shutdown_connections 2024-03-30T01:42:35.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.834+0000 7f7fb9b5c700 1 -- 172.21.15.31:0/3438738014 wait complete. 2024-03-30T01:42:35.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-30T01:42:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.987+0000 7f8fb0c96700 1 Processor -- start 2024-03-30T01:42:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.987+0000 7f8fb0c96700 1 -- start start 2024-03-30T01:42:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.987+0000 7f8fb0c96700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c46c0 0x7f8fac070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.987+0000 7f8fb0c96700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8fac0c4aa0 0x7f8fac06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.987+0000 7f8fb0c96700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fac0c37c0 con 0x7f8fac0a6e90 2024-03-30T01:42:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.987+0000 7f8fb0c96700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fac070c60 con 0x7f8fac0c4aa0 2024-03-30T01:42:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.987+0000 7f8fb0c96700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0c3090 con 0x7f8fac0c46c0 2024-03-30T01:42:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8fac0c4aa0 0x7f8fac06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8faad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8fac0a6e90 0x7f8fac0c5bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50636/0 (socket says 172.21.15.31:50636) 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8faad9d700 1 -- 172.21.15.31:0/2841500911 learned_addr learned my addr 172.21.15.31:0/2841500911 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8faa59c700 1 --2- 172.21.15.31:0/2841500911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c46c0 0x7f8fac070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa3fff700 1 -- 172.21.15.31:0/2841500911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c46c0 msgr2=0x7f8fac070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa3fff700 1 --2- 172.21.15.31:0/2841500911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c46c0 0x7f8fac070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa3fff700 1 -- 172.21.15.31:0/2841500911 >> v1:172.21.15.31:6789/0 conn(0x7f8fac0a6e90 legacy=0x7f8fac0c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa9d9b700 1 -- 172.21.15.31:0/2841500911 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756067626 0 0) 0x7f8fac0c37c0 con 0x7f8fac0a6e90 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa3fff700 1 -- 172.21.15.31:0/2841500911 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac0c2890 con 0x7f8fac0c4aa0 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8faa59c700 1 --2- 172.21.15.31:0/2841500911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c46c0 0x7f8fac070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa3fff700 1 --2- 172.21.15.31:0/2841500911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c4aa0 0x7f8fac06bd40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009840 tx=0x7f8f94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab50ea46ed48ca71 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa9d9b700 1 -- 172.21.15.31:0/2841500911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94011830 con 0x7f8fac0c4aa0 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa9d9b700 1 -- 172.21.15.31:0/2841500911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f94011990 con 0x7f8fac0c4aa0 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.988+0000 7f8fa9d9b700 1 -- 172.21.15.31:0/2841500911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94011d20 con 0x7f8fac0c4aa0 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.989+0000 7f8fb0c96700 1 -- 172.21.15.31:0/2841500911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c4aa0 msgr2=0x7f8fac06bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.989+0000 7f8fb0c96700 1 --2- 172.21.15.31:0/2841500911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c4aa0 0x7f8fac06bd40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009840 tx=0x7f8f94009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.989+0000 7f8fb0c96700 1 -- 172.21.15.31:0/2841500911 shutdown_connections 2024-03-30T01:42:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.989+0000 7f8fb0c96700 1 --2- 172.21.15.31:0/2841500911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c46c0 0x7f8fac070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.989+0000 7f8fb0c96700 1 --2- 172.21.15.31:0/2841500911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c4aa0 0x7f8fac06bd40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.989+0000 7f8fb0c96700 1 -- 172.21.15.31:0/2841500911 >> 172.21.15.31:0/2841500911 conn(0x7f8fac0c7cf0 msgr2=0x7f8fac0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.989+0000 7f8fb0c96700 1 -- 172.21.15.31:0/2841500911 shutdown_connections 2024-03-30T01:42:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.989+0000 7f8fb0c96700 1 -- 172.21.15.31:0/2841500911 wait complete. 2024-03-30T01:42:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fb0c96700 1 Processor -- start 2024-03-30T01:42:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fb0c96700 1 -- start start 2024-03-30T01:42:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fb0c96700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fac0a6e90 0x7f8fac1de470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fb0c96700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c46c0 0x7f8fac153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fb0c96700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c4aa0 0x7f8fac0abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fb0c96700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1dd4d0 con 0x7f8fac0c46c0 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fb0c96700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0ab7f0 con 0x7f8fac0a6e90 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fb0c96700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1dd6e0 con 0x7f8fac0c4aa0 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fa3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c46c0 0x7f8fac153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fa3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c46c0 0x7f8fac153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34842/0 (socket says 172.21.15.31:34842) 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.991+0000 7f8fa3fff700 1 -- 172.21.15.31:0/3903228090 learned_addr learned my addr 172.21.15.31:0/3903228090 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8faa59c700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fac0a6e90 0x7f8fac1de470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8faad9d700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c4aa0 0x7f8fac0abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fa3fff700 1 -- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c4aa0 msgr2=0x7f8fac0abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fa3fff700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c4aa0 0x7f8fac0abc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fa3fff700 1 -- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fac0a6e90 msgr2=0x7f8fac1de470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fa3fff700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fac0a6e90 0x7f8fac1de470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fa3fff700 1 -- 172.21.15.31:0/3903228090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940094a0 con 0x7f8fac0c46c0 2024-03-30T01:42:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8faad9d700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c4aa0 0x7f8fac0abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8faa59c700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fac0a6e90 0x7f8fac1de470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fa3fff700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c46c0 0x7f8fac153720 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8f94014040 tx=0x7f8f94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fa2ffd700 1 -- 172.21.15.31:0/3903228090 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f940045f0 con 0x7f8fac0c46c0 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fa2ffd700 1 -- 172.21.15.31:0/3903228090 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9401e070 con 0x7f8fac0c46c0 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac1dffa0 con 0x7f8fac0c46c0 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fa2ffd700 1 -- 172.21.15.31:0/3903228090 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94003ae0 con 0x7f8fac0c46c0 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.992+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac1e0280 con 0x7f8fac0c46c0 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.993+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac04ea80 con 0x7f8fac0c46c0 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.994+0000 7f8fa2ffd700 1 -- 172.21.15.31:0/3903228090 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8f94003c40 con 0x7f8fac0c46c0 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.994+0000 7f8fa2ffd700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8f8c06ac90 0x7f8f8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.995+0000 7f8fa2ffd700 1 -- 172.21.15.31:0/3903228090 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f8f9408a8e0 con 0x7f8fac0c46c0 2024-03-30T01:42:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.995+0000 7f8faa59c700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8f8c06ac90 0x7f8f8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.995+0000 7f8faa59c700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8f8c06ac90 0x7f8f8c06d150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0089b0 tx=0x7f8f9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:35.999+0000 7f8fa2ffd700 1 -- 172.21.15.31:0/3903228090 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8f940556b0 con 0x7f8fac0c46c0 2024-03-30T01:42:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.211+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fac1572a0 con 0x7f8fac0c46c0 2024-03-30T01:42:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.212+0000 7f8fa2ffd700 1 -- 172.21.15.31:0/3903228090 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v43) v1 ==== 105+0+11 (secure 0 0 0) 0x7f8f94015020 con 0x7f8fac0c46c0 2024-03-30T01:42:36.218 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8f8c06ac90 msgr2=0x7f8f8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8f8c06ac90 0x7f8f8c06d150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0089b0 tx=0x7f8f9c008040 comp rx=0 tx=0).stop 2024-03-30T01:42:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c46c0 msgr2=0x7f8fac153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c46c0 0x7f8fac153720 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8f94014040 tx=0x7f8f94004750 comp rx=0 tx=0).stop 2024-03-30T01:42:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 shutdown_connections 2024-03-30T01:42:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8f8c06ac90 0x7f8f8c06d150 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8fac0c4aa0 0x7f8fac0abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fac0a6e90 0x7f8fac1de470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 --2- 172.21.15.31:0/3903228090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fac0c46c0 0x7f8fac153720 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 >> 172.21.15.31:0/3903228090 conn(0x7f8fac0c7cf0 msgr2=0x7f8fac0c1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 shutdown_connections 2024-03-30T01:42:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.214+0000 7f8fb0c96700 1 -- 172.21.15.31:0/3903228090 wait complete. 2024-03-30T01:42:36.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-30T01:42:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.367+0000 7f3cd8303700 1 Processor -- start 2024-03-30T01:42:36.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.367+0000 7f3cd8303700 1 -- start start 2024-03-30T01:42:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.368+0000 7f3cd8303700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151b60 0x7f3cd0070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.368+0000 7f3cd8303700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3cd00c5d40 0x7f3cd00c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.368+0000 7f3cd8303700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3cd0151f40 0x7f3cd00c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.368+0000 7f3cd8303700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd0155610 con 0x7f3cd00c5d40 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.368+0000 7f3cd8303700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd01547a0 con 0x7f3cd0151f40 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.368+0000 7f3cd8303700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd0154990 con 0x7f3cd0151b60 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.368+0000 7f3cd589e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3cd00c5d40 0x7f3cd00c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd589e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3cd00c5d40 0x7f3cd00c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34854/0 (socket says 172.21.15.31:34854) 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd589e700 1 -- 172.21.15.31:0/4206027770 learned_addr learned my addr 172.21.15.31:0/4206027770 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd68a0700 1 --2- 172.21.15.31:0/4206027770 >> v2:172.21.15.59:3300/0 conn(0x7f3cd0151f40 0x7f3cd00c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd609f700 1 --2- 172.21.15.31:0/4206027770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151b60 0x7f3cd0070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd589e700 1 -- 172.21.15.31:0/4206027770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151b60 msgr2=0x7f3cd0070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd589e700 1 --2- 172.21.15.31:0/4206027770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151b60 0x7f3cd0070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd589e700 1 -- 172.21.15.31:0/4206027770 >> v2:172.21.15.59:3300/0 conn(0x7f3cd0151f40 msgr2=0x7f3cd00c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd589e700 1 --2- 172.21.15.31:0/4206027770 >> v2:172.21.15.59:3300/0 conn(0x7f3cd0151f40 0x7f3cd00c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd589e700 1 -- 172.21.15.31:0/4206027770 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd0154c70 con 0x7f3cd00c5d40 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd589e700 1 --2- 172.21.15.31:0/4206027770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 0x7f3cd00c2760 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00a7a10 tx=0x7f3cc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8a0c0cda12e321 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd509d700 1 -- 172.21.15.31:0/4206027770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0011830 con 0x7f3cd00c5d40 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd509d700 1 -- 172.21.15.31:0/4206027770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc0011990 con 0x7f3cd00c5d40 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.369+0000 7f3cd509d700 1 -- 172.21.15.31:0/4206027770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0011d20 con 0x7f3cd00c5d40 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.370+0000 7f3cd8303700 1 -- 172.21.15.31:0/4206027770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 msgr2=0x7f3cd00c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.370+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4206027770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 0x7f3cd00c2760 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00a7a10 tx=0x7f3cc0009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.370+0000 7f3cd8303700 1 -- 172.21.15.31:0/4206027770 shutdown_connections 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.370+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4206027770 >> v2:172.21.15.59:3300/0 conn(0x7f3cd0151f40 0x7f3cd00c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.370+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4206027770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151b60 0x7f3cd0070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.370+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4206027770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 0x7f3cd00c2760 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.370+0000 7f3cd8303700 1 -- 172.21.15.31:0/4206027770 >> 172.21.15.31:0/4206027770 conn(0x7f3cd00c7d60 msgr2=0x7f3cd00c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.370+0000 7f3cd8303700 1 -- 172.21.15.31:0/4206027770 shutdown_connections 2024-03-30T01:42:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.370+0000 7f3cd8303700 1 -- 172.21.15.31:0/4206027770 wait complete. 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.371+0000 7f3cd8303700 1 Processor -- start 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.371+0000 7f3cd8303700 1 -- start start 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd8303700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 0x7f3cd0153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd8303700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0151b60 0x7f3cd0149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd8303700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151f40 0x7f3cd014a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd8303700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd01ea380 con 0x7f3cd00c5d40 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd8303700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd01e99e0 con 0x7f3cd0151b60 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd8303700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd01e9cc0 con 0x7f3cd0151f40 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd609f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 0x7f3cd0153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd609f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 0x7f3cd0153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34870/0 (socket says 172.21.15.31:34870) 2024-03-30T01:42:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd609f700 1 -- 172.21.15.31:0/4087659901 learned_addr learned my addr 172.21.15.31:0/4087659901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd68a0700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151f40 0x7f3cd014a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.372+0000 7f3cd589e700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0151b60 0x7f3cd0149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cd609f700 1 -- 172.21.15.31:0/4087659901 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151f40 msgr2=0x7f3cd014a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cd609f700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151f40 0x7f3cd014a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cd609f700 1 -- 172.21.15.31:0/4087659901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0151b60 msgr2=0x7f3cd0149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cd609f700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0151b60 0x7f3cd0149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cd609f700 1 -- 172.21.15.31:0/4087659901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0009550 con 0x7f3cd00c5d40 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cd609f700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 0x7f3cd0153660 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00e9b0 tx=0x7f3ccc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cc6ffd700 1 -- 172.21.15.31:0/4087659901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc009710 con 0x7f3cd00c5d40 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cc6ffd700 1 -- 172.21.15.31:0/4087659901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc00ee50 con 0x7f3cd00c5d40 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd01e9fa0 con 0x7f3cd00c5d40 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cc6ffd700 1 -- 172.21.15.31:0/4087659901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00ca50 con 0x7f3cd00c5d40 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.373+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd014d660 con 0x7f3cd00c5d40 2024-03-30T01:42:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.374+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd01ea5f0 con 0x7f3cd00c5d40 2024-03-30T01:42:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.375+0000 7f3cc6ffd700 1 -- 172.21.15.31:0/4087659901 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3ccc00cbb0 con 0x7f3cd00c5d40 2024-03-30T01:42:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.375+0000 7f3cc6ffd700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3cb406ac90 0x7f3cb406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.375+0000 7f3cc6ffd700 1 -- 172.21.15.31:0/4087659901 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f3ccc089780 con 0x7f3cd00c5d40 2024-03-30T01:42:36.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.375+0000 7f3cd589e700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3cb406ac90 0x7f3cb406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.376+0000 7f3cd589e700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3cb406ac90 0x7f3cb406d150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0014040 tx=0x7f3cc0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:36.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.380+0000 7f3cc6ffd700 1 -- 172.21.15.31:0/4087659901 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3ccc054550 con 0x7f3cd00c5d40 2024-03-30T01:42:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.598+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f3cd00747d0 con 0x7f3cd00c5d40 2024-03-30T01:42:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.598+0000 7f3cc6ffd700 1 -- 172.21.15.31:0/4087659901 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v43) v1 ==== 101+0+11 (secure 0 0 0) 0x7f3ccc00ce60 con 0x7f3cd00c5d40 2024-03-30T01:42:36.604 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:36.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.600+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3cb406ac90 msgr2=0x7f3cb406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.600+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3cb406ac90 0x7f3cb406d150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0014040 tx=0x7f3cc0008030 comp rx=0 tx=0).stop 2024-03-30T01:42:36.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.600+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 msgr2=0x7f3cd0153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.600+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 0x7f3cd0153660 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00e9b0 tx=0x7f3ccc00ecc0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.600+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 shutdown_connections 2024-03-30T01:42:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.600+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cd00c5d40 0x7f3cd0153660 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.600+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3cb406ac90 0x7f3cb406d150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.601+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd0151f40 0x7f3cd014a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.601+0000 7f3cd8303700 1 --2- 172.21.15.31:0/4087659901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd0151b60 0x7f3cd0149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.601+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 >> 172.21.15.31:0/4087659901 conn(0x7f3cd00c7d60 msgr2=0x7f3cd00c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.601+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 shutdown_connections 2024-03-30T01:42:36.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.601+0000 7f3cd8303700 1 -- 172.21.15.31:0/4087659901 wait complete. 2024-03-30T01:42:36.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T01:42:36.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.751+0000 7f70da40a700 1 Processor -- start 2024-03-30T01:42:36.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.752+0000 7f70da40a700 1 -- start start 2024-03-30T01:42:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.752+0000 7f70da40a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70d4154d20 0x7f70d4155100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.752+0000 7f70da40a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f70d40722b0 0x7f70d406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.752+0000 7f70da40a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d40a88c0 con 0x7f70d406db20 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.752+0000 7f70da40a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d40c3110 con 0x7f70d40722b0 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.752+0000 7f70da40a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d40a85e0 con 0x7f70d4154d20 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f70d40722b0 0x7f70d406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d89a7700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f70d406db20 0x7f70d41479d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50640/0 (socket says 172.21.15.31:50640) 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d89a7700 1 -- 172.21.15.31:0/3502114534 learned_addr learned my addr 172.21.15.31:0/3502114534 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f70d40722b0 0x7f70d406d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34874/0 (socket says 172.21.15.31:34874) 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d3fff700 1 --2- 172.21.15.31:0/3502114534 >> v2:172.21.15.59:3300/0 conn(0x7f70d4154d20 0x7f70d4155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d37fe700 1 -- 172.21.15.31:0/3502114534 >> v2:172.21.15.59:3300/0 conn(0x7f70d4154d20 msgr2=0x7f70d4155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d37fe700 1 --2- 172.21.15.31:0/3502114534 >> v2:172.21.15.59:3300/0 conn(0x7f70d4154d20 0x7f70d4155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d37fe700 1 -- 172.21.15.31:0/3502114534 >> v1:172.21.15.31:6789/0 conn(0x7f70d406db20 legacy=0x7f70d41479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d37fe700 1 -- 172.21.15.31:0/3502114534 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d40c2a50 con 0x7f70d40722b0 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d37fe700 1 --2- 172.21.15.31:0/3502114534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 0x7f70d406d5e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f70d4067370 tx=0x7f70c800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee67815ce121c290 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d2ffd700 1 -- 172.21.15.31:0/3502114534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c800dc70 con 0x7f70d40722b0 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d2ffd700 1 -- 172.21.15.31:0/3502114534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c800ddd0 con 0x7f70d40722b0 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.753+0000 7f70d2ffd700 1 -- 172.21.15.31:0/3502114534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8010690 con 0x7f70d40722b0 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.754+0000 7f70da40a700 1 -- 172.21.15.31:0/3502114534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 msgr2=0x7f70d406d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.754+0000 7f70da40a700 1 --2- 172.21.15.31:0/3502114534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 0x7f70d406d5e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f70d4067370 tx=0x7f70c800bdc0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.754+0000 7f70da40a700 1 -- 172.21.15.31:0/3502114534 shutdown_connections 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.754+0000 7f70da40a700 1 --2- 172.21.15.31:0/3502114534 >> v2:172.21.15.59:3300/0 conn(0x7f70d4154d20 0x7f70d4155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.754+0000 7f70da40a700 1 --2- 172.21.15.31:0/3502114534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 0x7f70d406d5e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.754+0000 7f70da40a700 1 -- 172.21.15.31:0/3502114534 >> 172.21.15.31:0/3502114534 conn(0x7f70d40c3a60 msgr2=0x7f70d40c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.754+0000 7f70da40a700 1 -- 172.21.15.31:0/3502114534 shutdown_connections 2024-03-30T01:42:36.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.754+0000 7f70da40a700 1 -- 172.21.15.31:0/3502114534 wait complete. 2024-03-30T01:42:36.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.755+0000 7f70da40a700 1 Processor -- start 2024-03-30T01:42:36.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.755+0000 7f70da40a700 1 -- start start 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70da40a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70d406db20 0x7f70d40bfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70da40a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 0x7f70d4143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70da40a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70d4154d20 0x7f70d40ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70da40a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d414ae50 con 0x7f70d40722b0 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70da40a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4144480 con 0x7f70d406db20 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70da40a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4147b50 con 0x7f70d4154d20 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 0x7f70d4143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 0x7f70d4143f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34886/0 (socket says 172.21.15.31:34886) 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70d37fe700 1 -- 172.21.15.31:0/2825590286 learned_addr learned my addr 172.21.15.31:0/2825590286 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70d3fff700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70d406db20 0x7f70d40bfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.756+0000 7f70d89a7700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70d4154d20 0x7f70d40ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70d37fe700 1 -- 172.21.15.31:0/2825590286 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70d4154d20 msgr2=0x7f70d40ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70d37fe700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70d4154d20 0x7f70d40ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70d37fe700 1 -- 172.21.15.31:0/2825590286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70d406db20 msgr2=0x7f70d40bfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:36.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70d37fe700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70d406db20 0x7f70d40bfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70d37fe700 1 -- 172.21.15.31:0/2825590286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c800b740 con 0x7f70d40722b0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70d37fe700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 0x7f70d4143f40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f70c800b2c0 tx=0x7f70c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70d0ff9700 1 -- 172.21.15.31:0/2825590286 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8003b70 con 0x7f70d40722b0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70d0ff9700 1 -- 172.21.15.31:0/2825590286 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c8003cf0 con 0x7f70d40722b0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d4147e30 con 0x7f70d40722b0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70d0ff9700 1 -- 172.21.15.31:0/2825590286 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c80106b0 con 0x7f70d40722b0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.757+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d40abfb0 con 0x7f70d40722b0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.758+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d406d190 con 0x7f70d40722b0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.759+0000 7f70d0ff9700 1 -- 172.21.15.31:0/2825590286 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f70c8019400 con 0x7f70d40722b0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.759+0000 7f70d0ff9700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70c006ac90 0x7f70c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.759+0000 7f70d0ff9700 1 -- 172.21.15.31:0/2825590286 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f70c808b1a0 con 0x7f70d40722b0 2024-03-30T01:42:36.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.759+0000 7f70d3fff700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70c006ac90 0x7f70c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:36.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.760+0000 7f70d3fff700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70c006ac90 0x7f70c006d150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f70d40c6de0 tx=0x7f70c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:36.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.764+0000 7f70d0ff9700 1 -- 172.21.15.31:0/2825590286 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f70c8055f70 con 0x7f70d40722b0 2024-03-30T01:42:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.997+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f70d404ea80 con 0x7f70d40722b0 2024-03-30T01:42:37.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.997+0000 7f70d0ff9700 1 -- 172.21.15.31:0/2825590286 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7f70c8003e60 con 0x7f70d40722b0 2024-03-30T01:42:37.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.999+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70c006ac90 msgr2=0x7f70c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:37.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:36.999+0000 7f70da40a700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70c006ac90 0x7f70c006d150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f70d40c6de0 tx=0x7f70c4008040 comp rx=0 tx=0).stop 2024-03-30T01:42:37.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 msgr2=0x7f70d4143f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:37.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 0x7f70d4143f40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f70c800b2c0 tx=0x7f70c8004750 comp rx=0 tx=0).stop 2024-03-30T01:42:37.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 shutdown_connections 2024-03-30T01:42:37.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70c006ac90 0x7f70c006d150 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70d4154d20 0x7f70d40ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70d406db20 0x7f70d40bfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 --2- 172.21.15.31:0/2825590286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70d40722b0 0x7f70d4143f40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 >> 172.21.15.31:0/2825590286 conn(0x7f70d40c3a60 msgr2=0x7f70d4145ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:37.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 shutdown_connections 2024-03-30T01:42:37.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.000+0000 7f70da40a700 1 -- 172.21.15.31:0/2825590286 wait complete. 2024-03-30T01:42:37.006 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-03-30T01:42:37.019 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-30T01:42:37.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:37.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:37.071 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:42:37.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:37.071 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T01:42:37.092 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:42:37.092 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:37.155 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247272k,nr_inodes=4061818,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1655 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-30T01:42:37.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:42:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.296+0000 7efc61852700 1 Processor -- start 2024-03-30T01:42:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.296+0000 7efc61852700 1 -- start start 2024-03-30T01:42:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc61852700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efc5c151c00 0x7efc5c155070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc61852700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efc5c06a700 0x7efc5c06ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc61852700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c1555b0 con 0x7efc5c06e9c0 2024-03-30T01:42:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc61852700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c157f60 con 0x7efc5c151c00 2024-03-30T01:42:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc61852700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c158240 con 0x7efc5c06a700 2024-03-30T01:42:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efc5c151c00 0x7efc5c155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5b7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efc5c06e9c0 0x7efc5c06ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50654/0 (socket says 172.21.15.31:50654) 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5b7fe700 1 -- 172.21.15.31:0/3208806047 learned_addr learned my addr 172.21.15.31:0/3208806047 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5affd700 1 --2- 172.21.15.31:0/3208806047 >> v2:172.21.15.31:3300/0 conn(0x7efc5c151c00 0x7efc5c155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34888/0 (socket says 172.21.15.31:34888) 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5a7fc700 1 --2- 172.21.15.31:0/3208806047 >> v2:172.21.15.59:3300/0 conn(0x7efc5c06a700 0x7efc5c06ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5affd700 1 -- 172.21.15.31:0/3208806047 >> v2:172.21.15.59:3300/0 conn(0x7efc5c06a700 msgr2=0x7efc5c06ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5affd700 1 --2- 172.21.15.31:0/3208806047 >> v2:172.21.15.59:3300/0 conn(0x7efc5c06a700 0x7efc5c06ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5affd700 1 -- 172.21.15.31:0/3208806047 >> v1:172.21.15.31:6789/0 conn(0x7efc5c06e9c0 legacy=0x7efc5c06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5affd700 1 -- 172.21.15.31:0/3208806047 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c158520 con 0x7efc5c151c00 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5a7fc700 1 --2- 172.21.15.31:0/3208806047 >> v2:172.21.15.59:3300/0 conn(0x7efc5c06a700 0x7efc5c06ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.297+0000 7efc5affd700 1 --2- 172.21.15.31:0/3208806047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c151c00 0x7efc5c155070 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0c1380 tx=0x7efc50009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbfc4b22f19570a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc59ffb700 1 -- 172.21.15.31:0/3208806047 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50011830 con 0x7efc5c151c00 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc59ffb700 1 -- 172.21.15.31:0/3208806047 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50011990 con 0x7efc5c151c00 2024-03-30T01:42:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc59ffb700 1 -- 172.21.15.31:0/3208806047 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50011d20 con 0x7efc5c151c00 2024-03-30T01:42:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc61852700 1 -- 172.21.15.31:0/3208806047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c151c00 msgr2=0x7efc5c155070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc61852700 1 --2- 172.21.15.31:0/3208806047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c151c00 0x7efc5c155070 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0c1380 tx=0x7efc50009b20 comp rx=0 tx=0).stop 2024-03-30T01:42:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc61852700 1 -- 172.21.15.31:0/3208806047 shutdown_connections 2024-03-30T01:42:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc61852700 1 --2- 172.21.15.31:0/3208806047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c151c00 0x7efc5c155070 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc61852700 1 --2- 172.21.15.31:0/3208806047 >> v2:172.21.15.59:3300/0 conn(0x7efc5c06a700 0x7efc5c06ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc61852700 1 -- 172.21.15.31:0/3208806047 >> 172.21.15.31:0/3208806047 conn(0x7efc5c06c230 msgr2=0x7efc5c0c6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc61852700 1 -- 172.21.15.31:0/3208806047 shutdown_connections 2024-03-30T01:42:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.298+0000 7efc61852700 1 -- 172.21.15.31:0/3208806047 wait complete. 2024-03-30T01:42:37.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.300+0000 7efc61852700 1 Processor -- start 2024-03-30T01:42:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.300+0000 7efc61852700 1 -- start start 2024-03-30T01:42:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc61852700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c06a700 0x7efc5c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc61852700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc5c06e9c0 0x7efc5c156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc61852700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc5c151c00 0x7efc5c1ee140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc61852700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c1ee680 con 0x7efc5c06a700 2024-03-30T01:42:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc61852700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c157330 con 0x7efc5c151c00 2024-03-30T01:42:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc61852700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c157610 con 0x7efc5c06e9c0 2024-03-30T01:42:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c06a700 0x7efc5c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c06a700 0x7efc5c154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34892/0 (socket says 172.21.15.31:34892) 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5affd700 1 -- 172.21.15.31:0/3636006343 learned_addr learned my addr 172.21.15.31:0/3636006343 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5b7fe700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc5c151c00 0x7efc5c1ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5a7fc700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc5c06e9c0 0x7efc5c156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5affd700 1 -- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc5c06e9c0 msgr2=0x7efc5c156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5affd700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc5c06e9c0 0x7efc5c156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5affd700 1 -- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc5c151c00 msgr2=0x7efc5c1ee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5affd700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc5c151c00 0x7efc5c1ee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5affd700 1 -- 172.21.15.31:0/3636006343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc500094a0 con 0x7efc5c06a700 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5a7fc700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc5c06e9c0 0x7efc5c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5b7fe700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc5c151c00 0x7efc5c1ee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc5affd700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c06a700 0x7efc5c154320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efc50000c00 tx=0x7efc50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:37.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.301+0000 7efc43fff700 1 -- 172.21.15.31:0/3636006343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500038a0 con 0x7efc5c06a700 2024-03-30T01:42:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.302+0000 7efc43fff700 1 -- 172.21.15.31:0/3636006343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5001e070 con 0x7efc5c06a700 2024-03-30T01:42:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.302+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c1ed940 con 0x7efc5c06a700 2024-03-30T01:42:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.302+0000 7efc43fff700 1 -- 172.21.15.31:0/3636006343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50003c20 con 0x7efc5c06a700 2024-03-30T01:42:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.302+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c149a20 con 0x7efc5c06a700 2024-03-30T01:42:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.303+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5c067bb0 con 0x7efc5c06a700 2024-03-30T01:42:37.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.303+0000 7efc43fff700 1 -- 172.21.15.31:0/3636006343 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efc5002d450 con 0x7efc5c06a700 2024-03-30T01:42:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.304+0000 7efc43fff700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc4806ac90 0x7efc4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.304+0000 7efc43fff700 1 -- 172.21.15.31:0/3636006343 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7efc5008a8d0 con 0x7efc5c06a700 2024-03-30T01:42:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.304+0000 7efc5a7fc700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc4806ac90 0x7efc4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:37.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.304+0000 7efc5a7fc700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc4806ac90 0x7efc4806d150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc5c14a250 tx=0x7efc4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:37.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.308+0000 7efc43fff700 1 -- 172.21.15.31:0/3636006343 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efc500556a0 con 0x7efc5c06a700 2024-03-30T01:42:37.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.571+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efc5c1ed7a0 con 0x7efc5c06a700 2024-03-30T01:42:37.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.572+0000 7efc43fff700 1 -- 172.21.15.31:0/3636006343 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+149 (secure 0 0 0) 0x7efc5c1ed7a0 con 0x7efc5c06a700 2024-03-30T01:42:37.577 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T01:42:37.577 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-30T01:42:37.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc4806ac90 msgr2=0x7efc4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:37.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc4806ac90 0x7efc4806d150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc5c14a250 tx=0x7efc4c008040 comp rx=0 tx=0).stop 2024-03-30T01:42:37.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c06a700 msgr2=0x7efc5c154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:37.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c06a700 0x7efc5c154320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efc50000c00 tx=0x7efc50004750 comp rx=0 tx=0).stop 2024-03-30T01:42:37.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 shutdown_connections 2024-03-30T01:42:37.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc5c06a700 0x7efc5c154320 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc4806ac90 0x7efc4806d150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc5c151c00 0x7efc5c1ee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 --2- 172.21.15.31:0/3636006343 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc5c06e9c0 0x7efc5c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:37.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 >> 172.21.15.31:0/3636006343 conn(0x7efc5c06c230 msgr2=0x7efc5c06ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:37.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 shutdown_connections 2024-03-30T01:42:37.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:37.574+0000 7efc61852700 1 -- 172.21.15.31:0/3636006343 wait complete. 2024-03-30T01:42:37.593 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-03-30T01:42:37.593 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:42:37.593 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-03-30T01:42:37.594 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T01:42:37.594 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:37.594 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-03-30T01:42:37.594 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-30T01:42:37.594 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T01:42:37.594 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:42:37.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:37.594 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-03-30T01:42:37.616 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:42:37.617 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42184sec preferred_lft 42184sec 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether e6:5a:8b:06:76:44 brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::5055:d3ff:fee5:dc2a/64 scope link 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:37.618 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T01:42:37.619 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether e6:5a:8b:06:76:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T01:42:37.619 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::e45a:8bff:fe06:7644/64 scope link 2024-03-30T01:42:37.619 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:37.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:37.619 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:42:37.689 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-03-30T01:42:37.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:37.691 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-03-30T01:42:37.756 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-03-30T01:42:37.756 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:42:37.756 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:42:37.756 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:42:37.757 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:42:37.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:37.757 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2024-03-30T01:42:37.757 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-03-30T01:42:37.757 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:42:37.757 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:42:37.757 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:42:37.757 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-03-30T01:42:37.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:37.881 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-03-30T01:42:37.901 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T01:42:37.902 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:42:37.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:37.902 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-03-30T01:42:37.977 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:42:37.977 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:38.039 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:42:38.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:42:38.039 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:38.100 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-30T01:42:38.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:38.101 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs2 2024-03-30T01:42:38.198 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs2 2024-03-30T01:42:38.198 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs2". 2024-03-30T01:42:38.198 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:42:38.208 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs2 2024-03-30T01:42:38.208 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs2". 2024-03-30T01:42:38.209 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:42:38.209 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:42:38.209 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:42:38.228 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs2 2024-03-30T01:42:38.228 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs2". 2024-03-30T01:42:38.228 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:42:38.228 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:42:38.229 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:42:38.229 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790" 2024-03-30T01:42:38.229 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790:/ 2024-03-30T01:42:38.229 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs2,key=1,fsid=9739890c-57ec-4503-9fd1-d4b0145023f4" will pass to kernel 2024-03-30T01:42:38.231 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T01:42:38.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:38.231 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:38.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:38.264 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:42:38.366 INFO:teuthology.orchestra.run.smithi059.stdout:client5619 2024-03-30T01:42:38.366 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:42:38.366 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:38.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs authorize cephfs client.testuser / rw 2024-03-30T01:42:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.532+0000 7f57bdf27700 1 Processor -- start 2024-03-30T01:42:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.532+0000 7f57bdf27700 1 -- start start 2024-03-30T01:42:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.533+0000 7f57bdf27700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f57b814ea30 0x7f57b8066580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.533+0000 7f57bdf27700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f57b8159d80 0x7f57b815c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.533+0000 7f57bdf27700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b806bbd0 con 0x7f57b80714d0 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.533+0000 7f57bdf27700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b8071300 con 0x7f57b814ea30 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.533+0000 7f57bdf27700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b80abf70 con 0x7f57b8159d80 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.533+0000 7f57b77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f57b814ea30 0x7f57b8066580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.533+0000 7f57b6ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f57b80714d0 0x7f57b8071980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50664/0 (socket says 172.21.15.31:50664) 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.533+0000 7f57b6ffd700 1 -- 172.21.15.31:0/2146301778 learned_addr learned my addr 172.21.15.31:0/2146301778 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.533+0000 7f57b7fff700 1 --2- 172.21.15.31:0/2146301778 >> v2:172.21.15.59:3300/0 conn(0x7f57b8159d80 0x7f57b815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b77fe700 1 -- 172.21.15.31:0/2146301778 >> v2:172.21.15.59:3300/0 conn(0x7f57b8159d80 msgr2=0x7f57b815c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b77fe700 1 --2- 172.21.15.31:0/2146301778 >> v2:172.21.15.59:3300/0 conn(0x7f57b8159d80 0x7f57b815c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b77fe700 1 -- 172.21.15.31:0/2146301778 >> v1:172.21.15.31:6789/0 conn(0x7f57b80714d0 legacy=0x7f57b8071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b77fe700 1 -- 172.21.15.31:0/2146301778 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8071020 con 0x7f57b814ea30 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b67fc700 1 -- 172.21.15.31:0/2146301778 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203481778 0 0) 0x7f57b806bbd0 con 0x7f57b80714d0 2024-03-30T01:42:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b7fff700 1 --2- 172.21.15.31:0/2146301778 >> v2:172.21.15.59:3300/0 conn(0x7f57b8159d80 0x7f57b815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b77fe700 1 --2- 172.21.15.31:0/2146301778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b814ea30 0x7f57b8066580 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f57b8155b00 tx=0x7f57a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d573477fb3afd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b67fc700 1 -- 172.21.15.31:0/2146301778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8011830 con 0x7f57b814ea30 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b67fc700 1 -- 172.21.15.31:0/2146301778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a8011990 con 0x7f57b814ea30 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.534+0000 7f57b67fc700 1 -- 172.21.15.31:0/2146301778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8011d20 con 0x7f57b814ea30 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.535+0000 7f57bdf27700 1 -- 172.21.15.31:0/2146301778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b814ea30 msgr2=0x7f57b8066580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.535+0000 7f57bdf27700 1 --2- 172.21.15.31:0/2146301778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b814ea30 0x7f57b8066580 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f57b8155b00 tx=0x7f57a8009b20 comp rx=0 tx=0).stop 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.535+0000 7f57bdf27700 1 -- 172.21.15.31:0/2146301778 shutdown_connections 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.535+0000 7f57bdf27700 1 --2- 172.21.15.31:0/2146301778 >> v2:172.21.15.59:3300/0 conn(0x7f57b8159d80 0x7f57b815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.535+0000 7f57bdf27700 1 --2- 172.21.15.31:0/2146301778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b814ea30 0x7f57b8066580 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.535+0000 7f57bdf27700 1 -- 172.21.15.31:0/2146301778 >> 172.21.15.31:0/2146301778 conn(0x7f57b80c30c0 msgr2=0x7f57b80c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.535+0000 7f57bdf27700 1 -- 172.21.15.31:0/2146301778 shutdown_connections 2024-03-30T01:42:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.535+0000 7f57bdf27700 1 -- 172.21.15.31:0/2146301778 wait complete. 2024-03-30T01:42:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.536+0000 7f57bdf27700 1 Processor -- start 2024-03-30T01:42:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.537+0000 7f57bdf27700 1 -- start start 2024-03-30T01:42:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.537+0000 7f57bdf27700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b80714d0 0x7f57b80c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.537+0000 7f57bdf27700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57b814ea30 0x7f57b8142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57bdf27700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b8159d80 0x7f57b81434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57bdf27700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8144770 con 0x7f57b8159d80 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57bdf27700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8144140 con 0x7f57b80714d0 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57bdf27700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80c4bd0 con 0x7f57b814ea30 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b8159d80 0x7f57b81434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b8159d80 0x7f57b81434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34908/0 (socket says 172.21.15.31:34908) 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b7fff700 1 -- 172.21.15.31:0/390421411 learned_addr learned my addr 172.21.15.31:0/390421411 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b77fe700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b80714d0 0x7f57b80c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b6ffd700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57b814ea30 0x7f57b8142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b7fff700 1 -- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57b814ea30 msgr2=0x7f57b8142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b7fff700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57b814ea30 0x7f57b8142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b7fff700 1 -- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b80714d0 msgr2=0x7f57b80c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b7fff700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b80714d0 0x7f57b80c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b7fff700 1 -- 172.21.15.31:0/390421411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80094a0 con 0x7f57b8159d80 2024-03-30T01:42:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b77fe700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b80714d0 0x7f57b80c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b6ffd700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57b814ea30 0x7f57b8142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57b7fff700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b8159d80 0x7f57b81434e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f57b0007ef0 tx=0x7f57b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f579ffff700 1 -- 172.21.15.31:0/390421411 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000de60 con 0x7f57b8159d80 2024-03-30T01:42:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f579ffff700 1 -- 172.21.15.31:0/390421411 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b000f040 con 0x7f57b8159d80 2024-03-30T01:42:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b80c4e70 con 0x7f57b8159d80 2024-03-30T01:42:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.538+0000 7f579ffff700 1 -- 172.21.15.31:0/390421411 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0003c60 con 0x7f57b8159d80 2024-03-30T01:42:38.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.539+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b80c48d0 con 0x7f57b8159d80 2024-03-30T01:42:38.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.540+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b81448b0 con 0x7f57b8159d80 2024-03-30T01:42:38.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.540+0000 7f579ffff700 1 -- 172.21.15.31:0/390421411 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f57b0004030 con 0x7f57b8159d80 2024-03-30T01:42:38.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.540+0000 7f579ffff700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57a406ac90 0x7f57a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:38.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.541+0000 7f579ffff700 1 -- 172.21.15.31:0/390421411 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f57b0089940 con 0x7f57b8159d80 2024-03-30T01:42:38.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.541+0000 7f57b77fe700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57a406ac90 0x7f57a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:38.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.541+0000 7f57b77fe700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57a406ac90 0x7f57a406d150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f57b81440d0 tx=0x7f57a8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:38.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.545+0000 7f579ffff700 1 -- 172.21.15.31:0/390421411 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f57b0054710 con 0x7f57b8159d80 2024-03-30T01:42:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.807+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs authorize", "filesystem": "cephfs", "entity": "client.testuser", "caps": ["/", "rw"]} v 0) v1 -- 0x7f57b804ea80 con 0x7f57b8159d80 2024-03-30T01:42:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.810+0000 7f579ffff700 1 -- 172.21.15.31:0/390421411 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs authorize", "filesystem": "cephfs", "entity": "client.testuser", "caps": ["/", "rw"]}]=0 v15) v1 ==== 134+0+66 (secure 0 0 0) 0x7f57b001a060 con 0x7f57b8159d80 2024-03-30T01:42:38.816 INFO:teuthology.orchestra.run.smithi031.stdout:[client.testuser] 2024-03-30T01:42:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAObgdm9FgmMBAAWTQz7GlIQ8SAlu2U9ss+0g== 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57a406ac90 msgr2=0x7f57a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57a406ac90 0x7f57a406d150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f57b81440d0 tx=0x7f57a8008030 comp rx=0 tx=0).stop 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b8159d80 msgr2=0x7f57b81434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b8159d80 0x7f57b81434e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f57b0007ef0 tx=0x7f57b000d3b0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 shutdown_connections 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57b8159d80 0x7f57b81434e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57a406ac90 0x7f57a406d150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57b80714d0 0x7f57b80c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 --2- 172.21.15.31:0/390421411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57b814ea30 0x7f57b8142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 >> 172.21.15.31:0/390421411 conn(0x7f57b80c30c0 msgr2=0x7f57b806b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 shutdown_connections 2024-03-30T01:42:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:38.813+0000 7f57bdf27700 1 -- 172.21.15.31:0/390421411 wait complete. 2024-03-30T01:42:38.832 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-03-30T01:42:38.851 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.5D0PwUI1Tu 2024-03-30T01:42:38.852 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:42:38.852 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/tmp/tmp.5D0PwUI1Tu 2024-03-30T01:42:38.967 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:42:38.968 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:38.990 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:42:38.990 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:39.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:39.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.232+0000 7f5984883700 1 Processor -- start 2024-03-30T01:42:39.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.232+0000 7f5984883700 1 -- start start 2024-03-30T01:42:39.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.233+0000 7f5984883700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59800ac520 0x7f59800ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.233+0000 7f5984883700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800a74e0 0x7f59800ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.233+0000 7f5984883700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5980153590 con 0x7f59800c6000 2024-03-30T01:42:39.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.233+0000 7f5984883700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59800ab680 con 0x7f59800ac520 2024-03-30T01:42:39.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.233+0000 7f5984883700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59801500c0 con 0x7f59800a74e0 2024-03-30T01:42:39.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.233+0000 7f597dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800a74e0 0x7f59800ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.233+0000 7f597e59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f59800ac520 0x7f59800ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.233+0000 7f597dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800a74e0 0x7f59800ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:38096/0 (socket says 172.21.15.31:38096) 2024-03-30T01:42:39.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.233+0000 7f597dd9b700 1 -- 172.21.15.31:0/1541179041 learned_addr learned my addr 172.21.15.31:0/1541179041 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.234+0000 7f597dd9b700 1 -- 172.21.15.31:0/1541179041 >> v2:172.21.15.59:3300/0 conn(0x7f59800ac520 msgr2=0x7f59800ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.234+0000 7f597dd9b700 1 --2- 172.21.15.31:0/1541179041 >> v2:172.21.15.59:3300/0 conn(0x7f59800ac520 0x7f59800ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.234+0000 7f597dd9b700 1 -- 172.21.15.31:0/1541179041 >> v1:172.21.15.59:6789/0 conn(0x7f59800c6000 legacy=0x7f59800c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.234+0000 7f597dd9b700 1 -- 172.21.15.31:0/1541179041 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800c7990 con 0x7f59800a74e0 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.234+0000 7f597e59c700 1 --2- 172.21.15.31:0/1541179041 >> v2:172.21.15.59:3300/0 conn(0x7f59800ac520 0x7f59800ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.234+0000 7f597dd9b700 1 --2- 172.21.15.31:0/1541179041 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800a74e0 0x7f59800ace40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59740098f0 tx=0x7f5974009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573382e890e230c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.234+0000 7f597d59a700 1 -- 172.21.15.31:0/1541179041 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974011830 con 0x7f59800a74e0 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.234+0000 7f597d59a700 1 -- 172.21.15.31:0/1541179041 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5974011990 con 0x7f59800a74e0 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.234+0000 7f597d59a700 1 -- 172.21.15.31:0/1541179041 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974011d20 con 0x7f59800a74e0 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.235+0000 7f5984883700 1 -- 172.21.15.31:0/1541179041 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800a74e0 msgr2=0x7f59800ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.235+0000 7f5984883700 1 --2- 172.21.15.31:0/1541179041 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800a74e0 0x7f59800ace40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59740098f0 tx=0x7f5974009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.235+0000 7f5984883700 1 -- 172.21.15.31:0/1541179041 shutdown_connections 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.235+0000 7f5984883700 1 --2- 172.21.15.31:0/1541179041 >> v2:172.21.15.59:3300/0 conn(0x7f59800ac520 0x7f59800ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.235+0000 7f5984883700 1 --2- 172.21.15.31:0/1541179041 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800a74e0 0x7f59800ace40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.235+0000 7f5984883700 1 -- 172.21.15.31:0/1541179041 >> 172.21.15.31:0/1541179041 conn(0x7f598006ad30 msgr2=0x7f59800a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:39.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.235+0000 7f5984883700 1 -- 172.21.15.31:0/1541179041 shutdown_connections 2024-03-30T01:42:39.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.235+0000 7f5984883700 1 -- 172.21.15.31:0/1541179041 wait complete. 2024-03-30T01:42:39.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.236+0000 7f5984883700 1 Processor -- start 2024-03-30T01:42:39.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.236+0000 7f5984883700 1 -- start start 2024-03-30T01:42:39.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f5984883700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800a74e0 0x7f59800c1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f5984883700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59800ac520 0x7f598014c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f5984883700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800c6000 0x7f598014c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f5984883700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59801e9cf0 con 0x7f59800a74e0 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f5984883700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59801514a0 con 0x7f59800ac520 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f5984883700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5980155520 con 0x7f59800c6000 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f597e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800a74e0 0x7f59800c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f597e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800a74e0 0x7f59800c1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34922/0 (socket says 172.21.15.31:34922) 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f597e59c700 1 -- 172.21.15.31:0/2842896160 learned_addr learned my addr 172.21.15.31:0/2842896160 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f597dd9b700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59800ac520 0x7f598014c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.237+0000 7f597ed9d700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800c6000 0x7f598014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f597e59c700 1 -- 172.21.15.31:0/2842896160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800c6000 msgr2=0x7f598014c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f597e59c700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800c6000 0x7f598014c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f597e59c700 1 -- 172.21.15.31:0/2842896160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59800ac520 msgr2=0x7f598014c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f597e59c700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59800ac520 0x7f598014c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f597e59c700 1 -- 172.21.15.31:0/2842896160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700094a0 con 0x7f59800a74e0 2024-03-30T01:42:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f597dd9b700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59800ac520 0x7f598014c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f597e59c700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800a74e0 0x7f59800c1480 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f597000ec10 tx=0x7f5970009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f596effd700 1 -- 172.21.15.31:0/2842896160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597000c940 con 0x7f59800a74e0 2024-03-30T01:42:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f596effd700 1 -- 172.21.15.31:0/2842896160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597000caa0 con 0x7f59800a74e0 2024-03-30T01:42:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f596effd700 1 -- 172.21.15.31:0/2842896160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597000ce10 con 0x7f59800a74e0 2024-03-30T01:42:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5974009550 con 0x7f59800a74e0 2024-03-30T01:42:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.238+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59801566c0 con 0x7f59800a74e0 2024-03-30T01:42:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.239+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59801e9f90 con 0x7f59800a74e0 2024-03-30T01:42:39.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.240+0000 7f596effd700 1 -- 172.21.15.31:0/2842896160 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5970013470 con 0x7f59800a74e0 2024-03-30T01:42:39.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.240+0000 7f596effd700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f596006ac90 0x7f596006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.240+0000 7f596effd700 1 -- 172.21.15.31:0/2842896160 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f5970089900 con 0x7f59800a74e0 2024-03-30T01:42:39.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.240+0000 7f597dd9b700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f596006ac90 0x7f596006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.241+0000 7f597dd9b700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f596006ac90 0x7f596006d150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5974009c40 tx=0x7f5974008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:39.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.245+0000 7f596effd700 1 -- 172.21.15.31:0/2842896160 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f59700546d0 con 0x7f59800a74e0 2024-03-30T01:42:39.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.460+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f598004ea80 con 0x7f59800a74e0 2024-03-30T01:42:39.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.462+0000 7f596effd700 1 -- 172.21.15.31:0/2842896160 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+19974 (secure 0 0 0) 0x7f5970057cf0 con 0x7f59800a74e0 2024-03-30T01:42:39.467 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:39.468 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":43,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:25.270018+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f596006ac90 msgr2=0x7f596006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f596006ac90 0x7f596006d150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5974009c40 tx=0x7f5974008030 comp rx=0 tx=0).stop 2024-03-30T01:42:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800a74e0 msgr2=0x7f59800c1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800a74e0 0x7f59800c1480 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f597000ec10 tx=0x7f5970009ec0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 shutdown_connections 2024-03-30T01:42:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59800a74e0 0x7f59800c1480 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f596006ac90 0x7f596006d150 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59800c6000 0x7f598014c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 --2- 172.21.15.31:0/2842896160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59800ac520 0x7f598014c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 >> 172.21.15.31:0/2842896160 conn(0x7f598006ad30 msgr2=0x7f59800c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:39.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 shutdown_connections 2024-03-30T01:42:39.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.464+0000 7f5984883700 1 -- 172.21.15.31:0/2842896160 wait complete. 2024-03-30T01:42:39.484 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:42:07.881738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:42:09.236870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-30T01:42:21.874769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-30T01:42:23.236623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:42:39.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:42:39.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.618+0000 7fa2869d6700 1 Processor -- start 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.618+0000 7fa2869d6700 1 -- start start 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa2869d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa28014fb40 0x7fa2800c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa2869d6700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa2800a7600 0x7fa280152f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa2869d6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2800c4620 con 0x7fa28014ff20 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa2869d6700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa280152570 con 0x7fa2800a7600 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa2869d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa280152ad0 con 0x7fa28014fb40 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa2800a7600 0x7fa280152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa2800a7600 0x7fa280152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34934/0 (socket says 172.21.15.31:34934) 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27f7fe700 1 -- 172.21.15.31:0/3015683411 learned_addr learned my addr 172.21.15.31:0/3015683411 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27ffff700 1 --2- 172.21.15.31:0/3015683411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa28014fb40 0x7fa2800c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27f7fe700 1 -- 172.21.15.31:0/3015683411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa28014fb40 msgr2=0x7fa2800c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:39.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27f7fe700 1 --2- 172.21.15.31:0/3015683411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa28014fb40 0x7fa2800c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27f7fe700 1 -- 172.21.15.31:0/3015683411 >> v1:172.21.15.59:6789/0 conn(0x7fa28014ff20 legacy=0x7fa2800c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27f7fe700 1 -- 172.21.15.31:0/3015683411 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2800c2ca0 con 0x7fa2800a7600 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27f7fe700 1 --2- 172.21.15.31:0/3015683411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2800a7600 0x7fa280152f60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa2740098f0 tx=0x7fa274009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8291d0b6dbf609ba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27effd700 1 -- 172.21.15.31:0/3015683411 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400b8a0 con 0x7fa2800a7600 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27effd700 1 -- 172.21.15.31:0/3015683411 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27400ba00 con 0x7fa2800a7600 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.619+0000 7fa27effd700 1 -- 172.21.15.31:0/3015683411 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400bd90 con 0x7fa2800a7600 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.620+0000 7fa2869d6700 1 -- 172.21.15.31:0/3015683411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2800a7600 msgr2=0x7fa280152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.620+0000 7fa2869d6700 1 --2- 172.21.15.31:0/3015683411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2800a7600 0x7fa280152f60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa2740098f0 tx=0x7fa274009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.620+0000 7fa2869d6700 1 -- 172.21.15.31:0/3015683411 shutdown_connections 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.620+0000 7fa2869d6700 1 --2- 172.21.15.31:0/3015683411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa28014fb40 0x7fa2800c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.620+0000 7fa2869d6700 1 --2- 172.21.15.31:0/3015683411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2800a7600 0x7fa280152f60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.620+0000 7fa2869d6700 1 -- 172.21.15.31:0/3015683411 >> 172.21.15.31:0/3015683411 conn(0x7fa2800c7d60 msgr2=0x7fa2800c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.620+0000 7fa2869d6700 1 -- 172.21.15.31:0/3015683411 shutdown_connections 2024-03-30T01:42:39.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.620+0000 7fa2869d6700 1 -- 172.21.15.31:0/3015683411 wait complete. 2024-03-30T01:42:39.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.621+0000 7fa2869d6700 1 Processor -- start 2024-03-30T01:42:39.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.622+0000 7fa2869d6700 1 -- start start 2024-03-30T01:42:39.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.622+0000 7fa2869d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2800a7600 0x7fa2801ed500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.622+0000 7fa2869d6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa28014fb40 0x7fa280099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.622+0000 7fa2869d6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa28014ff20 0x7fa2800c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.622+0000 7fa2869d6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28006c3f0 con 0x7fa28014ff20 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.622+0000 7fa2869d6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2801eda40 con 0x7fa28014fb40 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.622+0000 7fa2869d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28006b8f0 con 0x7fa2800a7600 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.622+0000 7fa284f73700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa28014ff20 0x7fa2800c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa284f73700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa28014ff20 0x7fa2800c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34950/0 (socket says 172.21.15.31:34950) 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa284f73700 1 -- 172.21.15.31:0/2448251290 learned_addr learned my addr 172.21.15.31:0/2448251290 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa27ffff700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2800a7600 0x7fa2801ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa27f7fe700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa28014fb40 0x7fa280099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa284f73700 1 -- 172.21.15.31:0/2448251290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2800a7600 msgr2=0x7fa2801ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa284f73700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2800a7600 0x7fa2801ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa284f73700 1 -- 172.21.15.31:0/2448251290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa28014fb40 msgr2=0x7fa280099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa284f73700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa28014fb40 0x7fa280099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa284f73700 1 -- 172.21.15.31:0/2448251290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa274009550 con 0x7fa28014ff20 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa284f73700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa28014ff20 0x7fa2800c1400 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa27800cbc0 tx=0x7fa27800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:39.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa27cff9700 1 -- 172.21.15.31:0/2448251290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278003c00 con 0x7fa28014ff20 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa27cff9700 1 -- 172.21.15.31:0/2448251290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa278003d60 con 0x7fa28014ff20 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa27cff9700 1 -- 172.21.15.31:0/2448251290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27800ab90 con 0x7fa28014ff20 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28006cbc0 con 0x7fa28014ff20 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.623+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa280099f40 con 0x7fa28014ff20 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.624+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28004ea80 con 0x7fa28014ff20 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.625+0000 7fa27cff9700 1 -- 172.21.15.31:0/2448251290 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa27800acf0 con 0x7fa28014ff20 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.625+0000 7fa27cff9700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa26c06ac90 0x7fa26c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.625+0000 7fa27cff9700 1 -- 172.21.15.31:0/2448251290 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fa278089610 con 0x7fa28014ff20 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.625+0000 7fa27ffff700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa26c06ac90 0x7fa26c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:39.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.626+0000 7fa27ffff700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa26c06ac90 0x7fa26c06d150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa270009930 tx=0x7fa270008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:39.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.630+0000 7fa27cff9700 1 -- 172.21.15.31:0/2448251290 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa2780543e0 con 0x7fa28014ff20 2024-03-30T01:42:39.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.843+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2800c4620 con 0x7fa28014ff20 2024-03-30T01:42:39.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.843+0000 7fa27cff9700 1 -- 172.21.15.31:0/2448251290 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2800c4620 con 0x7fa28014ff20 2024-03-30T01:42:39.849 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa26c06ac90 msgr2=0x7fa26c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa26c06ac90 0x7fa26c06d150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa270009930 tx=0x7fa270008040 comp rx=0 tx=0).stop 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa28014ff20 msgr2=0x7fa2800c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa28014ff20 0x7fa2800c1400 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa27800cbc0 tx=0x7fa27800cf80 comp rx=0 tx=0).stop 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 shutdown_connections 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa28014ff20 0x7fa2800c1400 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa26c06ac90 0x7fa26c06d150 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2800a7600 0x7fa2801ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 --2- 172.21.15.31:0/2448251290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa28014fb40 0x7fa280099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 >> 172.21.15.31:0/2448251290 conn(0x7fa2800c7d60 msgr2=0x7fa2800c47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 shutdown_connections 2024-03-30T01:42:39.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.846+0000 7fa2869d6700 1 -- 172.21.15.31:0/2448251290 wait complete. 2024-03-30T01:42:39.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:42:40.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.999+0000 7f75bcd9a700 1 Processor -- start 2024-03-30T01:42:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:39.999+0000 7f75bcd9a700 1 -- start start 2024-03-30T01:42:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.000+0000 7f75bcd9a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75b80c7ef0 0x7f75b81553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.000+0000 7f75bcd9a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75b80c4970 0x7f75b80c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.000+0000 7f75bcd9a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b81584c0 con 0x7f75b814e650 2024-03-30T01:42:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.000+0000 7f75bcd9a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b80c6430 con 0x7f75b80c4970 2024-03-30T01:42:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.000+0000 7f75bcd9a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b806cae0 con 0x7f75b80c7ef0 2024-03-30T01:42:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.000+0000 7f75b6d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75b80c4970 0x7f75b80c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.000+0000 7f75b659c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f75b814e650 0x7f75b8157db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50672/0 (socket says 172.21.15.31:50672) 2024-03-30T01:42:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.000+0000 7f75b6d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f75b80c4970 0x7f75b80c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34966/0 (socket says 172.21.15.31:34966) 2024-03-30T01:42:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.000+0000 7f75b659c700 1 -- 172.21.15.31:0/3000892391 learned_addr learned my addr 172.21.15.31:0/3000892391 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b5d9b700 1 --2- 172.21.15.31:0/3000892391 >> v2:172.21.15.59:3300/0 conn(0x7f75b80c7ef0 0x7f75b81553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b6d9d700 1 -- 172.21.15.31:0/3000892391 >> v2:172.21.15.59:3300/0 conn(0x7f75b80c7ef0 msgr2=0x7f75b81553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b6d9d700 1 --2- 172.21.15.31:0/3000892391 >> v2:172.21.15.59:3300/0 conn(0x7f75b80c7ef0 0x7f75b81553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b6d9d700 1 -- 172.21.15.31:0/3000892391 >> v1:172.21.15.31:6789/0 conn(0x7f75b814e650 legacy=0x7f75b8157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b6d9d700 1 -- 172.21.15.31:0/3000892391 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b80c8400 con 0x7f75b80c4970 2024-03-30T01:42:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b559a700 1 -- 172.21.15.31:0/3000892391 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473733251 0 0) 0x7f75b81584c0 con 0x7f75b814e650 2024-03-30T01:42:40.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b5d9b700 1 --2- 172.21.15.31:0/3000892391 >> v2:172.21.15.59:3300/0 conn(0x7f75b80c7ef0 0x7f75b81553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b6d9d700 1 --2- 172.21.15.31:0/3000892391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b80c4970 0x7f75b80c4da0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f75b80c67e0 tx=0x7f75b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c1d4537baf65b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b559a700 1 -- 172.21.15.31:0/3000892391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b000e070 con 0x7f75b80c4970 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b559a700 1 -- 172.21.15.31:0/3000892391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b000bbd0 con 0x7f75b80c4970 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.001+0000 7f75b559a700 1 -- 172.21.15.31:0/3000892391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0012430 con 0x7f75b80c4970 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.002+0000 7f75bcd9a700 1 -- 172.21.15.31:0/3000892391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b80c4970 msgr2=0x7f75b80c4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.002+0000 7f75bcd9a700 1 --2- 172.21.15.31:0/3000892391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b80c4970 0x7f75b80c4da0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f75b80c67e0 tx=0x7f75b000b330 comp rx=0 tx=0).stop 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.002+0000 7f75bcd9a700 1 -- 172.21.15.31:0/3000892391 shutdown_connections 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.002+0000 7f75bcd9a700 1 --2- 172.21.15.31:0/3000892391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b80c4970 0x7f75b80c4da0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.002+0000 7f75bcd9a700 1 --2- 172.21.15.31:0/3000892391 >> v2:172.21.15.59:3300/0 conn(0x7f75b80c7ef0 0x7f75b81553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.002+0000 7f75bcd9a700 1 -- 172.21.15.31:0/3000892391 >> 172.21.15.31:0/3000892391 conn(0x7f75b8072010 msgr2=0x7f75b8072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:40.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.002+0000 7f75bcd9a700 1 -- 172.21.15.31:0/3000892391 shutdown_connections 2024-03-30T01:42:40.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.002+0000 7f75bcd9a700 1 -- 172.21.15.31:0/3000892391 wait complete. 2024-03-30T01:42:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.003+0000 7f75bcd9a700 1 Processor -- start 2024-03-30T01:42:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.003+0000 7f75bcd9a700 1 -- start start 2024-03-30T01:42:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75bcd9a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75b80c4970 0x7f75b80bff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75bcd9a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75b80c7ef0 0x7f75b814d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75bcd9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b814e650 0x7f75b814d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75bcd9a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b80c3ec0 con 0x7f75b814e650 2024-03-30T01:42:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75bcd9a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b814da90 con 0x7f75b80c4970 2024-03-30T01:42:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75bcd9a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b80c2600 con 0x7f75b80c7ef0 2024-03-30T01:42:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75b6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b814e650 0x7f75b814d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75b6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b814e650 0x7f75b814d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34976/0 (socket says 172.21.15.31:34976) 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75b6d9d700 1 -- 172.21.15.31:0/2284556528 learned_addr learned my addr 172.21.15.31:0/2284556528 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75b659c700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75b80c4970 0x7f75b80bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75b5d9b700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75b80c7ef0 0x7f75b814d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75b6d9d700 1 -- 172.21.15.31:0/2284556528 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75b80c7ef0 msgr2=0x7f75b814d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75b6d9d700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75b80c7ef0 0x7f75b814d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75b6d9d700 1 -- 172.21.15.31:0/2284556528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75b80c4970 msgr2=0x7f75b80bff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.004+0000 7f75b6d9d700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75b80c4970 0x7f75b80bff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.005+0000 7f75b6d9d700 1 -- 172.21.15.31:0/2284556528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b0009a70 con 0x7f75b814e650 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.005+0000 7f75b5d9b700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75b80c7ef0 0x7f75b814d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.005+0000 7f75b6d9d700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b814e650 0x7f75b814d550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009fd0 tx=0x7f75b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.005+0000 7f75aeffd700 1 -- 172.21.15.31:0/2284556528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0012430 con 0x7f75b814e650 2024-03-30T01:42:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.005+0000 7f75aeffd700 1 -- 172.21.15.31:0/2284556528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b0020070 con 0x7f75b814e650 2024-03-30T01:42:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.005+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b81535e0 con 0x7f75b814e650 2024-03-30T01:42:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.005+0000 7f75aeffd700 1 -- 172.21.15.31:0/2284556528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0004280 con 0x7f75b814e650 2024-03-30T01:42:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.005+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b80c4350 con 0x7f75b814e650 2024-03-30T01:42:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.006+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b806c4d0 con 0x7f75b814e650 2024-03-30T01:42:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.007+0000 7f75aeffd700 1 -- 172.21.15.31:0/2284556528 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f75b00043e0 con 0x7f75b814e650 2024-03-30T01:42:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.007+0000 7f75aeffd700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f759806ac90 0x7f759806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.007+0000 7f75aeffd700 1 -- 172.21.15.31:0/2284556528 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f75b0022070 con 0x7f75b814e650 2024-03-30T01:42:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.007+0000 7f75b659c700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f759806ac90 0x7f759806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.008+0000 7f75b659c700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f759806ac90 0x7f759806d150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f75a8000f60 tx=0x7f75a8007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.012+0000 7f75aeffd700 1 -- 172.21.15.31:0/2284556528 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f75b0056450 con 0x7f75b814e650 2024-03-30T01:42:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.227+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75b80c4070 con 0x7f75b814e650 2024-03-30T01:42:40.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.228+0000 7f75aeffd700 1 -- 172.21.15.31:0/2284556528 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75b80c4070 con 0x7f75b814e650 2024-03-30T01:42:40.233 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f759806ac90 msgr2=0x7f759806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f759806ac90 0x7f759806d150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f75a8000f60 tx=0x7f75a8007be0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b814e650 msgr2=0x7f75b814d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b814e650 0x7f75b814d550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009fd0 tx=0x7f75b0004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 shutdown_connections 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b814e650 0x7f75b814d550 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f759806ac90 0x7f759806d150 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75b80c4970 0x7f75b80bff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 --2- 172.21.15.31:0/2284556528 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75b80c7ef0 0x7f75b814d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 >> 172.21.15.31:0/2284556528 conn(0x7f75b8072010 msgr2=0x7f75b80c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:40.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 shutdown_connections 2024-03-30T01:42:40.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.230+0000 7f75bcd9a700 1 -- 172.21.15.31:0/2284556528 wait complete. 2024-03-30T01:42:40.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:42:40.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.385+0000 7f69ebf5a700 1 Processor -- start 2024-03-30T01:42:40.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.386+0000 7f69ebf5a700 1 -- start start 2024-03-30T01:42:40.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.386+0000 7f69ebf5a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69e40721f0 0x7f69e406a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.386+0000 7f69ebf5a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e4155490 0x7f69e406ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.386+0000 7f69ebf5a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e4155000 con 0x7f69e4151ea0 2024-03-30T01:42:40.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.386+0000 7f69ebf5a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e406e930 con 0x7f69e40721f0 2024-03-30T01:42:40.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.386+0000 7f69ebf5a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e406df40 con 0x7f69e4155490 2024-03-30T01:42:40.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69e94f5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69e40721f0 0x7f69e406a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69ea4f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e4155490 0x7f69e406ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69e94f5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69e40721f0 0x7f69e406a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:49982/0 (socket says 172.21.15.31:49982) 2024-03-30T01:42:40.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69ea4f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e4155490 0x7f69e406ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:38152/0 (socket says 172.21.15.31:38152) 2024-03-30T01:42:40.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69ea4f7700 1 -- 172.21.15.31:0/3004861284 learned_addr learned my addr 172.21.15.31:0/3004861284 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:40.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69e9cf6700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f69e4151ea0 0x7f69e40c02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:52184/0 (socket says 172.21.15.31:52184) 2024-03-30T01:42:40.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69e8cf4700 1 -- 172.21.15.31:0/3004861284 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182349811 0 0) 0x7f69e4155000 con 0x7f69e4151ea0 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69e8cf4700 1 -- 172.21.15.31:0/3004861284 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d00035b0 con 0x7f69e4151ea0 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69e94f5700 1 -- 172.21.15.31:0/3004861284 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e4155490 msgr2=0x7f69e406ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69e94f5700 1 --2- 172.21.15.31:0/3004861284 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e4155490 0x7f69e406ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69e94f5700 1 -- 172.21.15.31:0/3004861284 >> v1:172.21.15.59:6789/0 conn(0x7f69e4151ea0 legacy=0x7f69e40c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69e94f5700 1 -- 172.21.15.31:0/3004861284 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e406e650 con 0x7f69e40721f0 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.387+0000 7f69ea4f7700 1 --2- 172.21.15.31:0/3004861284 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e4155490 0x7f69e406ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69e94f5700 1 --2- 172.21.15.31:0/3004861284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69e40721f0 0x7f69e406a700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009840 tx=0x7f69e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66dff234d5ae9271 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69e8cf4700 1 -- 172.21.15.31:0/3004861284 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0011830 con 0x7f69e40721f0 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69e8cf4700 1 -- 172.21.15.31:0/3004861284 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e0011990 con 0x7f69e40721f0 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69e8cf4700 1 -- 172.21.15.31:0/3004861284 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0011d20 con 0x7f69e40721f0 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69ebf5a700 1 -- 172.21.15.31:0/3004861284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69e40721f0 msgr2=0x7f69e406a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69ebf5a700 1 --2- 172.21.15.31:0/3004861284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69e40721f0 0x7f69e406a700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009840 tx=0x7f69e0009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69ebf5a700 1 -- 172.21.15.31:0/3004861284 shutdown_connections 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69ebf5a700 1 --2- 172.21.15.31:0/3004861284 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e4155490 0x7f69e406ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69ebf5a700 1 --2- 172.21.15.31:0/3004861284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69e40721f0 0x7f69e406a700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.388+0000 7f69ebf5a700 1 -- 172.21.15.31:0/3004861284 >> 172.21.15.31:0/3004861284 conn(0x7f69e40c3980 msgr2=0x7f69e40c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:40.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.389+0000 7f69ebf5a700 1 -- 172.21.15.31:0/3004861284 shutdown_connections 2024-03-30T01:42:40.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.389+0000 7f69ebf5a700 1 -- 172.21.15.31:0/3004861284 wait complete. 2024-03-30T01:42:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.390+0000 7f69ebf5a700 1 Processor -- start 2024-03-30T01:42:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.390+0000 7f69ebf5a700 1 -- start start 2024-03-30T01:42:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.391+0000 7f69ebf5a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e40721f0 0x7f69e4154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.391+0000 7f69ebf5a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4151ea0 0x7f69e4144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.391+0000 7f69ebf5a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69e4155490 0x7f69e4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.391+0000 7f69ebf5a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e4153ae0 con 0x7f69e4151ea0 2024-03-30T01:42:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.391+0000 7f69ebf5a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41ea4e0 con 0x7f69e4155490 2024-03-30T01:42:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.391+0000 7f69ebf5a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41ea7c0 con 0x7f69e40721f0 2024-03-30T01:42:40.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.391+0000 7f69e94f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4151ea0 0x7f69e4144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.391+0000 7f69e94f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4151ea0 0x7f69e4144c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34984/0 (socket says 172.21.15.31:34984) 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.391+0000 7f69e94f5700 1 -- 172.21.15.31:0/197886375 learned_addr learned my addr 172.21.15.31:0/197886375 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69e9cf6700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e40721f0 0x7f69e4154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69ea4f7700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69e4155490 0x7f69e4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69e94f5700 1 -- 172.21.15.31:0/197886375 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e40721f0 msgr2=0x7f69e4154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69e94f5700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e40721f0 0x7f69e4154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69e94f5700 1 -- 172.21.15.31:0/197886375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69e4155490 msgr2=0x7f69e4152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69e94f5700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69e4155490 0x7f69e4152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69e94f5700 1 -- 172.21.15.31:0/197886375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc009b30 con 0x7f69e4151ea0 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69e94f5700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4151ea0 0x7f69e4144c00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009fd0 tx=0x7f69e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69d67fc700 1 -- 172.21.15.31:0/197886375 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00036a0 con 0x7f69e4151ea0 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69d67fc700 1 -- 172.21.15.31:0/197886375 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e001e070 con 0x7f69e4151ea0 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e00094a0 con 0x7f69e4151ea0 2024-03-30T01:42:40.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69d67fc700 1 -- 172.21.15.31:0/197886375 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0003a60 con 0x7f69e4151ea0 2024-03-30T01:42:40.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.392+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e4145140 con 0x7f69e4151ea0 2024-03-30T01:42:40.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.393+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e41535d0 con 0x7f69e4151ea0 2024-03-30T01:42:40.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.394+0000 7f69d67fc700 1 -- 172.21.15.31:0/197886375 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f69e0003bc0 con 0x7f69e4151ea0 2024-03-30T01:42:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.394+0000 7f69d67fc700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69c806ac90 0x7f69c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.394+0000 7f69d67fc700 1 -- 172.21.15.31:0/197886375 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f69e008a770 con 0x7f69e4151ea0 2024-03-30T01:42:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.394+0000 7f69e9cf6700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69c806ac90 0x7f69c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.395+0000 7f69e9cf6700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69c806ac90 0x7f69c806d150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f69d80011e0 tx=0x7f69d8007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:40.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.399+0000 7f69d67fc700 1 -- 172.21.15.31:0/197886375 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f69e41535d0 con 0x7f69e4151ea0 2024-03-30T01:42:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.612+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69e41535d0 con 0x7f69e4151ea0 2024-03-30T01:42:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.612+0000 7f69d67fc700 1 -- 172.21.15.31:0/197886375 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f69e41535d0 con 0x7f69e4151ea0 2024-03-30T01:42:40.618 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69c806ac90 msgr2=0x7f69c806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69c806ac90 0x7f69c806d150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f69d80011e0 tx=0x7f69d8007be0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4151ea0 msgr2=0x7f69e4144c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4151ea0 0x7f69e4144c00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009fd0 tx=0x7f69e0004750 comp rx=0 tx=0).stop 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 shutdown_connections 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69c806ac90 0x7f69c806d150 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69e4155490 0x7f69e4152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69e40721f0 0x7f69e4154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 --2- 172.21.15.31:0/197886375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69e4151ea0 0x7f69e4144c00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 >> 172.21.15.31:0/197886375 conn(0x7f69e40c3980 msgr2=0x7f69e414fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 shutdown_connections 2024-03-30T01:42:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.615+0000 7f69ebf5a700 1 -- 172.21.15.31:0/197886375 wait complete. 2024-03-30T01:42:40.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-30T01:42:40.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.767+0000 7f95ea258700 1 Processor -- start 2024-03-30T01:42:40.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.767+0000 7f95ea258700 1 -- start start 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.768+0000 7f95ea258700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 0x7f95e40c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.768+0000 7f95ea258700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e4155240 con 0x7f95e4151d80 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.768+0000 7f95ea258700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e41556d0 con 0x7f95e40c8420 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.768+0000 7f95ea258700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e406cb20 con 0x7f95e40c7f10 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.768+0000 7f95e37fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f95e4151d80 0x7f95e40ca350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50684/0 (socket says 172.21.15.31:50684) 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.768+0000 7f95e37fe700 1 -- 172.21.15.31:0/3852366862 learned_addr learned my addr 172.21.15.31:0/3852366862 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.769+0000 7f95e2ffd700 1 --2- 172.21.15.31:0/3852366862 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 0x7f95e40c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.769+0000 7f95e27fc700 1 -- 172.21.15.31:0/3852366862 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419936880 0 0) 0x7f95e4155240 con 0x7f95e4151d80 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.769+0000 7f95e27fc700 1 -- 172.21.15.31:0/3852366862 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95cc0035b0 con 0x7f95e4151d80 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.769+0000 7f95e27fc700 1 -- 172.21.15.31:0/3852366862 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577135169 0 0) 0x7f95e41556d0 con 0x7f95e40c8420 2024-03-30T01:42:40.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.769+0000 7f95e2ffd700 1 -- 172.21.15.31:0/3852366862 >> v1:172.21.15.59:6789/0 conn(0x7f95e40c8420 legacy=0x7f95e40c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.769+0000 7f95e2ffd700 1 -- 172.21.15.31:0/3852366862 >> v1:172.21.15.31:6789/0 conn(0x7f95e4151d80 legacy=0x7f95e40ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.769+0000 7f95e2ffd700 1 -- 172.21.15.31:0/3852366862 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4154910 con 0x7f95e40c7f10 2024-03-30T01:42:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.769+0000 7f95e2ffd700 1 --2- 172.21.15.31:0/3852366862 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 0x7f95e40c6a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009870 tx=0x7f95d8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad0cb80187ffefe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.769+0000 7f95e27fc700 1 -- 172.21.15.31:0/3852366862 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8011830 con 0x7f95e40c7f10 2024-03-30T01:42:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.770+0000 7f95e27fc700 1 -- 172.21.15.31:0/3852366862 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d8011990 con 0x7f95e40c7f10 2024-03-30T01:42:40.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.770+0000 7f95e27fc700 1 -- 172.21.15.31:0/3852366862 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8011d20 con 0x7f95e40c7f10 2024-03-30T01:42:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.770+0000 7f95ea258700 1 -- 172.21.15.31:0/3852366862 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 msgr2=0x7f95e40c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.770+0000 7f95ea258700 1 --2- 172.21.15.31:0/3852366862 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 0x7f95e40c6a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009870 tx=0x7f95d8009b80 comp rx=0 tx=0).stop 2024-03-30T01:42:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.770+0000 7f95ea258700 1 -- 172.21.15.31:0/3852366862 shutdown_connections 2024-03-30T01:42:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.770+0000 7f95ea258700 1 --2- 172.21.15.31:0/3852366862 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 0x7f95e40c6a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.770+0000 7f95ea258700 1 -- 172.21.15.31:0/3852366862 >> 172.21.15.31:0/3852366862 conn(0x7f95e4072010 msgr2=0x7f95e4072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.770+0000 7f95ea258700 1 -- 172.21.15.31:0/3852366862 shutdown_connections 2024-03-30T01:42:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.770+0000 7f95ea258700 1 -- 172.21.15.31:0/3852366862 wait complete. 2024-03-30T01:42:40.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.772+0000 7f95ea258700 1 Processor -- start 2024-03-30T01:42:40.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.772+0000 7f95ea258700 1 -- start start 2024-03-30T01:42:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95ea258700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 0x7f95e414d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95ea258700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95e40c8420 0x7f95e40bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95ea258700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e4151d80 0x7f95e4155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95ea258700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40aa680 con 0x7f95e40c8420 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95ea258700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4157620 con 0x7f95e4151d80 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95ea258700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40ac910 con 0x7f95e40c7f10 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95e40c8420 0x7f95e40bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95e40c8420 0x7f95e40bfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34998/0 (socket says 172.21.15.31:34998) 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95e2ffd700 1 -- 172.21.15.31:0/3523804357 learned_addr learned my addr 172.21.15.31:0/3523804357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95e3fff700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e4151d80 0x7f95e4155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95e37fe700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 0x7f95e414d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95e2ffd700 1 -- 172.21.15.31:0/3523804357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 msgr2=0x7f95e414d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.773+0000 7f95e2ffd700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 0x7f95e414d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.774+0000 7f95e2ffd700 1 -- 172.21.15.31:0/3523804357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e4151d80 msgr2=0x7f95e4155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.774+0000 7f95e2ffd700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e4151d80 0x7f95e4155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.774+0000 7f95e2ffd700 1 -- 172.21.15.31:0/3523804357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d80094a0 con 0x7f95e40c8420 2024-03-30T01:42:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.774+0000 7f95e2ffd700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95e40c8420 0x7f95e40bfd90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f95d80090b0 tx=0x7f95d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.774+0000 7f95cbfff700 1 -- 172.21.15.31:0/3523804357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d801a9f0 con 0x7f95e40c8420 2024-03-30T01:42:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.774+0000 7f95cbfff700 1 -- 172.21.15.31:0/3523804357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d801ab50 con 0x7f95e40c8420 2024-03-30T01:42:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.774+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e40a9a70 con 0x7f95e40c8420 2024-03-30T01:42:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.774+0000 7f95cbfff700 1 -- 172.21.15.31:0/3523804357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d80243f0 con 0x7f95e40c8420 2024-03-30T01:42:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.774+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e40aa030 con 0x7f95e40c8420 2024-03-30T01:42:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.775+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e40747d0 con 0x7f95e40c8420 2024-03-30T01:42:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.775+0000 7f95cbfff700 1 -- 172.21.15.31:0/3523804357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f95d802d450 con 0x7f95e40c8420 2024-03-30T01:42:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.776+0000 7f95cbfff700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f95d006ac90 0x7f95d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.776+0000 7f95cbfff700 1 -- 172.21.15.31:0/3523804357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f95d808aa60 con 0x7f95e40c8420 2024-03-30T01:42:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.776+0000 7f95e37fe700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f95d006ac90 0x7f95d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.776+0000 7f95e37fe700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f95d006ac90 0x7f95d006d150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f95d4007fd0 tx=0x7f95d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:40.781+0000 7f95cbfff700 1 -- 172.21.15.31:0/3523804357 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f95d80558c0 con 0x7f95e40c8420 2024-03-30T01:42:41.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.003+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95e414f9d0 con 0x7f95e40c8420 2024-03-30T01:42:41.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.003+0000 7f95cbfff700 1 -- 172.21.15.31:0/3523804357 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v43) v1 ==== 105+0+11 (secure 0 0 0) 0x7f95d8015020 con 0x7f95e40c8420 2024-03-30T01:42:41.009 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f95d006ac90 msgr2=0x7f95d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f95d006ac90 0x7f95d006d150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f95d4007fd0 tx=0x7f95d4009040 comp rx=0 tx=0).stop 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95e40c8420 msgr2=0x7f95e40bfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95e40c8420 0x7f95e40bfd90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f95d80090b0 tx=0x7f95d8004750 comp rx=0 tx=0).stop 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 shutdown_connections 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f95d006ac90 0x7f95d006d150 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e4151d80 0x7f95e4155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e40c7f10 0x7f95e414d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 --2- 172.21.15.31:0/3523804357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95e40c8420 0x7f95e40bfd90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 >> 172.21.15.31:0/3523804357 conn(0x7f95e4072010 msgr2=0x7f95e40723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 shutdown_connections 2024-03-30T01:42:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.006+0000 7f95ea258700 1 -- 172.21.15.31:0/3523804357 wait complete. 2024-03-30T01:42:41.025 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-30T01:42:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.158+0000 7fc7df225700 1 Processor -- start 2024-03-30T01:42:41.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.158+0000 7fc7df225700 1 -- start start 2024-03-30T01:42:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7df225700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc7d80721f0 0x7fc7d806a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7df225700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7d8155490 0x7fc7d806ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7df225700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d8155000 con 0x7fc7d8151ea0 2024-03-30T01:42:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7df225700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d806e930 con 0x7fc7d8155490 2024-03-30T01:42:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7df225700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d806df40 con 0x7fc7d80721f0 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7dd7c2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7d8155490 0x7fc7d806ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7dd7c2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7d8155490 0x7fc7d806ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35002/0 (socket says 172.21.15.31:35002) 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7dcfc1700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc7d8151ea0 0x7fc7d80c02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50692/0 (socket says 172.21.15.31:50692) 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7dd7c2700 1 -- 172.21.15.31:0/1385842538 learned_addr learned my addr 172.21.15.31:0/1385842538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.159+0000 7fc7d7fff700 1 --2- 172.21.15.31:0/1385842538 >> v2:172.21.15.59:3300/0 conn(0x7fc7d80721f0 0x7fc7d806a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.160+0000 7fc7dd7c2700 1 -- 172.21.15.31:0/1385842538 >> v2:172.21.15.59:3300/0 conn(0x7fc7d80721f0 msgr2=0x7fc7d806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.160+0000 7fc7dd7c2700 1 --2- 172.21.15.31:0/1385842538 >> v2:172.21.15.59:3300/0 conn(0x7fc7d80721f0 0x7fc7d806a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.160+0000 7fc7dd7c2700 1 -- 172.21.15.31:0/1385842538 >> v1:172.21.15.31:6789/0 conn(0x7fc7d8151ea0 legacy=0x7fc7d80c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.160+0000 7fc7dd7c2700 1 -- 172.21.15.31:0/1385842538 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d806e650 con 0x7fc7d8155490 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.160+0000 7fc7dd7c2700 1 --2- 172.21.15.31:0/1385842538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 0x7fc7d806ac40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8073420 tx=0x7fc7d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45095892bd07df42 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.160+0000 7fc7d77fe700 1 -- 172.21.15.31:0/1385842538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000e070 con 0x7fc7d8155490 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.160+0000 7fc7d77fe700 1 -- 172.21.15.31:0/1385842538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d000b910 con 0x7fc7d8155490 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.160+0000 7fc7d77fe700 1 -- 172.21.15.31:0/1385842538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000bcc0 con 0x7fc7d8155490 2024-03-30T01:42:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.161+0000 7fc7df225700 1 -- 172.21.15.31:0/1385842538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 msgr2=0x7fc7d806ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.161+0000 7fc7df225700 1 --2- 172.21.15.31:0/1385842538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 0x7fc7d806ac40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8073420 tx=0x7fc7d000b330 comp rx=0 tx=0).stop 2024-03-30T01:42:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.161+0000 7fc7df225700 1 -- 172.21.15.31:0/1385842538 shutdown_connections 2024-03-30T01:42:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.161+0000 7fc7df225700 1 --2- 172.21.15.31:0/1385842538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 0x7fc7d806ac40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.161+0000 7fc7df225700 1 --2- 172.21.15.31:0/1385842538 >> v2:172.21.15.59:3300/0 conn(0x7fc7d80721f0 0x7fc7d806a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.161+0000 7fc7df225700 1 -- 172.21.15.31:0/1385842538 >> 172.21.15.31:0/1385842538 conn(0x7fc7d80c3980 msgr2=0x7fc7d80c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.161+0000 7fc7df225700 1 -- 172.21.15.31:0/1385842538 shutdown_connections 2024-03-30T01:42:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.161+0000 7fc7df225700 1 -- 172.21.15.31:0/1385842538 wait complete. 2024-03-30T01:42:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.162+0000 7fc7df225700 1 Processor -- start 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.162+0000 7fc7df225700 1 -- start start 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7df225700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d80721f0 0x7fc7d80c2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7df225700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7d8151ea0 0x7fc7d8144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7df225700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 0x7fc7d81449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7df225700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8149b40 con 0x7fc7d8155490 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7df225700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8158690 con 0x7fc7d80721f0 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7df225700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8145a40 con 0x7fc7d8151ea0 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dd7c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 0x7fc7d81449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dd7c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 0x7fc7d81449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35012/0 (socket says 172.21.15.31:35012) 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dd7c2700 1 -- 172.21.15.31:0/2464957588 learned_addr learned my addr 172.21.15.31:0/2464957588 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dcfc1700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d80721f0 0x7fc7d80c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7d7fff700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7d8151ea0 0x7fc7d8144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dd7c2700 1 -- 172.21.15.31:0/2464957588 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7d8151ea0 msgr2=0x7fc7d8144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dd7c2700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7d8151ea0 0x7fc7d8144470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dd7c2700 1 -- 172.21.15.31:0/2464957588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d80721f0 msgr2=0x7fc7d80c2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dd7c2700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d80721f0 0x7fc7d80c2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dd7c2700 1 -- 172.21.15.31:0/2464957588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d0009b30 con 0x7fc7d8155490 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.163+0000 7fc7dd7c2700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 0x7fc7d81449b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000b370 tx=0x7fc7d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.164+0000 7fc7d57fa700 1 -- 172.21.15.31:0/2464957588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0004880 con 0x7fc7d8155490 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.164+0000 7fc7d57fa700 1 -- 172.21.15.31:0/2464957588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d00099b0 con 0x7fc7d8155490 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.164+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d8149780 con 0x7fc7d8155490 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.164+0000 7fc7d57fa700 1 -- 172.21.15.31:0/2464957588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d001b670 con 0x7fc7d8155490 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.164+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d80c2570 con 0x7fc7d8155490 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.165+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d8067fa0 con 0x7fc7d8155490 2024-03-30T01:42:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.165+0000 7fc7d57fa700 1 -- 172.21.15.31:0/2464957588 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc7d001b7d0 con 0x7fc7d8155490 2024-03-30T01:42:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.166+0000 7fc7d57fa700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7b806ac90 0x7fc7b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.166+0000 7fc7d57fa700 1 -- 172.21.15.31:0/2464957588 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fc7d008af00 con 0x7fc7d8155490 2024-03-30T01:42:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.166+0000 7fc7dcfc1700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7b806ac90 0x7fc7b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.166+0000 7fc7dcfc1700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7b806ac90 0x7fc7b806d150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8000f60 tx=0x7fc7c8007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.170+0000 7fc7d57fa700 1 -- 172.21.15.31:0/2464957588 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc7d0055cd0 con 0x7fc7d8155490 2024-03-30T01:42:41.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.385+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fc7d80747d0 con 0x7fc7d8155490 2024-03-30T01:42:41.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.385+0000 7fc7d57fa700 1 -- 172.21.15.31:0/2464957588 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v43) v1 ==== 101+0+11 (secure 0 0 0) 0x7fc7d001f020 con 0x7fc7d8155490 2024-03-30T01:42:41.391 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:41.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7b806ac90 msgr2=0x7fc7b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7b806ac90 0x7fc7b806d150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8000f60 tx=0x7fc7c8007ed0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 msgr2=0x7fc7d81449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 0x7fc7d81449b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000b370 tx=0x7fc7d0004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 shutdown_connections 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d8155490 0x7fc7d81449b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7b806ac90 0x7fc7b806d150 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7d80721f0 0x7fc7d80c2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 --2- 172.21.15.31:0/2464957588 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7d8151ea0 0x7fc7d8144470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 >> 172.21.15.31:0/2464957588 conn(0x7fc7d80c3980 msgr2=0x7fc7d80c58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 shutdown_connections 2024-03-30T01:42:41.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.388+0000 7fc7df225700 1 -- 172.21.15.31:0/2464957588 wait complete. 2024-03-30T01:42:41.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T01:42:41.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.539+0000 7fab5ed03700 1 Processor -- start 2024-03-30T01:42:41.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.539+0000 7fab5ed03700 1 -- start start 2024-03-30T01:42:41.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.540+0000 7fab5ed03700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab580ac520 0x7fab580ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.540+0000 7fab5ed03700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab58153590 con 0x7fab580a74e0 2024-03-30T01:42:41.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.540+0000 7fab5ed03700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab580c7990 con 0x7fab580ace40 2024-03-30T01:42:41.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.540+0000 7fab5ed03700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab580ab680 con 0x7fab580ac520 2024-03-30T01:42:41.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.540+0000 7fab5ca9f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab580ac520 0x7fab580ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.540+0000 7fab57fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fab580a74e0 0x7fab580c6000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50706/0 (socket says 172.21.15.31:50706) 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.540+0000 7fab57fff700 1 -- 172.21.15.31:0/3916386413 learned_addr learned my addr 172.21.15.31:0/3916386413 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab5ca9f700 1 -- 172.21.15.31:0/3916386413 >> v1:172.21.15.59:6789/0 conn(0x7fab580ace40 legacy=0x7fab580c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab5ca9f700 1 -- 172.21.15.31:0/3916386413 >> v1:172.21.15.31:6789/0 conn(0x7fab580a74e0 legacy=0x7fab580c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab5ca9f700 1 -- 172.21.15.31:0/3916386413 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab58153030 con 0x7fab580ac520 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab577fe700 1 -- 172.21.15.31:0/3916386413 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421493742 0 0) 0x7fab58153590 con 0x7fab580a74e0 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab5ca9f700 1 --2- 172.21.15.31:0/3916386413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580ac520 0x7fab580ac900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fab48009870 tx=0x7fab48009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596049321b4b6260 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab577fe700 1 -- 172.21.15.31:0/3916386413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48011830 con 0x7fab580ac520 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab577fe700 1 -- 172.21.15.31:0/3916386413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab48011990 con 0x7fab580ac520 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab577fe700 1 -- 172.21.15.31:0/3916386413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48011d20 con 0x7fab580ac520 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab5ed03700 1 -- 172.21.15.31:0/3916386413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580ac520 msgr2=0x7fab580ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.541+0000 7fab5ed03700 1 --2- 172.21.15.31:0/3916386413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580ac520 0x7fab580ac900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fab48009870 tx=0x7fab48009b80 comp rx=0 tx=0).stop 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.542+0000 7fab5ed03700 1 -- 172.21.15.31:0/3916386413 shutdown_connections 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.542+0000 7fab5ed03700 1 --2- 172.21.15.31:0/3916386413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580ac520 0x7fab580ac900 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.542+0000 7fab5ed03700 1 -- 172.21.15.31:0/3916386413 >> 172.21.15.31:0/3916386413 conn(0x7fab5806ad30 msgr2=0x7fab580a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:41.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.542+0000 7fab5ed03700 1 -- 172.21.15.31:0/3916386413 shutdown_connections 2024-03-30T01:42:41.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.542+0000 7fab5ed03700 1 -- 172.21.15.31:0/3916386413 wait complete. 2024-03-30T01:42:41.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.543+0000 7fab5ed03700 1 Processor -- start 2024-03-30T01:42:41.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.543+0000 7fab5ed03700 1 -- start start 2024-03-30T01:42:41.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ed03700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580a74e0 0x7fab5809b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ed03700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab580ac520 0x7fab58150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ed03700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab580ace40 0x7fab5814f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ed03700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5806d6d0 con 0x7fab580a74e0 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ed03700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5809a220 con 0x7fab580ace40 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ed03700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab58099b80 con 0x7fab580ac520 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ca9f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580a74e0 0x7fab5809b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ca9f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580a74e0 0x7fab5809b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35024/0 (socket says 172.21.15.31:35024) 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ca9f700 1 -- 172.21.15.31:0/886254004 learned_addr learned my addr 172.21.15.31:0/886254004 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab57fff700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab580ac520 0x7fab58150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5d2a0700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab580ace40 0x7fab5814f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ca9f700 1 -- 172.21.15.31:0/886254004 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab580ac520 msgr2=0x7fab58150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ca9f700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab580ac520 0x7fab58150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ca9f700 1 -- 172.21.15.31:0/886254004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab580ace40 msgr2=0x7fab5814f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ca9f700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab580ace40 0x7fab5814f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab5ca9f700 1 -- 172.21.15.31:0/886254004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480094a0 con 0x7fab580a74e0 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.544+0000 7fab57fff700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab580ac520 0x7fab58150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.545+0000 7fab5ca9f700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580a74e0 0x7fab5809b310 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fab480090b0 tx=0x7fab48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.545+0000 7fab557fa700 1 -- 172.21.15.31:0/886254004 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab480044f0 con 0x7fab580a74e0 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.545+0000 7fab557fa700 1 -- 172.21.15.31:0/886254004 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4801f070 con 0x7fab580a74e0 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.545+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5806d3f0 con 0x7fab580a74e0 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.545+0000 7fab557fa700 1 -- 172.21.15.31:0/886254004 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab480038e0 con 0x7fab580a74e0 2024-03-30T01:42:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.545+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab581511a0 con 0x7fab580a74e0 2024-03-30T01:42:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.546+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab581ed130 con 0x7fab580a74e0 2024-03-30T01:42:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.546+0000 7fab557fa700 1 -- 172.21.15.31:0/886254004 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fab48021030 con 0x7fab580a74e0 2024-03-30T01:42:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.547+0000 7fab557fa700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab3806ac90 0x7fab3806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.547+0000 7fab557fa700 1 -- 172.21.15.31:0/886254004 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fab4808ac30 con 0x7fab580a74e0 2024-03-30T01:42:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.547+0000 7fab57fff700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab3806ac90 0x7fab3806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.547+0000 7fab57fff700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab3806ac90 0x7fab3806d150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab58151430 tx=0x7fab40007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:41.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.552+0000 7fab557fa700 1 -- 172.21.15.31:0/886254004 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fab581ed130 con 0x7fab580a74e0 2024-03-30T01:42:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.781+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fab5804ea80 con 0x7fab580a74e0 2024-03-30T01:42:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.782+0000 7fab557fa700 1 -- 172.21.15.31:0/886254004 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7fab48055a00 con 0x7fab580a74e0 2024-03-30T01:42:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab3806ac90 msgr2=0x7fab3806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab3806ac90 0x7fab3806d150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab58151430 tx=0x7fab40007ce0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580a74e0 msgr2=0x7fab5809b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580a74e0 0x7fab5809b310 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fab480090b0 tx=0x7fab48004750 comp rx=0 tx=0).stop 2024-03-30T01:42:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 shutdown_connections 2024-03-30T01:42:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab580a74e0 0x7fab5809b310 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fab3806ac90 0x7fab3806d150 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab580ace40 0x7fab5814f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 --2- 172.21.15.31:0/886254004 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab580ac520 0x7fab58150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 >> 172.21.15.31:0/886254004 conn(0x7fab5806ad30 msgr2=0x7fab5806e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 shutdown_connections 2024-03-30T01:42:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:41.784+0000 7fab5ed03700 1 -- 172.21.15.31:0/886254004 wait complete. 2024-03-30T01:42:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-03-30T01:42:41.803 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-30T01:42:41.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:41.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-30T01:42:41.845 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-03-30T01:42:41.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:41.846 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T01:42:41.905 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:42:41.906 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:41.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247268k,nr_inodes=4061817,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19833 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253136k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-30T01:42:41.970 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:42:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.149+0000 7f4fe90cd700 1 Processor -- start 2024-03-30T01:42:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.149+0000 7f4fe90cd700 1 -- start start 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe90cd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fe40c3180 0x7f4fe406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe90cd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 0x7f4fe406f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe90cd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4fe40c4710 0x7f4fe40c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe90cd700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe41547d0 con 0x7f4fe40c3180 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe90cd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe4154a90 con 0x7f4fe40c4710 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe90cd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4154d70 con 0x7f4fe406a700 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fe40c3180 0x7f4fe406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe2d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fe40c3180 0x7f4fe406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35040/0 (socket says 172.21.15.31:35040) 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe2d9d700 1 -- 172.21.15.31:0/48942164 learned_addr learned my addr 172.21.15.31:0/48942164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe259c700 1 --2- 172.21.15.31:0/48942164 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 0x7f4fe406f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.150+0000 7f4fe359e700 1 --2- 172.21.15.31:0/48942164 >> v2:172.21.15.59:3300/0 conn(0x7f4fe40c4710 0x7f4fe40c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe2d9d700 1 -- 172.21.15.31:0/48942164 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 msgr2=0x7f4fe406f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe2d9d700 1 --2- 172.21.15.31:0/48942164 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 0x7f4fe406f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe2d9d700 1 -- 172.21.15.31:0/48942164 >> v2:172.21.15.59:3300/0 conn(0x7f4fe40c4710 msgr2=0x7f4fe40c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe2d9d700 1 --2- 172.21.15.31:0/48942164 >> v2:172.21.15.59:3300/0 conn(0x7f4fe40c4710 0x7f4fe40c0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe2d9d700 1 -- 172.21.15.31:0/48942164 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4155050 con 0x7f4fe40c3180 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe359e700 1 --2- 172.21.15.31:0/48942164 >> v2:172.21.15.59:3300/0 conn(0x7f4fe40c4710 0x7f4fe40c0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe2d9d700 1 --2- 172.21.15.31:0/48942164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 0x7f4fe406eb00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4152310 tx=0x7f4fcc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48933986512b9f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe1d9b700 1 -- 172.21.15.31:0/48942164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc011830 con 0x7f4fe40c3180 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe1d9b700 1 -- 172.21.15.31:0/48942164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc011990 con 0x7f4fe40c3180 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.151+0000 7f4fe1d9b700 1 -- 172.21.15.31:0/48942164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc011d20 con 0x7f4fe40c3180 2024-03-30T01:42:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.152+0000 7f4fe90cd700 1 -- 172.21.15.31:0/48942164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 msgr2=0x7f4fe406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:42.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.152+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/48942164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 0x7f4fe406eb00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4152310 tx=0x7f4fcc009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.152+0000 7f4fe90cd700 1 -- 172.21.15.31:0/48942164 shutdown_connections 2024-03-30T01:42:42.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.152+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/48942164 >> v2:172.21.15.59:3300/0 conn(0x7f4fe40c4710 0x7f4fe40c0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.152+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/48942164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 0x7f4fe406eb00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.152+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/48942164 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 0x7f4fe406f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.152+0000 7f4fe90cd700 1 -- 172.21.15.31:0/48942164 >> 172.21.15.31:0/48942164 conn(0x7f4fe40c3a60 msgr2=0x7f4fe40c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:42.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.152+0000 7f4fe90cd700 1 -- 172.21.15.31:0/48942164 shutdown_connections 2024-03-30T01:42:42.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.152+0000 7f4fe90cd700 1 -- 172.21.15.31:0/48942164 wait complete. 2024-03-30T01:42:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.153+0000 7f4fe90cd700 1 Processor -- start 2024-03-30T01:42:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.153+0000 7f4fe90cd700 1 -- start start 2024-03-30T01:42:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe90cd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 0x7f4fe414f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe90cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 0x7f4fe41eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe90cd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe40c4710 0x7f4fe41eb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe90cd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4155b90 con 0x7f4fe40c3180 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe90cd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41e9c90 con 0x7f4fe40c4710 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe90cd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4147940 con 0x7f4fe406a700 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 0x7f4fe41eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 0x7f4fe41eadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35044/0 (socket says 172.21.15.31:35044) 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe259c700 1 -- 172.21.15.31:0/138534663 learned_addr learned my addr 172.21.15.31:0/138534663 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe2d9d700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 0x7f4fe414f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.154+0000 7f4fe359e700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe40c4710 0x7f4fe41eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fe259c700 1 -- 172.21.15.31:0/138534663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 msgr2=0x7f4fe414f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fe259c700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 0x7f4fe414f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fe259c700 1 -- 172.21.15.31:0/138534663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe40c4710 msgr2=0x7f4fe41eb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fe259c700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe40c4710 0x7f4fe41eb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fe259c700 1 -- 172.21.15.31:0/138534663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc009550 con 0x7f4fe40c3180 2024-03-30T01:42:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fe359e700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe40c4710 0x7f4fe41eb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fe259c700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 0x7f4fe41eadf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400e930 tx=0x7f4fd400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fdb7fe700 1 -- 172.21.15.31:0/138534663 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400ee80 con 0x7f4fe40c3180 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fdb7fe700 1 -- 172.21.15.31:0/138534663 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd400c850 con 0x7f4fe40c3180 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe4147c20 con 0x7f4fe40c3180 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fdb7fe700 1 -- 172.21.15.31:0/138534663 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400cbe0 con 0x7f4fe40c3180 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.155+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe41481e0 con 0x7f4fe40c3180 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.156+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe404ea80 con 0x7f4fe40c3180 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.157+0000 7f4fdb7fe700 1 -- 172.21.15.31:0/138534663 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4fd400cd40 con 0x7f4fe40c3180 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.157+0000 7f4fdb7fe700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4fc406ac90 0x7f4fc406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.157+0000 7f4fdb7fe700 1 -- 172.21.15.31:0/138534663 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f4fd4089780 con 0x7f4fe40c3180 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.157+0000 7f4fe2d9d700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4fc406ac90 0x7f4fc406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:42.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.158+0000 7f4fe2d9d700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4fc406ac90 0x7f4fc406d150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009c10 tx=0x7f4fcc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:42.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.162+0000 7f4fdb7fe700 1 -- 172.21.15.31:0/138534663 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4fd4054550 con 0x7f4fe40c3180 2024-03-30T01:42:42.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.428+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4fe40722d0 con 0x7f4fe40c3180 2024-03-30T01:42:42.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.429+0000 7f4fdb7fe700 1 -- 172.21.15.31:0/138534663 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+149 (secure 0 0 0) 0x7f4fd4057b70 con 0x7f4fe40c3180 2024-03-30T01:42:42.434 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T01:42:42.434 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-30T01:42:42.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4fc406ac90 msgr2=0x7f4fc406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:42.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4fc406ac90 0x7f4fc406d150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009c10 tx=0x7f4fcc008030 comp rx=0 tx=0).stop 2024-03-30T01:42:42.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 msgr2=0x7f4fe41eadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:42.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 0x7f4fe41eadf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400e930 tx=0x7f4fd400ec40 comp rx=0 tx=0).stop 2024-03-30T01:42:42.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 shutdown_connections 2024-03-30T01:42:42.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4fc406ac90 0x7f4fc406d150 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe40c4710 0x7f4fe41eb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe406a700 0x7f4fe414f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 --2- 172.21.15.31:0/138534663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe40c3180 0x7f4fe41eadf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:42.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 >> 172.21.15.31:0/138534663 conn(0x7f4fe40c3a60 msgr2=0x7f4fe4152600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:42.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 shutdown_connections 2024-03-30T01:42:42.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:42.431+0000 7f4fe90cd700 1 -- 172.21.15.31:0/138534663 wait complete. 2024-03-30T01:42:42.450 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-03-30T01:42:42.450 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:42:42.450 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-03-30T01:42:42.450 INFO:tasks.cephfs.mount:self.client.name = client.testuser 2024-03-30T01:42:42.451 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T01:42:42.451 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-03-30T01:42:42.451 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-30T01:42:42.451 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.5D0PwUI1Tu 2024-03-30T01:42:42.451 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-03-30T01:42:42.470 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.UdzOJBtUXw 2024-03-30T01:42:42.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /tmp/tmp.5D0PwUI1Tu /tmp/tmp.UdzOJBtUXw 2024-03-30T01:42:42.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.UdzOJBtUXw 2024-03-30T01:42:42.685 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.UdzOJBtUXw is 66B 2024-03-30T01:42:42.713 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.UdzOJBtUXw 2024-03-30T01:42:42.734 INFO:tasks.cephfs.mount:keyring content - [client.testuser] key = AQAObgdm9FgmMBAAWTQz7GlIQ8SAlu2U9ss+0g== 2024-03-30T01:42:42.734 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:42:42.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:42.734 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-03-30T01:42:42.797 INFO:teuthology.orchestra.run.smithi031.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42162sec preferred_lft 42162sec 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-03-30T01:42:42.798 INFO:teuthology.orchestra.run.smithi031.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:37:22:42:e7:ee brd ff:ff:ff:ff:ff:ff 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::2cb6:4cff:fefa:d1a5/64 scope link 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:37:22:42:e7:ee brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c37:22ff:fe42:e7ee/64 scope link 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:42:42.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:42.799 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:42:42.871 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-03-30T01:42:42.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:42.874 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-03-30T01:42:42.939 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-03-30T01:42:42.939 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:42:42.939 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:42:42.939 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:42:42.939 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:42:42.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:42.940 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:42:42.940 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:42:42.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:42:42.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:42:42.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:42:42.940 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:42:43.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:43.068 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-03-30T01:42:43.088 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T01:42:43.089 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:42:43.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:43.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-03-30T01:42:43.166 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:42:43.166 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T01:42:43.228 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:42:43.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:42:43.229 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T01:42:43.290 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:42:43.290 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/tmp/tmp.5D0PwUI1Tu of=/dev/stdout 2024-03-30T01:42:43.352 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-30T01:42:43.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:43.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,secret=AQAObgdm9FgmMBAAWTQz7GlIQ8SAlu2U9ss+0g==,conf=/etc/ceph/ceph.conf,norbytes,name=testuser,mds_namespace=cephfs2 2024-03-30T01:42:43.452 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,secret=AQAObgdm9FgmMBAAWTQz7GlIQ8SAlu2U9ss+0g==,conf=/etc/ceph/ceph.conf,norbytes,name=testuser,mds_namespace=cephfs2 2024-03-30T01:42:43.452 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,mds_namespace=cephfs2". 2024-03-30T01:42:43.452 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:42:43.452 INFO:teuthology.orchestra.run.smithi031.stdout:keyring.get_secret failed 2024-03-30T01:42:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,secret=AQAObgdm9FgmMBAAWTQz7GlIQ8SAlu2U9ss+0g==,conf=/etc/ceph/ceph.conf,norbytes,name=testuser,mds_namespace=cephfs2 2024-03-30T01:42:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,mds_namespace=cephfs2". 2024-03-30T01:42:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:42:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:42:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:42:43.463 INFO:teuthology.orchestra.run.smithi031.stdout:keyring.get_secret failed 2024-03-30T01:42:43.484 INFO:teuthology.orchestra.run.smithi031.stderr:mount error 2 = No such file or directory 2024-03-30T01:42:43.485 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,secret=AQAObgdm9FgmMBAAWTQz7GlIQ8SAlu2U9ss+0g==,conf=/etc/ceph/ceph.conf,norbytes,name=testuser,mds_namespace=cephfs2 2024-03-30T01:42:43.485 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,mds_namespace=cephfs2". 2024-03-30T01:42:43.485 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:42:43.485 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:42:43.485 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:42:43.485 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790" 2024-03-30T01:42:43.485 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790:/ 2024-03-30T01:42:43.485 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,mds_namespace=cephfs2,key=testuser,fsid=9739890c-57ec-4503-9fd1-d4b0145023f4" will pass to kernel 2024-03-30T01:42:43.488 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T01:42:43.488 INFO:tasks.cephfs.kernel_mount:mount command failed 2024-03-30T01:42:43.488 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T01:42:43.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T01:42:43.538 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T01:42:43.592 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T01:42:43.592 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:42:43.592 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:43.615 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247268k,nr_inodes=4061817,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19833 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253136k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-30T01:42:43.615 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:42:43.615 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:43.638 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:42:43.638 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:42:43.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:43.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.840+0000 7fa55d949700 1 Processor -- start 2024-03-30T01:42:43.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.840+0000 7fa55d949700 1 -- start start 2024-03-30T01:42:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.840+0000 7fa55d949700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5580a86d0 0x7fa558153040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.840+0000 7fa55d949700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa55806cf20 0x7fa55806d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:43.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.840+0000 7fa55d949700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa558156680 con 0x7fa5580c6990 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.840+0000 7fa55d949700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5580a9c10 con 0x7fa5580a86d0 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.840+0000 7fa55d949700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa558153620 con 0x7fa55806cf20 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa556ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5580a86d0 0x7fa558153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa5567fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa5580c6990 0x7fa55814f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50718/0 (socket says 172.21.15.31:50718) 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa5567fc700 1 -- 172.21.15.31:0/3044748468 learned_addr learned my addr 172.21.15.31:0/3044748468 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa5577fe700 1 --2- 172.21.15.31:0/3044748468 >> v2:172.21.15.59:3300/0 conn(0x7fa55806cf20 0x7fa55806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa556ffd700 1 -- 172.21.15.31:0/3044748468 >> v2:172.21.15.59:3300/0 conn(0x7fa55806cf20 msgr2=0x7fa55806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa556ffd700 1 --2- 172.21.15.31:0/3044748468 >> v2:172.21.15.59:3300/0 conn(0x7fa55806cf20 0x7fa55806d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa556ffd700 1 -- 172.21.15.31:0/3044748468 >> v1:172.21.15.31:6789/0 conn(0x7fa5580c6990 legacy=0x7fa55814f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa556ffd700 1 -- 172.21.15.31:0/3044748468 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558156b10 con 0x7fa5580a86d0 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa555ffb700 1 -- 172.21.15.31:0/3044748468 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865500396 0 0) 0x7fa558156680 con 0x7fa5580c6990 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa556ffd700 1 --2- 172.21.15.31:0/3044748468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 0x7fa558153040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa5580c67e0 tx=0x7fa548009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fa0158e8f598fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa555ffb700 1 -- 172.21.15.31:0/3044748468 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548011830 con 0x7fa5580a86d0 2024-03-30T01:42:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa555ffb700 1 -- 172.21.15.31:0/3044748468 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa548011990 con 0x7fa5580a86d0 2024-03-30T01:42:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.841+0000 7fa555ffb700 1 -- 172.21.15.31:0/3044748468 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548011d20 con 0x7fa5580a86d0 2024-03-30T01:42:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.842+0000 7fa55d949700 1 -- 172.21.15.31:0/3044748468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 msgr2=0x7fa558153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.842+0000 7fa55d949700 1 --2- 172.21.15.31:0/3044748468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 0x7fa558153040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa5580c67e0 tx=0x7fa548009b20 comp rx=0 tx=0).stop 2024-03-30T01:42:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.842+0000 7fa55d949700 1 -- 172.21.15.31:0/3044748468 shutdown_connections 2024-03-30T01:42:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.842+0000 7fa55d949700 1 --2- 172.21.15.31:0/3044748468 >> v2:172.21.15.59:3300/0 conn(0x7fa55806cf20 0x7fa55806d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.842+0000 7fa55d949700 1 --2- 172.21.15.31:0/3044748468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 0x7fa558153040 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.842+0000 7fa55d949700 1 -- 172.21.15.31:0/3044748468 >> 172.21.15.31:0/3044748468 conn(0x7fa55806af80 msgr2=0x7fa5580c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.842+0000 7fa55d949700 1 -- 172.21.15.31:0/3044748468 shutdown_connections 2024-03-30T01:42:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.842+0000 7fa55d949700 1 -- 172.21.15.31:0/3044748468 wait complete. 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.843+0000 7fa55d949700 1 Processor -- start 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.843+0000 7fa55d949700 1 -- start start 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa55d949700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa55806cf20 0x7fa5581e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa55d949700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 0x7fa558151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa55d949700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5580c6990 0x7fa5580c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa55d949700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5581e8630 con 0x7fa5580a86d0 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa55d949700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5581429c0 con 0x7fa5580c6990 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa55d949700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa558142c60 con 0x7fa55806cf20 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa5567fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 0x7fa558151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa5567fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 0x7fa558151030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35062/0 (socket says 172.21.15.31:35062) 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa5567fc700 1 -- 172.21.15.31:0/3933962383 learned_addr learned my addr 172.21.15.31:0/3933962383 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa5577fe700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5580c6990 0x7fa5580c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.844+0000 7fa556ffd700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa55806cf20 0x7fa5581e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa5567fc700 1 -- 172.21.15.31:0/3933962383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa55806cf20 msgr2=0x7fa5581e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa5567fc700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa55806cf20 0x7fa5581e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa5567fc700 1 -- 172.21.15.31:0/3933962383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5580c6990 msgr2=0x7fa5580c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa5567fc700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5580c6990 0x7fa5580c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa5567fc700 1 -- 172.21.15.31:0/3933962383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480094a0 con 0x7fa5580a86d0 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa5577fe700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5580c6990 0x7fa5580c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa5567fc700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 0x7fa558151030 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c550 tx=0x7fa54c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa53ffff700 1 -- 172.21.15.31:0/3933962383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c004c80 con 0x7fa5580a86d0 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa53ffff700 1 -- 172.21.15.31:0/3933962383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54c004e00 con 0x7fa5580a86d0 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa558142f40 con 0x7fa5580a86d0 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa53ffff700 1 -- 172.21.15.31:0/3933962383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c007310 con 0x7fa5580a86d0 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.845+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5581447b0 con 0x7fa5580a86d0 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.846+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5580c5c60 con 0x7fa5580a86d0 2024-03-30T01:42:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.846+0000 7fa53ffff700 1 -- 172.21.15.31:0/3933962383 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa54c007470 con 0x7fa5580a86d0 2024-03-30T01:42:43.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.847+0000 7fa53ffff700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa54406ac90 0x7fa54406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:43.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.847+0000 7fa53ffff700 1 -- 172.21.15.31:0/3933962383 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6972+0+0 (secure 0 0 0) 0x7fa54c0894e0 con 0x7fa5580a86d0 2024-03-30T01:42:43.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.847+0000 7fa556ffd700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa54406ac90 0x7fa54406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:43.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.847+0000 7fa556ffd700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa54406ac90 0x7fa54406d150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa5480090b0 tx=0x7fa548008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:43.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:43.852+0000 7fa53ffff700 1 -- 172.21.15.31:0/3933962383 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa5580c5c60 con 0x7fa5580a86d0 2024-03-30T01:42:44.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.074+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5580c5c60 con 0x7fa5580a86d0 2024-03-30T01:42:44.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.076+0000 7fa53ffff700 1 -- 172.21.15.31:0/3933962383 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20061 (secure 0 0 0) 0x7fa5580c5c60 con 0x7fa5580a86d0 2024-03-30T01:42:44.082 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:44.082 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:43.871647+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:44.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.078+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa54406ac90 msgr2=0x7fa54406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.078+0000 7fa55d949700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa54406ac90 0x7fa54406d150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa5480090b0 tx=0x7fa548008030 comp rx=0 tx=0).stop 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.078+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 msgr2=0x7fa558151030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.078+0000 7fa55d949700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 0x7fa558151030 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c550 tx=0x7fa54c00c910 comp rx=0 tx=0).stop 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.079+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 shutdown_connections 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.079+0000 7fa55d949700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa54406ac90 0x7fa54406d150 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.079+0000 7fa55d949700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5580c6990 0x7fa5580c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.079+0000 7fa55d949700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa55806cf20 0x7fa5581e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.079+0000 7fa55d949700 1 --2- 172.21.15.31:0/3933962383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5580a86d0 0x7fa558151030 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.079+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 >> 172.21.15.31:0/3933962383 conn(0x7fa55806af80 msgr2=0x7fa55806dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.079+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 shutdown_connections 2024-03-30T01:42:44.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.079+0000 7fa55d949700 1 -- 172.21.15.31:0/3933962383 wait complete. 2024-03-30T01:42:44.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:42:07.881738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:42:09.236870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-30T01:42:21.874769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-30T01:42:23.236623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:42:44.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:42:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.232+0000 7fa85bc2e700 1 Processor -- start 2024-03-30T01:42:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.232+0000 7fa85bc2e700 1 -- start start 2024-03-30T01:42:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.233+0000 7fa85bc2e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8540c1560 0x7fa85406c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.233+0000 7fa85bc2e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85406cad0 con 0x7fa8540721b0 2024-03-30T01:42:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.233+0000 7fa85bc2e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85406d2e0 con 0x7fa8540c1940 2024-03-30T01:42:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.233+0000 7fa85bc2e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540bafa0 con 0x7fa8540c1560 2024-03-30T01:42:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.233+0000 7fa8591c9700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa8540721b0 0x7fa8540bb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50728/0 (socket says 172.21.15.31:50728) 2024-03-30T01:42:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.233+0000 7fa8591c9700 1 -- 172.21.15.31:0/1012453371 learned_addr learned my addr 172.21.15.31:0/1012453371 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8599ca700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8540c1560 0x7fa85406c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142708093 0 0) 0x7fa85406cad0 con 0x7fa8540721b0 2024-03-30T01:42:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8400035b0 con 0x7fa8540721b0 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535017320 0 0) 0x7fa85406d2e0 con 0x7fa8540c1940 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa85406cad0 con 0x7fa8540c1940 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3457407369 0 0) 0x7fa8400035b0 con 0x7fa8540721b0 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8540c1560 msgr2=0x7fa85406c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8540c1560 0x7fa85406c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 >> v1:172.21.15.59:6789/0 conn(0x7fa8540c1940 legacy=0x7fa85415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540c1db0 con 0x7fa8540721b0 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7fa844001670 con 0x7fa8540721b0 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 >> v1:172.21.15.31:6789/0 conn(0x7fa8540721b0 legacy=0x7fa8540bb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8540c1560 0x7fa840003ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa840004250 0x7fa840004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa840004d90 0x7fa840005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540bafa0 con 0x7fa840004d90 2024-03-30T01:42:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84000bca0 con 0x7fa840004250 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84000be60 con 0x7fa8540c1560 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa85a1cb700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa840004d90 0x7fa840005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8599ca700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8540c1560 0x7fa840003ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.234+0000 7fa8591c9700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa840004250 0x7fa840004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa85a1cb700 1 -- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8540c1560 msgr2=0x7fa840003ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa85a1cb700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8540c1560 0x7fa840003ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa85a1cb700 1 -- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa840004250 msgr2=0x7fa840004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa85a1cb700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa840004250 0x7fa840004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa85a1cb700 1 -- 172.21.15.31:0/1012453371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400035b0 con 0x7fa840004d90 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa8599ca700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8540c1560 0x7fa840003ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa85a1cb700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa840004d90 0x7fa840005210 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c2f0 tx=0x7fa840005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43772651791062c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c001f30 con 0x7fa840004d90 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c00ce90 con 0x7fa840004d90 2024-03-30T01:42:44.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.235+0000 7fa8589c8700 1 -- 172.21.15.31:0/1012453371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c003c40 con 0x7fa840004d90 2024-03-30T01:42:44.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.236+0000 7fa85bc2e700 1 -- 172.21.15.31:0/1012453371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa840004d90 msgr2=0x7fa840005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.236+0000 7fa85bc2e700 1 --2- 172.21.15.31:0/1012453371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa840004d90 0x7fa840005210 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c2f0 tx=0x7fa840005880 comp rx=0 tx=0).stop 2024-03-30T01:42:44.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.236+0000 7fa85a1cb700 1 -- 172.21.15.31:0/1012453371 reap_dead start 2024-03-30T01:42:44.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.236+0000 7fa85bc2e700 1 -- 172.21.15.31:0/1012453371 shutdown_connections 2024-03-30T01:42:44.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.236+0000 7fa85bc2e700 1 -- 172.21.15.31:0/1012453371 >> 172.21.15.31:0/1012453371 conn(0x7fa85406b110 msgr2=0x7fa8540a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:44.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.236+0000 7fa85bc2e700 1 -- 172.21.15.31:0/1012453371 shutdown_connections 2024-03-30T01:42:44.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.236+0000 7fa85bc2e700 1 -- 172.21.15.31:0/1012453371 wait complete. 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.238+0000 7fa85bc2e700 1 Processor -- start 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.238+0000 7fa85bc2e700 1 -- start start 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa85bc2e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa840004d90 0x7fa8540a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa85bc2e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540a73b0 0x7fa8541e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa85bc2e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8541d8460 0x7fa8541e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa85bc2e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8541dc320 con 0x7fa8540a73b0 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa85bc2e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8541e7f10 con 0x7fa8541d8460 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa85bc2e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa854070530 con 0x7fa840004d90 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8591c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540a73b0 0x7fa8541e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8591c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540a73b0 0x7fa8541e83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35074/0 (socket says 172.21.15.31:35074) 2024-03-30T01:42:44.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8591c9700 1 -- 172.21.15.31:0/4173722111 learned_addr learned my addr 172.21.15.31:0/4173722111 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8599ca700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa840004d90 0x7fa8540a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa85a1cb700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8541d8460 0x7fa8541e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8591c9700 1 -- 172.21.15.31:0/4173722111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa840004d90 msgr2=0x7fa8540a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8591c9700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa840004d90 0x7fa8540a6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8591c9700 1 -- 172.21.15.31:0/4173722111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8541d8460 msgr2=0x7fa8541e88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8591c9700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8541d8460 0x7fa8541e88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8591c9700 1 -- 172.21.15.31:0/4173722111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8541ec0d0 con 0x7fa8540a73b0 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa85a1cb700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8541d8460 0x7fa8541e88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa8591c9700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540a73b0 0x7fa8541e83a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa844004f50 tx=0x7fa8440073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa84a7fc700 1 -- 172.21.15.31:0/4173722111 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844011580 con 0x7fa8540a73b0 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa84a7fc700 1 -- 172.21.15.31:0/4173722111 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa844011720 con 0x7fa8540a73b0 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.239+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8540707f0 con 0x7fa8540a73b0 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.240+0000 7fa84a7fc700 1 -- 172.21.15.31:0/4173722111 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844011ad0 con 0x7fa8540a73b0 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.240+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8541ebcf0 con 0x7fa8540a73b0 2024-03-30T01:42:44.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.240+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8540747d0 con 0x7fa8540a73b0 2024-03-30T01:42:44.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.241+0000 7fa84a7fc700 1 -- 172.21.15.31:0/4173722111 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa844011c30 con 0x7fa8540a73b0 2024-03-30T01:42:44.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.242+0000 7fa84a7fc700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa83806ac90 0x7fa83806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.242+0000 7fa84a7fc700 1 -- 172.21.15.31:0/4173722111 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7fa844089800 con 0x7fa8540a73b0 2024-03-30T01:42:44.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.242+0000 7fa8599ca700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa83806ac90 0x7fa83806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.242+0000 7fa8599ca700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa83806ac90 0x7fa83806d150 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa850009d40 tx=0x7fa850008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:44.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.246+0000 7fa84a7fc700 1 -- 172.21.15.31:0/4173722111 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa844054580 con 0x7fa8540a73b0 2024-03-30T01:42:44.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.459+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8541e8050 con 0x7fa8540a73b0 2024-03-30T01:42:44.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.460+0000 7fa84a7fc700 1 -- 172.21.15.31:0/4173722111 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa844057ba0 con 0x7fa8540a73b0 2024-03-30T01:42:44.465 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:42:44.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa83806ac90 msgr2=0x7fa83806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa83806ac90 0x7fa83806d150 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa850009d40 tx=0x7fa850008040 comp rx=0 tx=0).stop 2024-03-30T01:42:44.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540a73b0 msgr2=0x7fa8541e83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540a73b0 0x7fa8541e83a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa844004f50 tx=0x7fa8440073f0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 shutdown_connections 2024-03-30T01:42:44.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa83806ac90 0x7fa83806d150 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8541d8460 0x7fa8541e88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa840004d90 0x7fa8540a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 --2- 172.21.15.31:0/4173722111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540a73b0 0x7fa8541e83a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 >> 172.21.15.31:0/4173722111 conn(0x7fa85406b110 msgr2=0x7fa8540c4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:44.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 shutdown_connections 2024-03-30T01:42:44.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.462+0000 7fa85bc2e700 1 -- 172.21.15.31:0/4173722111 wait complete. 2024-03-30T01:42:44.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:42:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.615+0000 7f7c24463700 1 Processor -- start 2024-03-30T01:42:44.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.616+0000 7f7c24463700 1 -- start start 2024-03-30T01:42:44.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c24463700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7c1c154d90 0x7f7c1c150b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c24463700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1c151160 con 0x7f7c1c06da30 2024-03-30T01:42:44.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c24463700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1c151440 con 0x7f7c1c0c2830 2024-03-30T01:42:44.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c24463700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1c151720 con 0x7f7c1c154d90 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c221ff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7c1c06da30 0x7f7c1c154680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50734/0 (socket says 172.21.15.31:50734) 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c221ff700 1 -- 172.21.15.31:0/126384101 learned_addr learned my addr 172.21.15.31:0/126384101 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c219fe700 1 --2- 172.21.15.31:0/126384101 >> v2:172.21.15.59:3300/0 conn(0x7f7c1c154d90 0x7f7c1c150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c211fd700 1 -- 172.21.15.31:0/126384101 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022328061 0 0) 0x7f7c1c151160 con 0x7f7c1c06da30 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c211fd700 1 -- 172.21.15.31:0/126384101 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c080035b0 con 0x7f7c1c06da30 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c219fe700 1 -- 172.21.15.31:0/126384101 >> v1:172.21.15.59:6789/0 conn(0x7f7c1c0c2830 legacy=0x7f7c1c14dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c219fe700 1 -- 172.21.15.31:0/126384101 >> v1:172.21.15.31:6789/0 conn(0x7f7c1c06da30 legacy=0x7f7c1c154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.617+0000 7f7c219fe700 1 -- 172.21.15.31:0/126384101 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c151a00 con 0x7f7c1c154d90 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.618+0000 7f7c219fe700 1 --2- 172.21.15.31:0/126384101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c1c154d90 0x7f7c1c150b30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009840 tx=0x7f7c10009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8d3a6acbb7c8661 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.618+0000 7f7c211fd700 1 -- 172.21.15.31:0/126384101 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10011830 con 0x7f7c1c154d90 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.618+0000 7f7c211fd700 1 -- 172.21.15.31:0/126384101 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c10011990 con 0x7f7c1c154d90 2024-03-30T01:42:44.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.618+0000 7f7c211fd700 1 -- 172.21.15.31:0/126384101 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10011d20 con 0x7f7c1c154d90 2024-03-30T01:42:44.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.618+0000 7f7c24463700 1 -- 172.21.15.31:0/126384101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c1c154d90 msgr2=0x7f7c1c150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.618+0000 7f7c24463700 1 --2- 172.21.15.31:0/126384101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c1c154d90 0x7f7c1c150b30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009840 tx=0x7f7c10009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:44.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.619+0000 7f7c24463700 1 -- 172.21.15.31:0/126384101 shutdown_connections 2024-03-30T01:42:44.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.619+0000 7f7c24463700 1 --2- 172.21.15.31:0/126384101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c1c154d90 0x7f7c1c150b30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.619+0000 7f7c24463700 1 -- 172.21.15.31:0/126384101 >> 172.21.15.31:0/126384101 conn(0x7f7c1c0c65c0 msgr2=0x7f7c1c0c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:44.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.619+0000 7f7c24463700 1 -- 172.21.15.31:0/126384101 shutdown_connections 2024-03-30T01:42:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.619+0000 7f7c24463700 1 -- 172.21.15.31:0/126384101 wait complete. 2024-03-30T01:42:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.620+0000 7f7c24463700 1 Processor -- start 2024-03-30T01:42:44.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.620+0000 7f7c24463700 1 -- start start 2024-03-30T01:42:44.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.621+0000 7f7c24463700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c1c06da30 0x7f7c1c1e6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.621+0000 7f7c24463700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c1c0c2830 0x7f7c1c14baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.621+0000 7f7c24463700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c1c154d90 0x7f7c1c14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.621+0000 7f7c24463700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0ac540 con 0x7f7c1c0c2830 2024-03-30T01:42:44.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.621+0000 7f7c24463700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c14f010 con 0x7f7c1c06da30 2024-03-30T01:42:44.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.621+0000 7f7c24463700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c150190 con 0x7f7c1c154d90 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.621+0000 7f7c219fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c1c0c2830 0x7f7c1c14baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c219fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c1c0c2830 0x7f7c1c14baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35078/0 (socket says 172.21.15.31:35078) 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c219fe700 1 -- 172.21.15.31:0/3818140273 learned_addr learned my addr 172.21.15.31:0/3818140273 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c221ff700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c1c06da30 0x7f7c1c1e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c22a00700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c1c154d90 0x7f7c1c14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c219fe700 1 -- 172.21.15.31:0/3818140273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c1c154d90 msgr2=0x7f7c1c14f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c219fe700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c1c154d90 0x7f7c1c14f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c219fe700 1 -- 172.21.15.31:0/3818140273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c1c06da30 msgr2=0x7f7c1c1e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c219fe700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c1c06da30 0x7f7c1c1e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c219fe700 1 -- 172.21.15.31:0/3818140273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c100094a0 con 0x7f7c1c0c2830 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c22a00700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c1c154d90 0x7f7c1c14f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c219fe700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c1c0c2830 0x7f7c1c14baf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009b90 tx=0x7f7c10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c0effd700 1 -- 172.21.15.31:0/3818140273 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c100038a0 con 0x7f7c1c0c2830 2024-03-30T01:42:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c0effd700 1 -- 172.21.15.31:0/3818140273 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1001e070 con 0x7f7c1c0c2830 2024-03-30T01:42:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1c149a60 con 0x7f7c1c0c2830 2024-03-30T01:42:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c0effd700 1 -- 172.21.15.31:0/3818140273 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10003c40 con 0x7f7c1c0c2830 2024-03-30T01:42:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.622+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c14b5d0 con 0x7f7c1c0c2830 2024-03-30T01:42:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.623+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1c06c010 con 0x7f7c1c0c2830 2024-03-30T01:42:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.624+0000 7f7c0effd700 1 -- 172.21.15.31:0/3818140273 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7c1002d450 con 0x7f7c1c0c2830 2024-03-30T01:42:44.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.624+0000 7f7c0effd700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7c0006ac90 0x7f7c0006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.625+0000 7f7c0effd700 1 -- 172.21.15.31:0/3818140273 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f7c1008a8a0 con 0x7f7c1c0c2830 2024-03-30T01:42:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.625+0000 7f7c221ff700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7c0006ac90 0x7f7c0006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.625+0000 7f7c221ff700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7c0006ac90 0x7f7c0006d150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7c180011e0 tx=0x7f7c18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:44.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.629+0000 7f7c0effd700 1 -- 172.21.15.31:0/3818140273 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7c10055500 con 0x7f7c1c0c2830 2024-03-30T01:42:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.844+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c1c14b7c0 con 0x7f7c1c0c2830 2024-03-30T01:42:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.844+0000 7f7c0effd700 1 -- 172.21.15.31:0/3818140273 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c10058b20 con 0x7f7c1c0c2830 2024-03-30T01:42:44.850 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7c0006ac90 msgr2=0x7f7c0006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7c0006ac90 0x7f7c0006d150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7c180011e0 tx=0x7f7c18009040 comp rx=0 tx=0).stop 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c1c0c2830 msgr2=0x7f7c1c14baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c1c0c2830 0x7f7c1c14baf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009b90 tx=0x7f7c10004750 comp rx=0 tx=0).stop 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 shutdown_connections 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7c0006ac90 0x7f7c0006d150 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c1c154d90 0x7f7c1c14f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c1c06da30 0x7f7c1c1e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 --2- 172.21.15.31:0/3818140273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c1c0c2830 0x7f7c1c14baf0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:44.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 >> 172.21.15.31:0/3818140273 conn(0x7f7c1c0c65c0 msgr2=0x7f7c1c0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 shutdown_connections 2024-03-30T01:42:44.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:44.847+0000 7f7c24463700 1 -- 172.21.15.31:0/3818140273 wait complete. 2024-03-30T01:42:44.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:42:45.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.000+0000 7f71ca914700 1 Processor -- start 2024-03-30T01:42:45.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.000+0000 7f71ca914700 1 -- start start 2024-03-30T01:42:45.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71ca914700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71c40c5d40 0x7f71c40c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71ca914700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c4154990 con 0x7f71c4151b60 2024-03-30T01:42:45.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71ca914700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c4154c70 con 0x7f71c4151f40 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71ca914700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c4155610 con 0x7f71c40c5d40 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71c3fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f71c4151b60 0x7f71c4070680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50740/0 (socket says 172.21.15.31:50740) 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71c3fff700 1 -- 172.21.15.31:0/581145038 learned_addr learned my addr 172.21.15.31:0/581145038 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71c37fe700 1 --2- 172.21.15.31:0/581145038 >> v2:172.21.15.59:3300/0 conn(0x7f71c40c5d40 0x7f71c40c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71c2ffd700 1 -- 172.21.15.31:0/581145038 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292094783 0 0) 0x7f71c4154990 con 0x7f71c4151b60 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71c2ffd700 1 -- 172.21.15.31:0/581145038 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b00035b0 con 0x7f71c4151b60 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71c2ffd700 1 -- 172.21.15.31:0/581145038 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858387357 0 0) 0x7f71c4154c70 con 0x7f71c4151f40 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71c2ffd700 1 -- 172.21.15.31:0/581145038 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c4154990 con 0x7f71c4151f40 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71c2ffd700 1 -- 172.21.15.31:0/581145038 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4155092971 0 0) 0x7f71b00035b0 con 0x7f71c4151b60 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.001+0000 7f71c37fe700 1 -- 172.21.15.31:0/581145038 >> v1:172.21.15.59:6789/0 conn(0x7f71c4151f40 legacy=0x7f71c40c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.002+0000 7f71c37fe700 1 -- 172.21.15.31:0/581145038 >> v1:172.21.15.31:6789/0 conn(0x7f71c4151b60 legacy=0x7f71c4070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.002+0000 7f71c37fe700 1 -- 172.21.15.31:0/581145038 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c4154f50 con 0x7f71c40c5d40 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.002+0000 7f71c37fe700 1 --2- 172.21.15.31:0/581145038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71c40c5d40 0x7f71c40c2760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009840 tx=0x7f71ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b28f7a2388fc3d1c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.002+0000 7f71c2ffd700 1 -- 172.21.15.31:0/581145038 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4001670 con 0x7f71c40c5d40 2024-03-30T01:42:45.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.002+0000 7f71c2ffd700 1 -- 172.21.15.31:0/581145038 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac004030 con 0x7f71c40c5d40 2024-03-30T01:42:45.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.002+0000 7f71c2ffd700 1 -- 172.21.15.31:0/581145038 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0043c0 con 0x7f71c40c5d40 2024-03-30T01:42:45.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.002+0000 7f71ca914700 1 -- 172.21.15.31:0/581145038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71c40c5d40 msgr2=0x7f71c40c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.003+0000 7f71ca914700 1 --2- 172.21.15.31:0/581145038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71c40c5d40 0x7f71c40c2760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009840 tx=0x7f71ac009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:45.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.003+0000 7f71ca914700 1 -- 172.21.15.31:0/581145038 shutdown_connections 2024-03-30T01:42:45.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.003+0000 7f71ca914700 1 --2- 172.21.15.31:0/581145038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71c40c5d40 0x7f71c40c2760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.003+0000 7f71ca914700 1 -- 172.21.15.31:0/581145038 >> 172.21.15.31:0/581145038 conn(0x7f71c40c7d60 msgr2=0x7f71c40c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:45.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.003+0000 7f71ca914700 1 -- 172.21.15.31:0/581145038 shutdown_connections 2024-03-30T01:42:45.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.003+0000 7f71ca914700 1 -- 172.21.15.31:0/581145038 wait complete. 2024-03-30T01:42:45.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.004+0000 7f71ca914700 1 Processor -- start 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.005+0000 7f71ca914700 1 -- start start 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.005+0000 7f71ca914700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71c40c5d40 0x7f71c40ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.005+0000 7f71ca914700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4151b60 0x7f71c40aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.005+0000 7f71ca914700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71c4151f40 0x7f71c40a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.005+0000 7f71ca914700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c406cf30 con 0x7f71c4151b60 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.005+0000 7f71ca914700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c406cc50 con 0x7f71c4151f40 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.005+0000 7f71ca914700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c41ecff0 con 0x7f71c40c5d40 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4151b60 0x7f71c40aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4151b60 0x7f71c40aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35084/0 (socket says 172.21.15.31:35084) 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c37fe700 1 -- 172.21.15.31:0/3988232849 learned_addr learned my addr 172.21.15.31:0/3988232849 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c3fff700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71c40c5d40 0x7f71c40ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c8eb1700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71c4151f40 0x7f71c40a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c37fe700 1 -- 172.21.15.31:0/3988232849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71c40c5d40 msgr2=0x7f71c40ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c37fe700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71c40c5d40 0x7f71c40ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c37fe700 1 -- 172.21.15.31:0/3988232849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71c4151f40 msgr2=0x7f71c40a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c37fe700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71c4151f40 0x7f71c40a9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c37fe700 1 -- 172.21.15.31:0/3988232849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0094a0 con 0x7f71c4151b60 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c37fe700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4151b60 0x7f71c40aac00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00a010 tx=0x7f71ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c0ff9700 1 -- 172.21.15.31:0/3988232849 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac003c50 con 0x7f71c4151b60 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c0ff9700 1 -- 172.21.15.31:0/3988232849 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac01f070 con 0x7f71c4151b60 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c41ed2d0 con 0x7f71c4151b60 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71c0ff9700 1 -- 172.21.15.31:0/3988232849 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac011630 con 0x7f71c4151b60 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.006+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c40ac850 con 0x7f71c4151b60 2024-03-30T01:42:45.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.007+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c41ece50 con 0x7f71c4151b60 2024-03-30T01:42:45.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.008+0000 7f71c0ff9700 1 -- 172.21.15.31:0/3988232849 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f71ac021030 con 0x7f71c4151b60 2024-03-30T01:42:45.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.008+0000 7f71c0ff9700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f71a406acc0 0x7f71a406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.008+0000 7f71c0ff9700 1 -- 172.21.15.31:0/3988232849 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f71ac08aa20 con 0x7f71c4151b60 2024-03-30T01:42:45.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.009+0000 7f71c3fff700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f71a406acc0 0x7f71a406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.009+0000 7f71c3fff700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f71a406acc0 0x7f71a406d180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f71b4001380 tx=0x7f71b4000f10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:45.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.013+0000 7f71c0ff9700 1 -- 172.21.15.31:0/3988232849 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f71c41ece50 con 0x7f71c4151b60 2024-03-30T01:42:45.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.224+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f71c41ed7f0 con 0x7f71c4151b60 2024-03-30T01:42:45.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.225+0000 7f71c0ff9700 1 -- 172.21.15.31:0/3988232849 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f71c41ed7f0 con 0x7f71c4151b60 2024-03-30T01:42:45.231 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f71a406acc0 msgr2=0x7f71a406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f71a406acc0 0x7f71a406d180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f71b4001380 tx=0x7f71b4000f10 comp rx=0 tx=0).stop 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4151b60 msgr2=0x7f71c40aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4151b60 0x7f71c40aac00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00a010 tx=0x7f71ac004750 comp rx=0 tx=0).stop 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 shutdown_connections 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f71a406acc0 0x7f71a406d180 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71c4151f40 0x7f71c40a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71c40c5d40 0x7f71c40ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 --2- 172.21.15.31:0/3988232849 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4151b60 0x7f71c40aac00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 >> 172.21.15.31:0/3988232849 conn(0x7f71c40c7d60 msgr2=0x7f71c406fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:45.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 shutdown_connections 2024-03-30T01:42:45.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.227+0000 7f71ca914700 1 -- 172.21.15.31:0/3988232849 wait complete. 2024-03-30T01:42:45.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-30T01:42:45.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.381+0000 7fb8c90e1700 1 Processor -- start 2024-03-30T01:42:45.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.381+0000 7fb8c90e1700 1 -- start start 2024-03-30T01:42:45.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.381+0000 7fb8c90e1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414fb40 0x7fb8c40c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.381+0000 7fb8c90e1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb8c40a7600 0x7fb8c4152f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.381+0000 7fb8c90e1700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c40c4620 con 0x7fb8c414ff20 2024-03-30T01:42:45.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.381+0000 7fb8c90e1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c4152570 con 0x7fb8c40a7600 2024-03-30T01:42:45.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.381+0000 7fb8c90e1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4152ad0 con 0x7fb8c414fb40 2024-03-30T01:42:45.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c359e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb8c414ff20 0x7fb8c40c47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36890/0 (socket says 172.21.15.31:36890) 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c359e700 1 -- 172.21.15.31:0/2902910580 learned_addr learned my addr 172.21.15.31:0/2902910580 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c2d9d700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414fb40 0x7fb8c40c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c259c700 1 --2- 172.21.15.31:0/2902910580 >> v2:172.21.15.59:3300/0 conn(0x7fb8c40a7600 0x7fb8c4152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031338397 0 0) 0x7fb8bc001490 con 0x7fb8c414ff20 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ac0035b0 con 0x7fb8c414ff20 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1225070744 0 0) 0x7fb8ac0035b0 con 0x7fb8c414ff20 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414fb40 msgr2=0x7fb8c40c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414fb40 0x7fb8c40c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 >> v2:172.21.15.59:3300/0 conn(0x7fb8c40a7600 msgr2=0x7fb8c4152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 --2- 172.21.15.31:0/2902910580 >> v2:172.21.15.59:3300/0 conn(0x7fb8c40a7600 0x7fb8c4152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40c2ca0 con 0x7fb8c414ff20 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7fb8bc001b80 con 0x7fb8c414ff20 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 >> v1:172.21.15.31:6789/0 conn(0x7fb8c414ff20 legacy=0x7fb8c40c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8ac003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414fb40 0x7fb8ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac004d90 0x7fb8ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c4152ad0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac00ba10 con 0x7fb8ac004d90 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac00bb80 con 0x7fb8c414fb40 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.382+0000 7fb8c2d9d700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8ac003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c259c700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414fb40 0x7fb8ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c359e700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac004d90 0x7fb8ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c2d9d700 1 -- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414fb40 msgr2=0x7fb8ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c2d9d700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414fb40 0x7fb8ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c2d9d700 1 -- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac004d90 msgr2=0x7fb8ac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c2d9d700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac004d90 0x7fb8ac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c2d9d700 1 -- 172.21.15.31:0/2902910580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac0035b0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c359e700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8ac004d90 0x7fb8ac005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:45.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c2d9d700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8ac003f00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009b60 tx=0x7fb8ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ff40abafe21381 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400e6c0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b400e820 con 0x7fb8ac003ae0 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.383+0000 7fb8c1d9b700 1 -- 172.21.15.31:0/2902910580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400eb90 con 0x7fb8ac003ae0 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.384+0000 7fb8c90e1700 1 -- 172.21.15.31:0/2902910580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 msgr2=0x7fb8ac003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.384+0000 7fb8c90e1700 1 --2- 172.21.15.31:0/2902910580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8ac003f00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009b60 tx=0x7fb8ac005880 comp rx=0 tx=0).stop 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.384+0000 7fb8c359e700 1 -- 172.21.15.31:0/2902910580 reap_dead start 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.384+0000 7fb8c90e1700 1 -- 172.21.15.31:0/2902910580 shutdown_connections 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.384+0000 7fb8c90e1700 1 -- 172.21.15.31:0/2902910580 >> 172.21.15.31:0/2902910580 conn(0x7fb8c40c7d60 msgr2=0x7fb8c40c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.384+0000 7fb8c90e1700 1 -- 172.21.15.31:0/2902910580 shutdown_connections 2024-03-30T01:42:45.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.384+0000 7fb8c90e1700 1 -- 172.21.15.31:0/2902910580 wait complete. 2024-03-30T01:42:45.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.385+0000 7fb8c90e1700 1 Processor -- start 2024-03-30T01:42:45.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.385+0000 7fb8c90e1700 1 -- start start 2024-03-30T01:42:45.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c90e1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8c414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c90e1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414b1d0 0x7fb8c414a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c90e1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8c414abf0 0x7fb8c41eb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c90e1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c414b070 con 0x7fb8ac003ae0 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c90e1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40bcd80 con 0x7fb8c414abf0 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c90e1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40c2060 con 0x7fb8c414b1d0 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8c414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8c414d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35004/0 (socket says 172.21.15.31:35004) 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c2d9d700 1 -- 172.21.15.31:0/175770450 learned_addr learned my addr 172.21.15.31:0/175770450 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c259c700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414b1d0 0x7fb8c414a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c359e700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8c414abf0 0x7fb8c41eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c2d9d700 1 -- 172.21.15.31:0/175770450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414b1d0 msgr2=0x7fb8c414a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c2d9d700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414b1d0 0x7fb8c414a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c2d9d700 1 -- 172.21.15.31:0/175770450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8c414abf0 msgr2=0x7fb8c41eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c2d9d700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8c414abf0 0x7fb8c41eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.386+0000 7fb8c2d9d700 1 -- 172.21.15.31:0/175770450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40bcfd0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.387+0000 7fb8c2d9d700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8c414d750 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400e4b0 tx=0x7fb8b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.387+0000 7fb8ab7fe700 1 -- 172.21.15.31:0/175770450 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400e6c0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.387+0000 7fb8ab7fe700 1 -- 172.21.15.31:0/175770450 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b4003ad0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.387+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c40bd2b0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.387+0000 7fb8ab7fe700 1 -- 172.21.15.31:0/175770450 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4003e80 con 0x7fb8ac003ae0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.387+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c414dc90 con 0x7fb8ac003ae0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.388+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c414afd0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.388+0000 7fb8ab7fe700 1 -- 172.21.15.31:0/175770450 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb8b4013470 con 0x7fb8ac003ae0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.389+0000 7fb8ab7fe700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb8b006ac90 0x7fb8b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.389+0000 7fb8ab7fe700 1 -- 172.21.15.31:0/175770450 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7fb8b408b460 con 0x7fb8ac003ae0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.389+0000 7fb8c259c700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb8b006ac90 0x7fb8b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.390+0000 7fb8c259c700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb8b006ac90 0x7fb8b006d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40a6cd0 tx=0x7fb8b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:45.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.394+0000 7fb8ab7fe700 1 -- 172.21.15.31:0/175770450 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb8b40563c0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.607+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8c41eb260 con 0x7fb8ac003ae0 2024-03-30T01:42:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.607+0000 7fb8ab7fe700 1 -- 172.21.15.31:0/175770450 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v45) v1 ==== 105+0+11 (secure 0 0 0) 0x7fb8b40599e0 con 0x7fb8ac003ae0 2024-03-30T01:42:45.613 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb8b006ac90 msgr2=0x7fb8b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb8b006ac90 0x7fb8b006d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40a6cd0 tx=0x7fb8b8008040 comp rx=0 tx=0).stop 2024-03-30T01:42:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 msgr2=0x7fb8c414d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8c414d750 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400e4b0 tx=0x7fb8b40047c0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 shutdown_connections 2024-03-30T01:42:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8ac003ae0 0x7fb8c414d750 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb8b006ac90 0x7fb8b006d150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8c414abf0 0x7fb8c41eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 --2- 172.21.15.31:0/175770450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8c414b1d0 0x7fb8c414a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 >> 172.21.15.31:0/175770450 conn(0x7fb8c40c7d60 msgr2=0x7fb8c40ac200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:45.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 shutdown_connections 2024-03-30T01:42:45.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.610+0000 7fb8c90e1700 1 -- 172.21.15.31:0/175770450 wait complete. 2024-03-30T01:42:45.629 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-30T01:42:45.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.762+0000 7f9c0abb3700 1 Processor -- start 2024-03-30T01:42:45.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.762+0000 7f9c0abb3700 1 -- start start 2024-03-30T01:42:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c0abb3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c0406d040 0x7f9c04072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c0abb3700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0414ee60 con 0x7f9c041549a0 2024-03-30T01:42:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c0abb3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c040c2290 con 0x7f9c0406d900 2024-03-30T01:42:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c0abb3700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c04155490 con 0x7f9c0406d040 2024-03-30T01:42:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c03fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c0406d040 0x7f9c04072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c0894f700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9c041549a0 0x7f9c04154d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36902/0 (socket says 172.21.15.31:36902) 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c0894f700 1 -- 172.21.15.31:0/1710531602 learned_addr learned my addr 172.21.15.31:0/1710531602 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c03fff700 1 -- 172.21.15.31:0/1710531602 >> v1:172.21.15.59:6789/0 conn(0x7f9c0406d900 legacy=0x7f9c0406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c03fff700 1 -- 172.21.15.31:0/1710531602 >> v1:172.21.15.31:6789/0 conn(0x7f9c041549a0 legacy=0x7f9c04154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.763+0000 7f9c03fff700 1 -- 172.21.15.31:0/1710531602 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c04155720 con 0x7f9c0406d040 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c03fff700 1 --2- 172.21.15.31:0/1710531602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d040 0x7f9c04072410 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009840 tx=0x7f9bf0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c7b7308a7c814d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c037fe700 1 -- 172.21.15.31:0/1710531602 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0011830 con 0x7f9c0406d040 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c037fe700 1 -- 172.21.15.31:0/1710531602 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0011990 con 0x7f9c0406d040 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c037fe700 1 -- 172.21.15.31:0/1710531602 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0011d20 con 0x7f9c0406d040 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1710531602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d040 msgr2=0x7f9c04072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c0abb3700 1 --2- 172.21.15.31:0/1710531602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d040 0x7f9c04072410 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009840 tx=0x7f9bf0009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1710531602 shutdown_connections 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c0abb3700 1 --2- 172.21.15.31:0/1710531602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d040 0x7f9c04072410 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1710531602 >> 172.21.15.31:0/1710531602 conn(0x7f9c040c3980 msgr2=0x7f9c040c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1710531602 shutdown_connections 2024-03-30T01:42:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.764+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1710531602 wait complete. 2024-03-30T01:42:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.766+0000 7f9c0abb3700 1 Processor -- start 2024-03-30T01:42:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.766+0000 7f9c0abb3700 1 -- start start 2024-03-30T01:42:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c0abb3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c0406d040 0x7f9c0414a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c0abb3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d900 0x7f9c041daa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c0abb3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c041549a0 0x7f9c04144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c0abb3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041e9300 con 0x7f9c0406d900 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c0abb3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041db7e0 con 0x7f9c041549a0 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c0abb3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040abf20 con 0x7f9c0406d040 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d900 0x7f9c041daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c03fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d900 0x7f9c041daa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35020/0 (socket says 172.21.15.31:35020) 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c03fff700 1 -- 172.21.15.31:0/1595120150 learned_addr learned my addr 172.21.15.31:0/1595120150 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c0894f700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c0406d040 0x7f9c0414a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c09150700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c041549a0 0x7f9c04144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c03fff700 1 -- 172.21.15.31:0/1595120150 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c0406d040 msgr2=0x7f9c0414a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c03fff700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c0406d040 0x7f9c0414a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c03fff700 1 -- 172.21.15.31:0/1595120150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c041549a0 msgr2=0x7f9c04144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c03fff700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c041549a0 0x7f9c04144590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c03fff700 1 -- 172.21.15.31:0/1595120150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00094a0 con 0x7f9c0406d900 2024-03-30T01:42:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c0894f700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c0406d040 0x7f9c0414a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c03fff700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d900 0x7f9c041daa10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00090b0 tx=0x7f9bf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c017fa700 1 -- 172.21.15.31:0/1595120150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00038a0 con 0x7f9c0406d900 2024-03-30T01:42:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c017fa700 1 -- 172.21.15.31:0/1595120150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf001f070 con 0x7f9c0406d900 2024-03-30T01:42:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c041dd710 con 0x7f9c0406d900 2024-03-30T01:42:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.767+0000 7f9c017fa700 1 -- 172.21.15.31:0/1595120150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0003c60 con 0x7f9c0406d900 2024-03-30T01:42:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.768+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0406e9c0 con 0x7f9c0406d900 2024-03-30T01:42:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.768+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c040747d0 con 0x7f9c0406d900 2024-03-30T01:42:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.769+0000 7f9c017fa700 1 -- 172.21.15.31:0/1595120150 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9bf001a430 con 0x7f9c0406d900 2024-03-30T01:42:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.769+0000 7f9c017fa700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9be806ac90 0x7f9be806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.770+0000 7f9c017fa700 1 -- 172.21.15.31:0/1595120150 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f9bf0021070 con 0x7f9c0406d900 2024-03-30T01:42:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.770+0000 7f9c0894f700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9be806ac90 0x7f9be806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.770+0000 7f9c0894f700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9be806ac90 0x7f9be806d150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005910 tx=0x7f9bf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:45.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.774+0000 7f9c017fa700 1 -- 172.21.15.31:0/1595120150 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9bf0055f10 con 0x7f9c0406d900 2024-03-30T01:42:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.986+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f9c040c1fd0 con 0x7f9c0406d900 2024-03-30T01:42:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.986+0000 7f9c017fa700 1 -- 172.21.15.31:0/1595120150 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v45) v1 ==== 101+0+11 (secure 0 0 0) 0x7f9bf0059530 con 0x7f9c0406d900 2024-03-30T01:42:45.992 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.988+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9be806ac90 msgr2=0x7f9be806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.988+0000 7f9c0abb3700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9be806ac90 0x7f9be806d150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005910 tx=0x7f9bf8009040 comp rx=0 tx=0).stop 2024-03-30T01:42:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.988+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d900 msgr2=0x7f9c041daa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.988+0000 7f9c0abb3700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d900 0x7f9c041daa10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00090b0 tx=0x7f9bf0004750 comp rx=0 tx=0).stop 2024-03-30T01:42:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.989+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 shutdown_connections 2024-03-30T01:42:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.989+0000 7f9c0abb3700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9be806ac90 0x7f9be806d150 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.989+0000 7f9c0abb3700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c041549a0 0x7f9c04144590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.989+0000 7f9c0abb3700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c0406d040 0x7f9c0414a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.989+0000 7f9c0abb3700 1 --2- 172.21.15.31:0/1595120150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c0406d900 0x7f9c041daa10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.989+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 >> 172.21.15.31:0/1595120150 conn(0x7f9c040c3980 msgr2=0x7f9c04154d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.989+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 shutdown_connections 2024-03-30T01:42:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:45.989+0000 7f9c0abb3700 1 -- 172.21.15.31:0/1595120150 wait complete. 2024-03-30T01:42:46.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T01:42:46.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.139+0000 7f48a38c2700 1 Processor -- start 2024-03-30T01:42:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.139+0000 7f48a38c2700 1 -- start start 2024-03-30T01:42:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.140+0000 7f48a38c2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f489c15c160 0x7f489c15c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.140+0000 7f48a38c2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f489c15caf0 0x7f489c14ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.140+0000 7f48a38c2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489c158bd0 con 0x7f489c0c3e30 2024-03-30T01:42:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.140+0000 7f48a38c2700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489c158eb0 con 0x7f489c15caf0 2024-03-30T01:42:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.140+0000 7f48a38c2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489c159190 con 0x7f489c15c160 2024-03-30T01:42:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.140+0000 7f48a1e5f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f489c15caf0 0x7f489c14ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.140+0000 7f48a1e5f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f489c15caf0 0x7f489c14ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35034/0 (socket says 172.21.15.31:35034) 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.140+0000 7f48a1e5f700 1 -- 172.21.15.31:0/3201907013 learned_addr learned my addr 172.21.15.31:0/3201907013 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.140+0000 7f48a0e5d700 1 --2- 172.21.15.31:0/3201907013 >> v2:172.21.15.59:3300/0 conn(0x7f489c15c160 0x7f489c15c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f48a1e5f700 1 -- 172.21.15.31:0/3201907013 >> v2:172.21.15.59:3300/0 conn(0x7f489c15c160 msgr2=0x7f489c15c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f48a1e5f700 1 --2- 172.21.15.31:0/3201907013 >> v2:172.21.15.59:3300/0 conn(0x7f489c15c160 0x7f489c15c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f48a1e5f700 1 -- 172.21.15.31:0/3201907013 >> v1:172.21.15.59:6789/0 conn(0x7f489c0c3e30 legacy=0x7f489c0c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f48a1e5f700 1 -- 172.21.15.31:0/3201907013 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c159470 con 0x7f489c15caf0 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f48a0e5d700 1 --2- 172.21.15.31:0/3201907013 >> v2:172.21.15.59:3300/0 conn(0x7f489c15c160 0x7f489c15c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f48a1e5f700 1 --2- 172.21.15.31:0/3201907013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c15caf0 0x7f489c14ea70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f489c0c3540 tx=0x7f489400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c9720e9f9dcf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f488ffff700 1 -- 172.21.15.31:0/3201907013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400e070 con 0x7f489c15caf0 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f488ffff700 1 -- 172.21.15.31:0/3201907013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489400b9c0 con 0x7f489c15caf0 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f488ffff700 1 -- 172.21.15.31:0/3201907013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400bd70 con 0x7f489c15caf0 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f48a38c2700 1 -- 172.21.15.31:0/3201907013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c15caf0 msgr2=0x7f489c14ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.141+0000 7f48a38c2700 1 --2- 172.21.15.31:0/3201907013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c15caf0 0x7f489c14ea70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f489c0c3540 tx=0x7f489400b330 comp rx=0 tx=0).stop 2024-03-30T01:42:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.142+0000 7f48a38c2700 1 -- 172.21.15.31:0/3201907013 shutdown_connections 2024-03-30T01:42:46.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.142+0000 7f48a38c2700 1 --2- 172.21.15.31:0/3201907013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c15caf0 0x7f489c14ea70 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.142+0000 7f48a38c2700 1 --2- 172.21.15.31:0/3201907013 >> v2:172.21.15.59:3300/0 conn(0x7f489c15c160 0x7f489c15c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.142+0000 7f48a38c2700 1 -- 172.21.15.31:0/3201907013 >> 172.21.15.31:0/3201907013 conn(0x7f489c06ba20 msgr2=0x7f489c06be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:46.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.142+0000 7f48a38c2700 1 -- 172.21.15.31:0/3201907013 shutdown_connections 2024-03-30T01:42:46.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.142+0000 7f48a38c2700 1 -- 172.21.15.31:0/3201907013 wait complete. 2024-03-30T01:42:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.143+0000 7f48a38c2700 1 Processor -- start 2024-03-30T01:42:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.143+0000 7f48a38c2700 1 -- start start 2024-03-30T01:42:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a38c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0c3e30 0x7f489c15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a38c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f489c15c160 0x7f489c1eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a38c2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f489c15caf0 0x7f489c1eb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a38c2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c152b90 con 0x7f489c0c3e30 2024-03-30T01:42:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a38c2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c1f1280 con 0x7f489c15c160 2024-03-30T01:42:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a38c2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c1eec20 con 0x7f489c15caf0 2024-03-30T01:42:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a165e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0c3e30 0x7f489c15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a165e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0c3e30 0x7f489c15a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35038/0 (socket says 172.21.15.31:35038) 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a165e700 1 -- 172.21.15.31:0/229171499 learned_addr learned my addr 172.21.15.31:0/229171499 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a0e5d700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f489c15c160 0x7f489c1eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a1e5f700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f489c15caf0 0x7f489c1eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a165e700 1 -- 172.21.15.31:0/229171499 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f489c15caf0 msgr2=0x7f489c1eb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a165e700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f489c15caf0 0x7f489c1eb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a165e700 1 -- 172.21.15.31:0/229171499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f489c15c160 msgr2=0x7f489c1eaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a165e700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f489c15c160 0x7f489c1eaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a165e700 1 -- 172.21.15.31:0/229171499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894009b30 con 0x7f489c0c3e30 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a1e5f700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f489c15caf0 0x7f489c1eb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a165e700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0c3e30 0x7f489c15a400 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f489800c490 tx=0x7f489800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f488dffb700 1 -- 172.21.15.31:0/229171499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48980052d0 con 0x7f489c0c3e30 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f488dffb700 1 -- 172.21.15.31:0/229171499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898005450 con 0x7f489c0c3e30 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489c151950 con 0x7f489c0c3e30 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.144+0000 7f488dffb700 1 -- 172.21.15.31:0/229171499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898006aa0 con 0x7f489c0c3e30 2024-03-30T01:42:46.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.145+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c151f10 con 0x7f489c0c3e30 2024-03-30T01:42:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.146+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c04ea80 con 0x7f489c0c3e30 2024-03-30T01:42:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.146+0000 7f488dffb700 1 -- 172.21.15.31:0/229171499 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4898003680 con 0x7f489c0c3e30 2024-03-30T01:42:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.147+0000 7f488dffb700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f488006ac90 0x7f488006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.147+0000 7f488dffb700 1 -- 172.21.15.31:0/229171499 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f4898088a90 con 0x7f489c0c3e30 2024-03-30T01:42:46.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.147+0000 7f48a0e5d700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f488006ac90 0x7f488006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.147+0000 7f48a0e5d700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f488006ac90 0x7f488006d150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f489c151770 tx=0x7f4890008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.152+0000 7f488dffb700 1 -- 172.21.15.31:0/229171499 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4898053770 con 0x7f489c0c3e30 2024-03-30T01:42:46.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.383+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f489c071660 con 0x7f489c0c3e30 2024-03-30T01:42:46.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.383+0000 7f488dffb700 1 -- 172.21.15.31:0/229171499 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4898056d90 con 0x7f489c0c3e30 2024-03-30T01:42:46.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f488006ac90 msgr2=0x7f488006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f488006ac90 0x7f488006d150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f489c151770 tx=0x7f4890008040 comp rx=0 tx=0).stop 2024-03-30T01:42:46.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0c3e30 msgr2=0x7f489c15a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0c3e30 0x7f489c15a400 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f489800c490 tx=0x7f489800c7a0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 shutdown_connections 2024-03-30T01:42:46.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0c3e30 0x7f489c15a400 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f488006ac90 0x7f488006d150 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f489c15caf0 0x7f489c1eb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 --2- 172.21.15.31:0/229171499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f489c15c160 0x7f489c1eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 >> 172.21.15.31:0/229171499 conn(0x7f489c06ba20 msgr2=0x7f489c15b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:46.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 shutdown_connections 2024-03-30T01:42:46.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.385+0000 7f48a38c2700 1 -- 172.21.15.31:0/229171499 wait complete. 2024-03-30T01:42:46.392 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-03-30T01:42:46.404 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-30T01:42:46.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:46.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-30T01:42:46.457 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:42:46.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:42:46.458 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T01:42:46.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:46.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.616+0000 7f61c9ad9700 1 Processor -- start 2024-03-30T01:42:46.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.616+0000 7f61c9ad9700 1 -- start start 2024-03-30T01:42:46.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.617+0000 7f61c9ad9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61c40c46a0 0x7f61c40c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.617+0000 7f61c9ad9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f61c4072270 0x7f61c4151490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.617+0000 7f61c9ad9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40a8460 con 0x7f61c4154cf0 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.617+0000 7f61c9ad9700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c406dcb0 con 0x7f61c4072270 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.617+0000 7f61c9ad9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c40c2e40 con 0x7f61c40c46a0 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.617+0000 7f61c3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f61c4072270 0x7f61c4151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.617+0000 7f61c3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f61c4072270 0x7f61c4151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35048/0 (socket says 172.21.15.31:35048) 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.617+0000 7f61c3fff700 1 -- 172.21.15.31:0/2454833474 learned_addr learned my addr 172.21.15.31:0/2454833474 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.617+0000 7f61c2ffd700 1 --2- 172.21.15.31:0/2454833474 >> v2:172.21.15.59:3300/0 conn(0x7f61c40c46a0 0x7f61c40c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.618+0000 7f61c3fff700 1 -- 172.21.15.31:0/2454833474 >> v2:172.21.15.59:3300/0 conn(0x7f61c40c46a0 msgr2=0x7f61c40c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.618+0000 7f61c3fff700 1 --2- 172.21.15.31:0/2454833474 >> v2:172.21.15.59:3300/0 conn(0x7f61c40c46a0 0x7f61c40c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.618+0000 7f61c3fff700 1 -- 172.21.15.31:0/2454833474 >> v1:172.21.15.59:6789/0 conn(0x7f61c4154cf0 legacy=0x7f61c406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.618+0000 7f61c3fff700 1 -- 172.21.15.31:0/2454833474 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4155630 con 0x7f61c4072270 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.618+0000 7f61c3fff700 1 --2- 172.21.15.31:0/2454833474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4072270 0x7f61c4151490 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ad2d0 tx=0x7f61bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e3986480c5d1a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.618+0000 7f61c27fc700 1 -- 172.21.15.31:0/2454833474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00e070 con 0x7f61c4072270 2024-03-30T01:42:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.618+0000 7f61c27fc700 1 -- 172.21.15.31:0/2454833474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc00b9c0 con 0x7f61c4072270 2024-03-30T01:42:46.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.618+0000 7f61c27fc700 1 -- 172.21.15.31:0/2454833474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00bd70 con 0x7f61c4072270 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.619+0000 7f61c9ad9700 1 -- 172.21.15.31:0/2454833474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4072270 msgr2=0x7f61c4151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.619+0000 7f61c9ad9700 1 --2- 172.21.15.31:0/2454833474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4072270 0x7f61c4151490 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ad2d0 tx=0x7f61bc00b330 comp rx=0 tx=0).stop 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.619+0000 7f61c9ad9700 1 -- 172.21.15.31:0/2454833474 shutdown_connections 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.619+0000 7f61c9ad9700 1 --2- 172.21.15.31:0/2454833474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4072270 0x7f61c4151490 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.619+0000 7f61c9ad9700 1 --2- 172.21.15.31:0/2454833474 >> v2:172.21.15.59:3300/0 conn(0x7f61c40c46a0 0x7f61c40c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.619+0000 7f61c9ad9700 1 -- 172.21.15.31:0/2454833474 >> 172.21.15.31:0/2454833474 conn(0x7f61c40c3a60 msgr2=0x7f61c40c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.619+0000 7f61c9ad9700 1 -- 172.21.15.31:0/2454833474 shutdown_connections 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.619+0000 7f61c9ad9700 1 -- 172.21.15.31:0/2454833474 wait complete. 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.620+0000 7f61c9ad9700 1 Processor -- start 2024-03-30T01:42:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.620+0000 7f61c9ad9700 1 -- start start 2024-03-30T01:42:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c9ad9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61c4072270 0x7f61c406a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c9ad9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61c40c46a0 0x7f61c4143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c9ad9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4154cf0 0x7f61c4143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c9ad9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40aa760 con 0x7f61c4154cf0 2024-03-30T01:42:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c9ad9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c41e53c0 con 0x7f61c4072270 2024-03-30T01:42:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c9ad9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4152740 con 0x7f61c40c46a0 2024-03-30T01:42:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4154cf0 0x7f61c4143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4154cf0 0x7f61c4143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35064/0 (socket says 172.21.15.31:35064) 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c3fff700 1 -- 172.21.15.31:0/1552293287 learned_addr learned my addr 172.21.15.31:0/1552293287 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c37fe700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61c4072270 0x7f61c406a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c2ffd700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61c40c46a0 0x7f61c4143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c3fff700 1 -- 172.21.15.31:0/1552293287 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61c40c46a0 msgr2=0x7f61c4143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c3fff700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61c40c46a0 0x7f61c4143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c3fff700 1 -- 172.21.15.31:0/1552293287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61c4072270 msgr2=0x7f61c406a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c3fff700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61c4072270 0x7f61c406a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c3fff700 1 -- 172.21.15.31:0/1552293287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc009b30 con 0x7f61c4154cf0 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c2ffd700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61c40c46a0 0x7f61c4143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.621+0000 7f61c3fff700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4154cf0 0x7f61c4143ba0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f61c4073220 tx=0x7f61bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.622+0000 7f61abfff700 1 -- 172.21.15.31:0/1552293287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0092e0 con 0x7f61c4154cf0 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.622+0000 7f61abfff700 1 -- 172.21.15.31:0/1552293287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc01c440 con 0x7f61c4154cf0 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.622+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c4152a20 con 0x7f61c4154cf0 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.622+0000 7f61abfff700 1 -- 172.21.15.31:0/1552293287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc01c7f0 con 0x7f61c4154cf0 2024-03-30T01:42:46.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.622+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c40c6c90 con 0x7f61c4154cf0 2024-03-30T01:42:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.623+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c41525a0 con 0x7f61c4154cf0 2024-03-30T01:42:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.623+0000 7f61abfff700 1 -- 172.21.15.31:0/1552293287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f61bc01c950 con 0x7f61c4154cf0 2024-03-30T01:42:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.624+0000 7f61abfff700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f61b006ac90 0x7f61b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.624+0000 7f61abfff700 1 -- 172.21.15.31:0/1552293287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f61bc08b1f0 con 0x7f61c4154cf0 2024-03-30T01:42:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.624+0000 7f61c37fe700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f61b006ac90 0x7f61b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:46.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.624+0000 7f61c37fe700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f61b006ac90 0x7f61b006d150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f61b4000f60 tx=0x7f61b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:46.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.628+0000 7f61abfff700 1 -- 172.21.15.31:0/1552293287 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f61c41525a0 con 0x7f61c4154cf0 2024-03-30T01:42:46.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.891+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61c41525a0 con 0x7f61c4154cf0 2024-03-30T01:42:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.892+0000 7f61abfff700 1 -- 172.21.15.31:0/1552293287 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5942 (secure 0 0 0) 0x7f61c41525a0 con 0x7f61c4154cf0 2024-03-30T01:42:46.897 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:46.897 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:10.264695+0000","modified":"2024-03-30T01:42:13.278071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/1616247357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1616247357},{"type":"v1","addr":"172.21.15.31:6837","nonce":1616247357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:24.267501+0000","modified":"2024-03-30T01:42:27.273615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f61b006ac90 msgr2=0x7f61b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f61b006ac90 0x7f61b006d150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f61b4000f60 tx=0x7f61b4009040 comp rx=0 tx=0).stop 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4154cf0 msgr2=0x7f61c4143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4154cf0 0x7f61c4143ba0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f61c4073220 tx=0x7f61bc004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 shutdown_connections 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61c4154cf0 0x7f61c4143ba0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f61b006ac90 0x7f61b006d150 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61c4072270 0x7f61c406a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 --2- 172.21.15.31:0/1552293287 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61c40c46a0 0x7f61c4143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 >> 172.21.15.31:0/1552293287 conn(0x7f61c40c3a60 msgr2=0x7f61c40c16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 shutdown_connections 2024-03-30T01:42:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:46.894+0000 7f61c9ad9700 1 -- 172.21.15.31:0/1552293287 wait complete. 2024-03-30T01:42:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-03-30T01:42:46.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:47.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.044+0000 7f441f6a0700 1 Processor -- start 2024-03-30T01:42:47.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.044+0000 7f441f6a0700 1 -- start start 2024-03-30T01:42:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.045+0000 7f441f6a0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f441814e5f0 0x7f44180c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.045+0000 7f441f6a0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4418071240 0x7f4418154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.045+0000 7f441f6a0700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4418072160 con 0x7f44180c4100 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.045+0000 7f441f6a0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44180c2bd0 con 0x7f441814e5f0 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.045+0000 7f441f6a0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44180c2e90 con 0x7f4418071240 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f441dc3d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f44180c4100 0x7f4418154dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36904/0 (socket says 172.21.15.31:36904) 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f441dc3d700 1 -- 172.21.15.31:0/4187145322 learned_addr learned my addr 172.21.15.31:0/4187145322 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f441d43c700 1 --2- 172.21.15.31:0/4187145322 >> v2:172.21.15.59:3300/0 conn(0x7f441814e5f0 0x7f44180c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f441cc3b700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4418071240 0x7f4418154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502287445 0 0) 0x7f4418072160 con 0x7f44180c4100 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44040035b0 con 0x7f44180c4100 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4027719863 0 0) 0x7f44040035b0 con 0x7f44180c4100 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4418071240 msgr2=0x7f4418154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4418071240 0x7f4418154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 >> v2:172.21.15.59:3300/0 conn(0x7f441814e5f0 msgr2=0x7f44180c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:47.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 --2- 172.21.15.31:0/4187145322 >> v2:172.21.15.59:3300/0 conn(0x7f441814e5f0 0x7f44180c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44180723b0 con 0x7f44180c4100 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f44100019f0 con 0x7f44180c4100 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 >> v1:172.21.15.31:6789/0 conn(0x7f44180c4100 legacy=0x7f4418154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4404003a00 0x7f4404003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4418071240 0x7f4404004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4404004d90 0x7f4404005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44180c2bd0 con 0x7f4404004d90 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44180c2e90 con 0x7f4404003a00 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.046+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440400ba90 con 0x7f4418071240 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f441d43c700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4404003a00 0x7f4404003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f441dc3d700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4404004d90 0x7f4404005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f441cc3b700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4418071240 0x7f4404004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f441dc3d700 1 -- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4418071240 msgr2=0x7f4404004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:47.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f441dc3d700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4418071240 0x7f4404004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f441dc3d700 1 -- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4404003a00 msgr2=0x7f4404003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f441dc3d700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4404003a00 0x7f4404003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f441dc3d700 1 -- 172.21.15.31:0/4187145322 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44040035b0 con 0x7f4404004d90 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f441dc3d700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4404004d90 0x7f4404005210 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f441000a830 tx=0x7f4404005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c8255a04fba322 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441000a320 con 0x7f4404004d90 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410012600 con 0x7f4404004d90 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.047+0000 7f440ffff700 1 -- 172.21.15.31:0/4187145322 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44100129b0 con 0x7f4404004d90 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.048+0000 7f441f6a0700 1 -- 172.21.15.31:0/4187145322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4404004d90 msgr2=0x7f4404005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.048+0000 7f441f6a0700 1 --2- 172.21.15.31:0/4187145322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4404004d90 0x7f4404005210 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f441000a830 tx=0x7f4404005880 comp rx=0 tx=0).stop 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.048+0000 7f441dc3d700 1 -- 172.21.15.31:0/4187145322 reap_dead start 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.048+0000 7f441f6a0700 1 -- 172.21.15.31:0/4187145322 shutdown_connections 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.048+0000 7f441f6a0700 1 -- 172.21.15.31:0/4187145322 >> 172.21.15.31:0/4187145322 conn(0x7f441806ba20 msgr2=0x7f441806be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.048+0000 7f441f6a0700 1 -- 172.21.15.31:0/4187145322 shutdown_connections 2024-03-30T01:42:47.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.048+0000 7f441f6a0700 1 -- 172.21.15.31:0/4187145322 wait complete. 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.049+0000 7f441f6a0700 1 Processor -- start 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.049+0000 7f441f6a0700 1 -- start start 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441f6a0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180ab330 0x7f4418149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441f6a0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f441814a320 0x7f4418148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441f6a0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44180abe90 0x7f4418148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441f6a0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44181582b0 con 0x7f44180ab330 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441f6a0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44181e9e00 con 0x7f441814a320 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441f6a0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44181ea0c0 con 0x7f44180abe90 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441d43c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180ab330 0x7f4418149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441d43c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180ab330 0x7f4418149de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35076/0 (socket says 172.21.15.31:35076) 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441d43c700 1 -- 172.21.15.31:0/1756639249 learned_addr learned my addr 172.21.15.31:0/1756639249 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441dc3d700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44180abe90 0x7f4418148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441cc3b700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f441814a320 0x7f4418148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441d43c700 1 -- 172.21.15.31:0/1756639249 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44180abe90 msgr2=0x7f4418148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441d43c700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44180abe90 0x7f4418148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441d43c700 1 -- 172.21.15.31:0/1756639249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f441814a320 msgr2=0x7f4418148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441d43c700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f441814a320 0x7f4418148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441d43c700 1 -- 172.21.15.31:0/1756639249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44181ea3a0 con 0x7f44180ab330 2024-03-30T01:42:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.050+0000 7f441d43c700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180ab330 0x7f4418149de0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4414011aa0 tx=0x7f4414011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.051+0000 7f440dffb700 1 -- 172.21.15.31:0/1756639249 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400f9e0 con 0x7f44180ab330 2024-03-30T01:42:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.051+0000 7f440dffb700 1 -- 172.21.15.31:0/1756639249 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441400fb40 con 0x7f44180ab330 2024-03-30T01:42:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.051+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44181ea680 con 0x7f44180ab330 2024-03-30T01:42:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.051+0000 7f440dffb700 1 -- 172.21.15.31:0/1756639249 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414014440 con 0x7f44180ab330 2024-03-30T01:42:47.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.051+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4418146180 con 0x7f44180ab330 2024-03-30T01:42:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.052+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44181eacd0 con 0x7f44180ab330 2024-03-30T01:42:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.052+0000 7f440dffb700 1 -- 172.21.15.31:0/1756639249 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f44140145a0 con 0x7f44180ab330 2024-03-30T01:42:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.053+0000 7f440dffb700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f43fc06ac90 0x7f43fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.053+0000 7f440dffb700 1 -- 172.21.15.31:0/1756639249 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f4414089fe0 con 0x7f44180ab330 2024-03-30T01:42:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.053+0000 7f441cc3b700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f43fc06ac90 0x7f43fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.054+0000 7f441cc3b700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f43fc06ac90 0x7f43fc06d150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f44180ac500 tx=0x7f4408008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:47.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.058+0000 7f440dffb700 1 -- 172.21.15.31:0/1756639249 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4414054d60 con 0x7f44180ab330 2024-03-30T01:42:47.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.275+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f441804ea80 con 0x7f44180ab330 2024-03-30T01:42:47.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.276+0000 7f440dffb700 1 -- 172.21.15.31:0/1756639249 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20061 (secure 0 0 0) 0x7f4414058380 con 0x7f44180ab330 2024-03-30T01:42:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:47.282 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:43.871647+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f43fc06ac90 msgr2=0x7f43fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f43fc06ac90 0x7f43fc06d150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f44180ac500 tx=0x7f4408008040 comp rx=0 tx=0).stop 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180ab330 msgr2=0x7f4418149de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180ab330 0x7f4418149de0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4414011aa0 tx=0x7f4414011e60 comp rx=0 tx=0).stop 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 shutdown_connections 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180ab330 0x7f4418149de0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f43fc06ac90 0x7f43fc06d150 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44180abe90 0x7f4418148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 --2- 172.21.15.31:0/1756639249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f441814a320 0x7f4418148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 >> 172.21.15.31:0/1756639249 conn(0x7f441806ba20 msgr2=0x7f441806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 shutdown_connections 2024-03-30T01:42:47.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.279+0000 7f441f6a0700 1 -- 172.21.15.31:0/1756639249 wait complete. 2024-03-30T01:42:47.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:42:07.881738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:42:09.236870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-30T01:42:21.874769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-30T01:42:23.236623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:42:47.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:42:47.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.434+0000 7f275f024700 1 Processor -- start 2024-03-30T01:42:47.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.434+0000 7f275f024700 1 -- start start 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.435+0000 7f275f024700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f27580c8420 0x7f27580c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.435+0000 7f275f024700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2758155240 con 0x7f27580c7f10 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.435+0000 7f275f024700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27581556d0 con 0x7f2758151d80 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.435+0000 7f275f024700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275806cb20 con 0x7f27580c8420 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.435+0000 7f2757fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f27580c7f10 0x7f27580c6a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36918/0 (socket says 172.21.15.31:36918) 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.435+0000 7f2757fff700 1 -- 172.21.15.31:0/3210326686 learned_addr learned my addr 172.21.15.31:0/3210326686 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f275d5c1700 1 --2- 172.21.15.31:0/3210326686 >> v2:172.21.15.59:3300/0 conn(0x7f27580c8420 0x7f27580c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f27577fe700 1 -- 172.21.15.31:0/3210326686 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139571951 0 0) 0x7f2758155240 con 0x7f27580c7f10 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f27577fe700 1 -- 172.21.15.31:0/3210326686 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27400035b0 con 0x7f27580c7f10 2024-03-30T01:42:47.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f275d5c1700 1 -- 172.21.15.31:0/3210326686 >> v1:172.21.15.59:6789/0 conn(0x7f2758151d80 legacy=0x7f27580ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f275d5c1700 1 -- 172.21.15.31:0/3210326686 >> v1:172.21.15.31:6789/0 conn(0x7f27580c7f10 legacy=0x7f27580c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f275d5c1700 1 -- 172.21.15.31:0/3210326686 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2758154910 con 0x7f27580c8420 2024-03-30T01:42:47.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f275d5c1700 1 --2- 172.21.15.31:0/3210326686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27580c8420 0x7f27580c7130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2750009d90 tx=0x7f275000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5fbe595414f49b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:47.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f27577fe700 1 -- 172.21.15.31:0/3210326686 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275000e070 con 0x7f27580c8420 2024-03-30T01:42:47.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f27577fe700 1 -- 172.21.15.31:0/3210326686 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275000bb20 con 0x7f27580c8420 2024-03-30T01:42:47.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.436+0000 7f27577fe700 1 -- 172.21.15.31:0/3210326686 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275000be90 con 0x7f27580c8420 2024-03-30T01:42:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.437+0000 7f275f024700 1 -- 172.21.15.31:0/3210326686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27580c8420 msgr2=0x7f27580c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.437+0000 7f275f024700 1 --2- 172.21.15.31:0/3210326686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27580c8420 0x7f27580c7130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2750009d90 tx=0x7f275000b330 comp rx=0 tx=0).stop 2024-03-30T01:42:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.437+0000 7f275f024700 1 -- 172.21.15.31:0/3210326686 shutdown_connections 2024-03-30T01:42:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.437+0000 7f275f024700 1 --2- 172.21.15.31:0/3210326686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27580c8420 0x7f27580c7130 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.437+0000 7f275f024700 1 -- 172.21.15.31:0/3210326686 >> 172.21.15.31:0/3210326686 conn(0x7f2758072010 msgr2=0x7f2758072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.437+0000 7f275f024700 1 -- 172.21.15.31:0/3210326686 shutdown_connections 2024-03-30T01:42:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.437+0000 7f275f024700 1 -- 172.21.15.31:0/3210326686 wait complete. 2024-03-30T01:42:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.438+0000 7f275f024700 1 Processor -- start 2024-03-30T01:42:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.438+0000 7f275f024700 1 -- start start 2024-03-30T01:42:47.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.439+0000 7f275f024700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27580c7f10 0x7f27580c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.439+0000 7f275f024700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27580c8420 0x7f27580be480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.439+0000 7f275f024700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758151d80 0x7f27580c03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.439+0000 7f275f024700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580c0f00 con 0x7f2758151d80 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.439+0000 7f275f024700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580bdff0 con 0x7f27580c8420 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.439+0000 7f275f024700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2758072720 con 0x7f27580c7f10 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.439+0000 7f275d5c1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758151d80 0x7f27580c03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.439+0000 7f275d5c1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758151d80 0x7f27580c03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35080/0 (socket says 172.21.15.31:35080) 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.439+0000 7f275d5c1700 1 -- 172.21.15.31:0/1893008318 learned_addr learned my addr 172.21.15.31:0/1893008318 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f2757fff700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27580c8420 0x7f27580be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f275cdc0700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27580c7f10 0x7f27580c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f275d5c1700 1 -- 172.21.15.31:0/1893008318 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27580c7f10 msgr2=0x7f27580c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f275d5c1700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27580c7f10 0x7f27580c10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f275d5c1700 1 -- 172.21.15.31:0/1893008318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27580c8420 msgr2=0x7f27580be480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f275d5c1700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27580c8420 0x7f27580be480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f275d5c1700 1 -- 172.21.15.31:0/1893008318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2750009a70 con 0x7f2758151d80 2024-03-30T01:42:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f275d5c1700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758151d80 0x7f27580c03a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f275000b370 tx=0x7f2750004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f27557fa700 1 -- 172.21.15.31:0/1893008318 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750004800 con 0x7f2758151d80 2024-03-30T01:42:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f27557fa700 1 -- 172.21.15.31:0/1893008318 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2750020070 con 0x7f2758151d80 2024-03-30T01:42:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27580bdc80 con 0x7f2758151d80 2024-03-30T01:42:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f27557fa700 1 -- 172.21.15.31:0/1893008318 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27500276c0 con 0x7f2758151d80 2024-03-30T01:42:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.440+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275814b900 con 0x7f2758151d80 2024-03-30T01:42:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.441+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27580bdae0 con 0x7f2758151d80 2024-03-30T01:42:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.442+0000 7f27557fa700 1 -- 172.21.15.31:0/1893008318 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2750012430 con 0x7f2758151d80 2024-03-30T01:42:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.443+0000 7f27557fa700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f274406ac90 0x7f274406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.443+0000 7f275cdc0700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f274406ac90 0x7f274406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.443+0000 7f27557fa700 1 -- 172.21.15.31:0/1893008318 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f2750022070 con 0x7f2758151d80 2024-03-30T01:42:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.443+0000 7f275cdc0700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f274406ac90 0x7f274406d150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2748007fd0 tx=0x7f2748009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.448+0000 7f27557fa700 1 -- 172.21.15.31:0/1893008318 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f27580bdae0 con 0x7f2758151d80 2024-03-30T01:42:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.659+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27580bdae0 con 0x7f2758151d80 2024-03-30T01:42:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.660+0000 7f27557fa700 1 -- 172.21.15.31:0/1893008318 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27580bdae0 con 0x7f2758151d80 2024-03-30T01:42:47.666 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:42:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f274406ac90 msgr2=0x7f274406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f274406ac90 0x7f274406d150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2748007fd0 tx=0x7f2748009040 comp rx=0 tx=0).stop 2024-03-30T01:42:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758151d80 msgr2=0x7f27580c03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758151d80 0x7f27580c03a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f275000b370 tx=0x7f2750004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 shutdown_connections 2024-03-30T01:42:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758151d80 0x7f27580c03a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f274406ac90 0x7f274406d150 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27580c7f10 0x7f27580c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 --2- 172.21.15.31:0/1893008318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27580c8420 0x7f27580be480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 >> 172.21.15.31:0/1893008318 conn(0x7f2758072010 msgr2=0x7f275814fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:47.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 shutdown_connections 2024-03-30T01:42:47.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.662+0000 7f275f024700 1 -- 172.21.15.31:0/1893008318 wait complete. 2024-03-30T01:42:47.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:42:47.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.815+0000 7f9f59dfb700 1 Processor -- start 2024-03-30T01:42:47.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.815+0000 7f9f59dfb700 1 -- start start 2024-03-30T01:42:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.816+0000 7f9f59dfb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9f541549a0 0x7f9f54072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.816+0000 7f9f59dfb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 0x7f9f5406dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.816+0000 7f9f59dfb700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5414ee60 con 0x7f9f5406d040 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.816+0000 7f9f59dfb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f540c2290 con 0x7f9f541549a0 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.816+0000 7f9f59dfb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5406e2b0 con 0x7f9f5406d900 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.816+0000 7f9f52ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9f5406d040 0x7f9f54154d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36922/0 (socket says 172.21.15.31:36922) 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.816+0000 7f9f52ffd700 1 -- 172.21.15.31:0/1529915834 learned_addr learned my addr 172.21.15.31:0/1529915834 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f537fe700 1 --2- 172.21.15.31:0/1529915834 >> v2:172.21.15.59:3300/0 conn(0x7f9f541549a0 0x7f9f54072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f53fff700 1 --2- 172.21.15.31:0/1529915834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 0x7f9f5406dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f527fc700 1 -- 172.21.15.31:0/1529915834 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873282649 0 0) 0x7f9f5414ee60 con 0x7f9f5406d040 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f527fc700 1 -- 172.21.15.31:0/1529915834 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f400035b0 con 0x7f9f5406d040 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f537fe700 1 -- 172.21.15.31:0/1529915834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 msgr2=0x7f9f5406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f537fe700 1 --2- 172.21.15.31:0/1529915834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 0x7f9f5406dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f537fe700 1 -- 172.21.15.31:0/1529915834 >> v1:172.21.15.31:6789/0 conn(0x7f9f5406d040 legacy=0x7f9f54154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f537fe700 1 -- 172.21.15.31:0/1529915834 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54155490 con 0x7f9f541549a0 2024-03-30T01:42:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f537fe700 1 --2- 172.21.15.31:0/1529915834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f541549a0 0x7f9f54072410 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009840 tx=0x7f9f44009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f47ec9038d86b270 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f527fc700 1 -- 172.21.15.31:0/1529915834 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44011830 con 0x7f9f541549a0 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f527fc700 1 -- 172.21.15.31:0/1529915834 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f44011990 con 0x7f9f541549a0 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.817+0000 7f9f527fc700 1 -- 172.21.15.31:0/1529915834 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44011d20 con 0x7f9f541549a0 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.818+0000 7f9f59dfb700 1 -- 172.21.15.31:0/1529915834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f541549a0 msgr2=0x7f9f54072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.818+0000 7f9f59dfb700 1 --2- 172.21.15.31:0/1529915834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f541549a0 0x7f9f54072410 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009840 tx=0x7f9f44009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.818+0000 7f9f59dfb700 1 -- 172.21.15.31:0/1529915834 shutdown_connections 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.818+0000 7f9f59dfb700 1 --2- 172.21.15.31:0/1529915834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 0x7f9f5406dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.818+0000 7f9f59dfb700 1 --2- 172.21.15.31:0/1529915834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f541549a0 0x7f9f54072410 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.818+0000 7f9f59dfb700 1 -- 172.21.15.31:0/1529915834 >> 172.21.15.31:0/1529915834 conn(0x7f9f540c3980 msgr2=0x7f9f540c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.818+0000 7f9f59dfb700 1 -- 172.21.15.31:0/1529915834 shutdown_connections 2024-03-30T01:42:47.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.818+0000 7f9f59dfb700 1 -- 172.21.15.31:0/1529915834 wait complete. 2024-03-30T01:42:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.819+0000 7f9f59dfb700 1 Processor -- start 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f59dfb700 1 -- start start 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f59dfb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f5406d040 0x7f9f5415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f59dfb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 0x7f9f540c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f59dfb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f541549a0 0x7f9f540c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f59dfb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5415dd60 con 0x7f9f541549a0 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f59dfb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540c5d90 con 0x7f9f5406d040 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f59dfb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54145700 con 0x7f9f5406d900 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f53fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f541549a0 0x7f9f540c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f53fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f541549a0 0x7f9f540c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35092/0 (socket says 172.21.15.31:35092) 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.820+0000 7f9f53fff700 1 -- 172.21.15.31:0/219856069 learned_addr learned my addr 172.21.15.31:0/219856069 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f537fe700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f5406d040 0x7f9f5415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f52ffd700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 0x7f9f540c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f53fff700 1 -- 172.21.15.31:0/219856069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 msgr2=0x7f9f540c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f53fff700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 0x7f9f540c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f53fff700 1 -- 172.21.15.31:0/219856069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f5406d040 msgr2=0x7f9f5415a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f53fff700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f5406d040 0x7f9f5415a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f53fff700 1 -- 172.21.15.31:0/219856069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f440094a0 con 0x7f9f541549a0 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f537fe700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f5406d040 0x7f9f5415a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f53fff700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f541549a0 0x7f9f540c5850 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00efc0 tx=0x7f9f4c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f4bfff700 1 -- 172.21.15.31:0/219856069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c009ea0 con 0x7f9f541549a0 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f4bfff700 1 -- 172.21.15.31:0/219856069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c00f040 con 0x7f9f541549a0 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f54145420 con 0x7f9f541549a0 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f4bfff700 1 -- 172.21.15.31:0/219856069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00ced0 con 0x7f9f541549a0 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.821+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f540c4df0 con 0x7f9f541549a0 2024-03-30T01:42:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.822+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f540c4870 con 0x7f9f541549a0 2024-03-30T01:42:47.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.823+0000 7f9f4bfff700 1 -- 172.21.15.31:0/219856069 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f4c014480 con 0x7f9f541549a0 2024-03-30T01:42:47.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.823+0000 7f9f4bfff700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9f3406ac90 0x7f9f3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:47.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.823+0000 7f9f4bfff700 1 -- 172.21.15.31:0/219856069 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f9f4c089f80 con 0x7f9f541549a0 2024-03-30T01:42:47.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.823+0000 7f9f537fe700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9f3406ac90 0x7f9f3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:47.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.824+0000 7f9f537fe700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9f3406ac90 0x7f9f3406d150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009fd0 tx=0x7f9f4401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:47.828+0000 7f9f4bfff700 1 -- 172.21.15.31:0/219856069 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9f4c0bd580 con 0x7f9f541549a0 2024-03-30T01:42:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.046+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f540c5010 con 0x7f9f541549a0 2024-03-30T01:42:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.046+0000 7f9f4bfff700 1 -- 172.21.15.31:0/219856069 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f4c0180a0 con 0x7f9f541549a0 2024-03-30T01:42:48.052 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:48.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.048+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9f3406ac90 msgr2=0x7f9f3406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.048+0000 7f9f59dfb700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9f3406ac90 0x7f9f3406d150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009fd0 tx=0x7f9f4401c040 comp rx=0 tx=0).stop 2024-03-30T01:42:48.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.048+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f541549a0 msgr2=0x7f9f540c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.048+0000 7f9f59dfb700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f541549a0 0x7f9f540c5850 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00efc0 tx=0x7f9f4c00c3a0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.049+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 shutdown_connections 2024-03-30T01:42:48.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.049+0000 7f9f59dfb700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f541549a0 0x7f9f540c5850 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.049+0000 7f9f59dfb700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9f3406ac90 0x7f9f3406d150 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.049+0000 7f9f59dfb700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f5406d040 0x7f9f5415a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.049+0000 7f9f59dfb700 1 --2- 172.21.15.31:0/219856069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9f5406d900 0x7f9f540c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.049+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 >> 172.21.15.31:0/219856069 conn(0x7f9f540c3980 msgr2=0x7f9f54154d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:48.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.049+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 shutdown_connections 2024-03-30T01:42:48.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.049+0000 7f9f59dfb700 1 -- 172.21.15.31:0/219856069 wait complete. 2024-03-30T01:42:48.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:42:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.201+0000 7fedef3c4700 1 Processor -- start 2024-03-30T01:42:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.201+0000 7fedef3c4700 1 -- start start 2024-03-30T01:42:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.202+0000 7fedef3c4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fede814ff20 0x7fede80c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.202+0000 7fedef3c4700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede80c4620 con 0x7fede814fb40 2024-03-30T01:42:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.202+0000 7fedef3c4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede8152ad0 con 0x7fede80a7600 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.202+0000 7fedef3c4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede8152570 con 0x7fede814ff20 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.202+0000 7feded160700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fede814fb40 0x7fede80c2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36924/0 (socket says 172.21.15.31:36924) 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.202+0000 7feded160700 1 -- 172.21.15.31:0/458449075 learned_addr learned my addr 172.21.15.31:0/458449075 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.202+0000 7feded961700 1 --2- 172.21.15.31:0/458449075 >> v2:172.21.15.59:3300/0 conn(0x7fede814ff20 0x7fede80c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.202+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185301990 0 0) 0x7fede4001110 con 0x7fede814fb40 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedd40035b0 con 0x7fede814fb40 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2615897554 0 0) 0x7fedd40035b0 con 0x7fede814fb40 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 >> v2:172.21.15.59:3300/0 conn(0x7fede814ff20 msgr2=0x7fede80c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 --2- 172.21.15.31:0/458449075 >> v2:172.21.15.59:3300/0 conn(0x7fede814ff20 0x7fede80c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 >> v1:172.21.15.59:6789/0 conn(0x7fede80a7600 legacy=0x7fede8152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede80c4060 con 0x7fede814fb40 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7fede4001800 con 0x7fede814fb40 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 >> v1:172.21.15.31:6789/0 conn(0x7fede814fb40 legacy=0x7fede80c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedd4003ae0 0x7fedd4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fedd4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd4005210 0x7fedd400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd400c0e0 con 0x7fedd4004850 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd400c250 con 0x7fedd4005210 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd400c410 con 0x7fedd4003ae0 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7fedec95f700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fedd4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feded160700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedd4003ae0 0x7fedd4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feded961700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd4005210 0x7fedd400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7fedec95f700 1 -- 172.21.15.31:0/458449075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedd4003ae0 msgr2=0x7fedd4003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7fedec95f700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedd4003ae0 0x7fedd4003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7fedec95f700 1 -- 172.21.15.31:0/458449075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd4005210 msgr2=0x7fedd400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7fedec95f700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd4005210 0x7fedd400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7feded961700 1 -- 172.21.15.31:0/458449075 reap_dead start 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.203+0000 7fedec95f700 1 -- 172.21.15.31:0/458449075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd40035b0 con 0x7fedd4004850 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7fedec95f700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fedd4004cd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7feddc0018a0 tx=0x7fedd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ddce201c984e5b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc0052d0 con 0x7fedd4004850 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc005450 con 0x7fedd4004850 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7feddbfff700 1 -- 172.21.15.31:0/458449075 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc0038c0 con 0x7fedd4004850 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7fedef3c4700 1 -- 172.21.15.31:0/458449075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 msgr2=0x7fedd4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7fedef3c4700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fedd4004cd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7feddc0018a0 tx=0x7fedd4005880 comp rx=0 tx=0).stop 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7fedef3c4700 1 -- 172.21.15.31:0/458449075 shutdown_connections 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7fedef3c4700 1 --2- 172.21.15.31:0/458449075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fedd4004cd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7fedef3c4700 1 -- 172.21.15.31:0/458449075 >> 172.21.15.31:0/458449075 conn(0x7fede80c7d60 msgr2=0x7fede80c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7fedef3c4700 1 -- 172.21.15.31:0/458449075 shutdown_connections 2024-03-30T01:42:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.204+0000 7fedef3c4700 1 -- 172.21.15.31:0/458449075 wait complete. 2024-03-30T01:42:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7fedef3c4700 1 Processor -- start 2024-03-30T01:42:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7fedef3c4700 1 -- start start 2024-03-30T01:42:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7fedef3c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fede81ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7fedef3c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fede814f270 0x7fede80c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7fedef3c4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fede80c6870 0x7fede81dee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7fedef3c4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede81e9cb0 con 0x7fedd4004850 2024-03-30T01:42:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7fedef3c4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede81ea210 con 0x7fede814f270 2024-03-30T01:42:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7fedef3c4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede81eabe0 con 0x7fede80c6870 2024-03-30T01:42:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7feded160700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fede81ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7feded160700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fede81ea6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35122/0 (socket says 172.21.15.31:35122) 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.206+0000 7feded160700 1 -- 172.21.15.31:0/2582393692 learned_addr learned my addr 172.21.15.31:0/2582393692 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7feded961700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fede80c6870 0x7fede81dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7fedec95f700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fede814f270 0x7fede80c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7feded160700 1 -- 172.21.15.31:0/2582393692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fede80c6870 msgr2=0x7fede81dee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7feded160700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fede80c6870 0x7fede81dee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7feded160700 1 -- 172.21.15.31:0/2582393692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fede814f270 msgr2=0x7fede80c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7feded160700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fede814f270 0x7fede80c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7feded160700 1 -- 172.21.15.31:0/2582393692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede806e2d0 con 0x7fedd4004850 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7feded160700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fede81ea6a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fede40072c0 tx=0x7fede4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7fedd9ffb700 1 -- 172.21.15.31:0/2582393692 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40077a0 con 0x7fedd4004850 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7fedd9ffb700 1 -- 172.21.15.31:0/2582393692 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede400f870 con 0x7fedd4004850 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede806e5b0 con 0x7fedd4004850 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7fedd9ffb700 1 -- 172.21.15.31:0/2582393692 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede400fc20 con 0x7fedd4004850 2024-03-30T01:42:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.207+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede80c6100 con 0x7fedd4004850 2024-03-30T01:42:48.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.208+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede806e7f0 con 0x7fedd4004850 2024-03-30T01:42:48.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.209+0000 7fedd9ffb700 1 -- 172.21.15.31:0/2582393692 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fede400fd80 con 0x7fedd4004850 2024-03-30T01:42:48.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.209+0000 7fedd9ffb700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fedcc06ac90 0x7fedcc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.209+0000 7fedd9ffb700 1 -- 172.21.15.31:0/2582393692 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7fede4088c70 con 0x7fedd4004850 2024-03-30T01:42:48.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.209+0000 7fedec95f700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fedcc06ac90 0x7fedcc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.210+0000 7fedec95f700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fedcc06ac90 0x7fedcc06d150 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feddc005040 tx=0x7feddc00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:48.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.214+0000 7fedd9ffb700 1 -- 172.21.15.31:0/2582393692 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fede806e7f0 con 0x7fedd4004850 2024-03-30T01:42:48.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.435+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fede80c4620 con 0x7fedd4004850 2024-03-30T01:42:48.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.436+0000 7fedd9ffb700 1 -- 172.21.15.31:0/2582393692 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7fede80c4620 con 0x7fedd4004850 2024-03-30T01:42:48.442 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:48.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.438+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fedcc06ac90 msgr2=0x7fedcc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.438+0000 7fedef3c4700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fedcc06ac90 0x7fedcc06d150 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feddc005040 tx=0x7feddc00a000 comp rx=0 tx=0).stop 2024-03-30T01:42:48.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.438+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 msgr2=0x7fede81ea6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.438+0000 7fedef3c4700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fede81ea6a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fede40072c0 tx=0x7fede4007680 comp rx=0 tx=0).stop 2024-03-30T01:42:48.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.438+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 shutdown_connections 2024-03-30T01:42:48.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.438+0000 7fedef3c4700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4004850 0x7fede81ea6a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.439+0000 7fedef3c4700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fedcc06ac90 0x7fedcc06d150 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.439+0000 7fedef3c4700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fede80c6870 0x7fede81dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.439+0000 7fedef3c4700 1 --2- 172.21.15.31:0/2582393692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fede814f270 0x7fede80c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.439+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 >> 172.21.15.31:0/2582393692 conn(0x7fede80c7d60 msgr2=0x7fede80c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:48.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.439+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 shutdown_connections 2024-03-30T01:42:48.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.439+0000 7fedef3c4700 1 -- 172.21.15.31:0/2582393692 wait complete. 2024-03-30T01:42:48.458 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-30T01:42:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.591+0000 7ff8136c8700 1 Processor -- start 2024-03-30T01:42:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.591+0000 7ff8136c8700 1 -- start start 2024-03-30T01:42:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.592+0000 7ff8136c8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff80c151ea0 0x7ff80c0c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.592+0000 7ff8136c8700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c155000 con 0x7ff80c0721f0 2024-03-30T01:42:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.592+0000 7ff8136c8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c06e650 con 0x7ff80c0c07f0 2024-03-30T01:42:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.592+0000 7ff8136c8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c06e930 con 0x7ff80c151ea0 2024-03-30T01:42:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.592+0000 7ff810c63700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff80c0721f0 0x7ff80c06a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36940/0 (socket says 172.21.15.31:36940) 2024-03-30T01:42:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.592+0000 7ff810c63700 1 -- 172.21.15.31:0/362380760 learned_addr learned my addr 172.21.15.31:0/362380760 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff811464700 1 --2- 172.21.15.31:0/362380760 >> v2:172.21.15.59:3300/0 conn(0x7ff80c151ea0 0x7ff80c0c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706217349 0 0) 0x7ff80c155000 con 0x7ff80c0721f0 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7f80035b0 con 0x7ff80c0721f0 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 618399490 0 0) 0x7ff7f80035b0 con 0x7ff80c0721f0 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 >> v2:172.21.15.59:3300/0 conn(0x7ff80c151ea0 msgr2=0x7ff80c0c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 --2- 172.21.15.31:0/362380760 >> v2:172.21.15.59:3300/0 conn(0x7ff80c151ea0 0x7ff80c0c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 >> v1:172.21.15.59:6789/0 conn(0x7ff80c0c07f0 legacy=0x7ff80c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c1549b0 con 0x7ff80c0721f0 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7ff800001670 con 0x7ff80c0721f0 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 >> v1:172.21.15.31:6789/0 conn(0x7ff80c0721f0 legacy=0x7ff80c06a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f8003ae0 0x7ff7f8003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7f8004850 0x7ff7f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f80052e0 0x7ff7f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c06e930 con 0x7ff7f8004850 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f800c230 con 0x7ff7f8003ae0 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f800c3f0 con 0x7ff7f80052e0 2024-03-30T01:42:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff810c63700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7f8004850 0x7ff7f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff811464700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f8003ae0 0x7ff7f8003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff811c65700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f80052e0 0x7ff7f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff810c63700 1 -- 172.21.15.31:0/362380760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f80052e0 msgr2=0x7ff7f800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff810c63700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f80052e0 0x7ff7f800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff810c63700 1 -- 172.21.15.31:0/362380760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f8003ae0 msgr2=0x7ff7f8003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.593+0000 7ff810c63700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f8003ae0 0x7ff7f8003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff810c63700 1 -- 172.21.15.31:0/362380760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f80035b0 con 0x7ff7f8004850 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff811c65700 1 -- 172.21.15.31:0/362380760 reap_dead start 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff810c63700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7f8004850 0x7ff7f8004cd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c240 tx=0x7ff7f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619d3c422d4add73 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800007c10 con 0x7ff7f8004850 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80000ce00 con 0x7ff7f8004850 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff7fffff700 1 -- 172.21.15.31:0/362380760 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800005ba0 con 0x7ff7f8004850 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff8136c8700 1 -- 172.21.15.31:0/362380760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7f8004850 msgr2=0x7ff7f8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff8136c8700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7f8004850 0x7ff7f8004cd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c240 tx=0x7ff7f8005880 comp rx=0 tx=0).stop 2024-03-30T01:42:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff8136c8700 1 -- 172.21.15.31:0/362380760 shutdown_connections 2024-03-30T01:42:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff8136c8700 1 --2- 172.21.15.31:0/362380760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7f8004850 0x7ff7f8004cd0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.594+0000 7ff8136c8700 1 -- 172.21.15.31:0/362380760 >> 172.21.15.31:0/362380760 conn(0x7ff80c0c3980 msgr2=0x7ff80c0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.595+0000 7ff8136c8700 1 -- 172.21.15.31:0/362380760 shutdown_connections 2024-03-30T01:42:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.595+0000 7ff8136c8700 1 -- 172.21.15.31:0/362380760 wait complete. 2024-03-30T01:42:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.596+0000 7ff8136c8700 1 Processor -- start 2024-03-30T01:42:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.596+0000 7ff8136c8700 1 -- start start 2024-03-30T01:42:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff8136c8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f8004850 0x7ff80c146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff8136c8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff80c1ed5e0 0x7ff80c14b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff8136c8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c14c900 0x7ff80c14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff8136c8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c14b040 con 0x7ff80c14c900 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff8136c8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c1490f0 con 0x7ff7f8004850 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff8136c8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c14d2c0 con 0x7ff80c1ed5e0 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811c65700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c14c900 0x7ff80c14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811c65700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c14c900 0x7ff80c14cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35144/0 (socket says 172.21.15.31:35144) 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811c65700 1 -- 172.21.15.31:0/2397951345 learned_addr learned my addr 172.21.15.31:0/2397951345 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811464700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f8004850 0x7ff80c146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff810c63700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff80c1ed5e0 0x7ff80c14b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811c65700 1 -- 172.21.15.31:0/2397951345 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff80c1ed5e0 msgr2=0x7ff80c14b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811c65700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff80c1ed5e0 0x7ff80c14b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811c65700 1 -- 172.21.15.31:0/2397951345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f8004850 msgr2=0x7ff80c146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811c65700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f8004850 0x7ff80c146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811c65700 1 -- 172.21.15.31:0/2397951345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c149e00 con 0x7ff80c14c900 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff811c65700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c14c900 0x7ff80c14cd80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff80c06d6a0 tx=0x7ff80400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff7fdffb700 1 -- 172.21.15.31:0/2397951345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80400ee70 con 0x7ff80c14c900 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff7fdffb700 1 -- 172.21.15.31:0/2397951345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff804003c10 con 0x7ff80c14c900 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c14a3c0 con 0x7ff80c14c900 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.597+0000 7ff7fdffb700 1 -- 172.21.15.31:0/2397951345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040050d0 con 0x7ff80c14c900 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.598+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c14a980 con 0x7ff80c14c900 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.599+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c14b180 con 0x7ff80c14c900 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.599+0000 7ff7fdffb700 1 -- 172.21.15.31:0/2397951345 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff804005230 con 0x7ff80c14c900 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.600+0000 7ff7fdffb700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff7f006ac90 0x7ff7f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.600+0000 7ff7fdffb700 1 -- 172.21.15.31:0/2397951345 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7ff8040895e0 con 0x7ff80c14c900 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.600+0000 7ff811464700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff7f006ac90 0x7ff7f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.600+0000 7ff811464700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff7f006ac90 0x7ff7f006d150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0c5fb0 tx=0x7ff808009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.605+0000 7ff7fdffb700 1 -- 172.21.15.31:0/2397951345 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff80c14b180 con 0x7ff80c14c900 2024-03-30T01:42:48.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.820+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff80c14b180 con 0x7ff80c14c900 2024-03-30T01:42:48.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.821+0000 7ff7fdffb700 1 -- 172.21.15.31:0/2397951345 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v45) v1 ==== 105+0+11 (secure 0 0 0) 0x7ff80c14b180 con 0x7ff80c14c900 2024-03-30T01:42:48.826 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff7f006ac90 msgr2=0x7ff7f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff7f006ac90 0x7ff7f006d150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0c5fb0 tx=0x7ff808009210 comp rx=0 tx=0).stop 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c14c900 msgr2=0x7ff80c14cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c14c900 0x7ff80c14cd80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff80c06d6a0 tx=0x7ff80400ed50 comp rx=0 tx=0).stop 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 shutdown_connections 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c14c900 0x7ff80c14cd80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff7f006ac90 0x7ff7f006d150 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f8004850 0x7ff80c146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 --2- 172.21.15.31:0/2397951345 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff80c1ed5e0 0x7ff80c14b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 >> 172.21.15.31:0/2397951345 conn(0x7ff80c0c3980 msgr2=0x7ff80c0c4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:48.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 shutdown_connections 2024-03-30T01:42:48.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.823+0000 7ff8136c8700 1 -- 172.21.15.31:0/2397951345 wait complete. 2024-03-30T01:42:48.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-30T01:42:48.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.976+0000 7f77b8626700 1 Processor -- start 2024-03-30T01:42:48.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.976+0000 7f77b8626700 1 -- start start 2024-03-30T01:42:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.977+0000 7f77b8626700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f77b0151ea0 0x7f77b00c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.977+0000 7f77b8626700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b0155000 con 0x7f77b00c07f0 2024-03-30T01:42:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.977+0000 7f77b8626700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b006e650 con 0x7f77b00721f0 2024-03-30T01:42:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.977+0000 7f77b8626700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b006e930 con 0x7f77b0151ea0 2024-03-30T01:42:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.977+0000 7f77b6bc3700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f77b00c07f0 0x7f77b006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36950/0 (socket says 172.21.15.31:36950) 2024-03-30T01:42:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.977+0000 7f77b6bc3700 1 -- 172.21.15.31:0/3388771171 learned_addr learned my addr 172.21.15.31:0/3388771171 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:48.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b63c2700 1 --2- 172.21.15.31:0/3388771171 >> v2:172.21.15.59:3300/0 conn(0x7f77b0151ea0 0x7f77b00c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b53c0700 1 -- 172.21.15.31:0/3388771171 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863305827 0 0) 0x7f77b0155000 con 0x7f77b00c07f0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b53c0700 1 -- 172.21.15.31:0/3388771171 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f779c0035b0 con 0x7f77b00c07f0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b53c0700 1 -- 172.21.15.31:0/3388771171 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302965456 0 0) 0x7f77b006e650 con 0x7f77b00721f0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b53c0700 1 -- 172.21.15.31:0/3388771171 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b0155000 con 0x7f77b00721f0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b63c2700 1 -- 172.21.15.31:0/3388771171 >> v1:172.21.15.59:6789/0 conn(0x7f77b00721f0 legacy=0x7f77b006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b63c2700 1 -- 172.21.15.31:0/3388771171 >> v1:172.21.15.31:6789/0 conn(0x7f77b00c07f0 legacy=0x7f77b006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b53c0700 1 -- 172.21.15.31:0/3388771171 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2251699799 0 0) 0x7f779c0035b0 con 0x7f77b00c07f0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b63c2700 1 -- 172.21.15.31:0/3388771171 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b01549b0 con 0x7f77b0151ea0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.978+0000 7f77b63c2700 1 --2- 172.21.15.31:0/3388771171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b0151ea0 0x7f77b00c02b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009840 tx=0x7f77ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2d7427319636b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b53c0700 1 -- 172.21.15.31:0/3388771171 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a80019f0 con 0x7f77b0151ea0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b53c0700 1 -- 172.21.15.31:0/3388771171 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac004030 con 0x7f77b0151ea0 2024-03-30T01:42:48.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b53c0700 1 -- 172.21.15.31:0/3388771171 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0043a0 con 0x7f77b0151ea0 2024-03-30T01:42:48.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b8626700 1 -- 172.21.15.31:0/3388771171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b0151ea0 msgr2=0x7f77b00c02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b8626700 1 --2- 172.21.15.31:0/3388771171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b0151ea0 0x7f77b00c02b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009840 tx=0x7f77ac009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:48.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b8626700 1 -- 172.21.15.31:0/3388771171 shutdown_connections 2024-03-30T01:42:48.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b8626700 1 --2- 172.21.15.31:0/3388771171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b0151ea0 0x7f77b00c02b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b8626700 1 -- 172.21.15.31:0/3388771171 >> 172.21.15.31:0/3388771171 conn(0x7f77b00c3980 msgr2=0x7f77b00c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:48.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b8626700 1 -- 172.21.15.31:0/3388771171 shutdown_connections 2024-03-30T01:42:48.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.979+0000 7f77b8626700 1 -- 172.21.15.31:0/3388771171 wait complete. 2024-03-30T01:42:48.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.980+0000 7f77b8626700 1 Processor -- start 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b8626700 1 -- start start 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b8626700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b00721f0 0x7f77b01539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b8626700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b00c07f0 0x7f77b0156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b8626700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b0151ea0 0x7f77b0157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b8626700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b0157660 con 0x7f77b00c07f0 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b8626700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b0149630 con 0x7f77b0151ea0 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b8626700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b01ea4e0 con 0x7f77b00721f0 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b5bc1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b00c07f0 0x7f77b0156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b5bc1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b00c07f0 0x7f77b0156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35160/0 (socket says 172.21.15.31:35160) 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b5bc1700 1 -- 172.21.15.31:0/1997454482 learned_addr learned my addr 172.21.15.31:0/1997454482 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.981+0000 7f77b6bc3700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b0151ea0 0x7f77b0157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b63c2700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b00721f0 0x7f77b01539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b5bc1700 1 -- 172.21.15.31:0/1997454482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b00721f0 msgr2=0x7f77b01539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b5bc1700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b00721f0 0x7f77b01539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b5bc1700 1 -- 172.21.15.31:0/1997454482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b0151ea0 msgr2=0x7f77b0157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b5bc1700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b0151ea0 0x7f77b0157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b5bc1700 1 -- 172.21.15.31:0/1997454482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0094a0 con 0x7f77b00c07f0 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b63c2700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b00721f0 0x7f77b01539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b5bc1700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b00c07f0 0x7f77b0156be0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f77a400c530 tx=0x7f77a400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77a2ffd700 1 -- 172.21.15.31:0/1997454482 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a40052d0 con 0x7f77b00c07f0 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77a2ffd700 1 -- 172.21.15.31:0/1997454482 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4005430 con 0x7f77b00c07f0 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77a2ffd700 1 -- 172.21.15.31:0/1997454482 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a40038c0 con 0x7f77b00c07f0 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b01eaa10 con 0x7f77b00c07f0 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.982+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b0149860 con 0x7f77b00c07f0 2024-03-30T01:42:48.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.983+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b00747d0 con 0x7f77b00c07f0 2024-03-30T01:42:48.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.984+0000 7f77a2ffd700 1 -- 172.21.15.31:0/1997454482 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f77a4003a20 con 0x7f77b00c07f0 2024-03-30T01:42:48.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.984+0000 7f77a2ffd700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779406ac90 0x7f779406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:48.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.984+0000 7f77a2ffd700 1 -- 172.21.15.31:0/1997454482 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f77a4088c90 con 0x7f77b00c07f0 2024-03-30T01:42:48.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.984+0000 7f77b63c2700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779406ac90 0x7f779406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:48.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.985+0000 7f77b63c2700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779406ac90 0x7f779406d150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f77ac000c00 tx=0x7f77ac013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:48.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:48.989+0000 7f77a2ffd700 1 -- 172.21.15.31:0/1997454482 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f77a4053990 con 0x7f77b00c07f0 2024-03-30T01:42:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.204+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f77b00c1520 con 0x7f77b00c07f0 2024-03-30T01:42:49.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.205+0000 7f77a2ffd700 1 -- 172.21.15.31:0/1997454482 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v45) v1 ==== 101+0+11 (secure 0 0 0) 0x7f77a4056fb0 con 0x7f77b00c07f0 2024-03-30T01:42:49.210 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779406ac90 msgr2=0x7f779406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779406ac90 0x7f779406d150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f77ac000c00 tx=0x7f77ac013040 comp rx=0 tx=0).stop 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b00c07f0 msgr2=0x7f77b0156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b00c07f0 0x7f77b0156be0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f77a400c530 tx=0x7f77a400c840 comp rx=0 tx=0).stop 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 shutdown_connections 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779406ac90 0x7f779406d150 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b0151ea0 0x7f77b0157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b00721f0 0x7f77b01539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 --2- 172.21.15.31:0/1997454482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b00c07f0 0x7f77b0156be0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 >> 172.21.15.31:0/1997454482 conn(0x7f77b00c3980 msgr2=0x7f77b0150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 shutdown_connections 2024-03-30T01:42:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.207+0000 7f77b8626700 1 -- 172.21.15.31:0/1997454482 wait complete. 2024-03-30T01:42:49.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.357+0000 7f3b92cbd700 1 Processor -- start 2024-03-30T01:42:49.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.357+0000 7f3b92cbd700 1 -- start start 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b92cbd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c3180 0x7f3b8c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b92cbd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3b8c06a700 0x7f3b8c06f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b92cbd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8c1547d0 con 0x7f3b8c0c4710 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b92cbd700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8c154a90 con 0x7f3b8c06a700 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b92cbd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c154d70 con 0x7f3b8c0c3180 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b8bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3b8c06a700 0x7f3b8c06f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b8bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3b8c06a700 0x7f3b8c06f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35164/0 (socket says 172.21.15.31:35164) 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b90a59700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c3180 0x7f3b8c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b8bfff700 1 -- 172.21.15.31:0/509147932 learned_addr learned my addr 172.21.15.31:0/509147932 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b8bfff700 1 -- 172.21.15.31:0/509147932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c3180 msgr2=0x7f3b8c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b8bfff700 1 --2- 172.21.15.31:0/509147932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c3180 0x7f3b8c06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b8bfff700 1 -- 172.21.15.31:0/509147932 >> v1:172.21.15.59:6789/0 conn(0x7f3b8c0c4710 legacy=0x7f3b8c0c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b8bfff700 1 -- 172.21.15.31:0/509147932 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c155050 con 0x7f3b8c06a700 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b8b7fe700 1 -- 172.21.15.31:0/509147932 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077637241 0 0) 0x7f3b8c1547d0 con 0x7f3b8c0c4710 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b90a59700 1 --2- 172.21.15.31:0/509147932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c3180 0x7f3b8c06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.358+0000 7f3b8bfff700 1 --2- 172.21.15.31:0/509147932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 0x7f3b8c06f040 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098f0 tx=0x7f3b78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aaf6751f8943752 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b8b7fe700 1 -- 172.21.15.31:0/509147932 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78011830 con 0x7f3b8c06a700 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b8b7fe700 1 -- 172.21.15.31:0/509147932 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b78011990 con 0x7f3b8c06a700 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b8b7fe700 1 -- 172.21.15.31:0/509147932 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78011d20 con 0x7f3b8c06a700 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b92cbd700 1 -- 172.21.15.31:0/509147932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 msgr2=0x7f3b8c06f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b92cbd700 1 --2- 172.21.15.31:0/509147932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 0x7f3b8c06f040 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098f0 tx=0x7f3b78009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b92cbd700 1 -- 172.21.15.31:0/509147932 shutdown_connections 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b92cbd700 1 --2- 172.21.15.31:0/509147932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c3180 0x7f3b8c06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b92cbd700 1 --2- 172.21.15.31:0/509147932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 0x7f3b8c06f040 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b92cbd700 1 -- 172.21.15.31:0/509147932 >> 172.21.15.31:0/509147932 conn(0x7f3b8c0c3a60 msgr2=0x7f3b8c0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:49.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b92cbd700 1 -- 172.21.15.31:0/509147932 shutdown_connections 2024-03-30T01:42:49.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.359+0000 7f3b92cbd700 1 -- 172.21.15.31:0/509147932 wait complete. 2024-03-30T01:42:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.360+0000 7f3b92cbd700 1 Processor -- start 2024-03-30T01:42:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.361+0000 7f3b92cbd700 1 -- start start 2024-03-30T01:42:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.361+0000 7f3b92cbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 0x7f3b8c1e9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.361+0000 7f3b92cbd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0c3180 0x7f3b8c158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.361+0000 7f3b92cbd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c4710 0x7f3b8c1ed540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.361+0000 7f3b92cbd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1e9c70 con 0x7f3b8c06a700 2024-03-30T01:42:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.361+0000 7f3b92cbd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c14ef30 con 0x7f3b8c0c3180 2024-03-30T01:42:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.361+0000 7f3b92cbd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c14f210 con 0x7f3b8c0c4710 2024-03-30T01:42:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b90a59700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 0x7f3b8c1e9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b90a59700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 0x7f3b8c1e9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35180/0 (socket says 172.21.15.31:35180) 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b90a59700 1 -- 172.21.15.31:0/962479734 learned_addr learned my addr 172.21.15.31:0/962479734 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b8bfff700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0c3180 0x7f3b8c158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b9125a700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c4710 0x7f3b8c1ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b90a59700 1 -- 172.21.15.31:0/962479734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c4710 msgr2=0x7f3b8c1ed540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b90a59700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c4710 0x7f3b8c1ed540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b90a59700 1 -- 172.21.15.31:0/962479734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0c3180 msgr2=0x7f3b8c158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b90a59700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0c3180 0x7f3b8c158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b90a59700 1 -- 172.21.15.31:0/962479734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b800094a0 con 0x7f3b8c06a700 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b90a59700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 0x7f3b8c1e9730 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000ebe0 tx=0x7f3b80009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b897fa700 1 -- 172.21.15.31:0/962479734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000c910 con 0x7f3b8c06a700 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b897fa700 1 -- 172.21.15.31:0/962479734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8000ca70 con 0x7f3b8c06a700 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b78009550 con 0x7f3b8c06a700 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b897fa700 1 -- 172.21.15.31:0/962479734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000cde0 con 0x7f3b8c06a700 2024-03-30T01:42:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.362+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c1ed020 con 0x7f3b8c06a700 2024-03-30T01:42:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.363+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8c04ea80 con 0x7f3b8c06a700 2024-03-30T01:42:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.364+0000 7f3b897fa700 1 -- 172.21.15.31:0/962479734 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3b80013470 con 0x7f3b8c06a700 2024-03-30T01:42:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.364+0000 7f3b897fa700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3b7006acc0 0x7f3b7006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.364+0000 7f3b897fa700 1 -- 172.21.15.31:0/962479734 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f3b80089970 con 0x7f3b8c06a700 2024-03-30T01:42:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.364+0000 7f3b8bfff700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3b7006acc0 0x7f3b7006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.365+0000 7f3b8bfff700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3b7006acc0 0x7f3b7006d180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c1546f0 tx=0x7f3b78008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.369+0000 7f3b897fa700 1 -- 172.21.15.31:0/962479734 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3b800546f0 con 0x7f3b8c06a700 2024-03-30T01:42:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.629+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b8c0722d0 con 0x7f3b8c06a700 2024-03-30T01:42:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.630+0000 7f3b897fa700 1 -- 172.21.15.31:0/962479734 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5942 (secure 0 0 0) 0x7f3b80057d10 con 0x7f3b8c06a700 2024-03-30T01:42:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:10.264695+0000","modified":"2024-03-30T01:42:13.278071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/1616247357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1616247357},{"type":"v1","addr":"172.21.15.31:6837","nonce":1616247357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:24.267501+0000","modified":"2024-03-30T01:42:27.273615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T01:42:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3b7006acc0 msgr2=0x7f3b7006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3b7006acc0 0x7f3b7006d180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c1546f0 tx=0x7f3b78008030 comp rx=0 tx=0).stop 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 msgr2=0x7f3b8c1e9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 0x7f3b8c1e9730 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000ebe0 tx=0x7f3b80009ec0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 shutdown_connections 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8c06a700 0x7f3b8c1e9730 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f3b7006acc0 0x7f3b7006d180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b8c0c4710 0x7f3b8c1ed540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 --2- 172.21.15.31:0/962479734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0c3180 0x7f3b8c158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 >> 172.21.15.31:0/962479734 conn(0x7f3b8c0c3a60 msgr2=0x7f3b8c14e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 shutdown_connections 2024-03-30T01:42:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.632+0000 7f3b92cbd700 1 -- 172.21.15.31:0/962479734 wait complete. 2024-03-30T01:42:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-03-30T01:42:49.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:49.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.783+0000 7f8d1ec7b700 1 Processor -- start 2024-03-30T01:42:49.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.783+0000 7f8d1ec7b700 1 -- start start 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ec7b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8d180ac520 0x7f8d180ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ec7b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8d180a74e0 0x7f8d180ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ec7b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d18153590 con 0x7f8d180c6000 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ec7b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d180ab680 con 0x7f8d180ac520 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ec7b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d181500c0 con 0x7f8d180a74e0 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ca17700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8d180ac520 0x7f8d180ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ca17700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8d180ac520 0x7f8d180ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35188/0 (socket says 172.21.15.31:35188) 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ca17700 1 -- 172.21.15.31:0/2949416903 learned_addr learned my addr 172.21.15.31:0/2949416903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d17fff700 1 --2- 172.21.15.31:0/2949416903 >> v2:172.21.15.59:3300/0 conn(0x7f8d180a74e0 0x7f8d180ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ca17700 1 -- 172.21.15.31:0/2949416903 >> v2:172.21.15.59:3300/0 conn(0x7f8d180a74e0 msgr2=0x7f8d180ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ca17700 1 --2- 172.21.15.31:0/2949416903 >> v2:172.21.15.59:3300/0 conn(0x7f8d180a74e0 0x7f8d180ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ca17700 1 -- 172.21.15.31:0/2949416903 >> v1:172.21.15.59:6789/0 conn(0x7f8d180c6000 legacy=0x7f8d180c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ca17700 1 -- 172.21.15.31:0/2949416903 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180c7990 con 0x7f8d180ac520 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.784+0000 7f8d1ca17700 1 --2- 172.21.15.31:0/2949416903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 0x7f8d180ac900 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0098f0 tx=0x7f8d0c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144bbe2920344952 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d177fe700 1 -- 172.21.15.31:0/2949416903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c004230 con 0x7f8d180ac520 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d177fe700 1 -- 172.21.15.31:0/2949416903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c004390 con 0x7f8d180ac520 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d177fe700 1 -- 172.21.15.31:0/2949416903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0036a0 con 0x7f8d180ac520 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/2949416903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 msgr2=0x7f8d180ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d1ec7b700 1 --2- 172.21.15.31:0/2949416903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 0x7f8d180ac900 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0098f0 tx=0x7f8d0c009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/2949416903 shutdown_connections 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d1ec7b700 1 --2- 172.21.15.31:0/2949416903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 0x7f8d180ac900 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d1ec7b700 1 --2- 172.21.15.31:0/2949416903 >> v2:172.21.15.59:3300/0 conn(0x7f8d180a74e0 0x7f8d180ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/2949416903 >> 172.21.15.31:0/2949416903 conn(0x7f8d1806ad30 msgr2=0x7f8d180a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/2949416903 shutdown_connections 2024-03-30T01:42:49.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.785+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/2949416903 wait complete. 2024-03-30T01:42:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.787+0000 7f8d1ec7b700 1 Processor -- start 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.787+0000 7f8d1ec7b700 1 -- start start 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.787+0000 7f8d1ec7b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d180a74e0 0x7f8d180a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.787+0000 7f8d1ec7b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 0x7f8d181e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d1ec7b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d180c6000 0x7f8d181e9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d1ec7b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d181e9110 con 0x7f8d180ac520 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d1ec7b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d181e8bb0 con 0x7f8d180c6000 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d1ec7b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d181ea050 con 0x7f8d180a74e0 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d17fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 0x7f8d181e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d17fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 0x7f8d181e95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35194/0 (socket says 172.21.15.31:35194) 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d17fff700 1 -- 172.21.15.31:0/4143192944 learned_addr learned my addr 172.21.15.31:0/4143192944 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d1ca17700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d180a74e0 0x7f8d180a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d1d218700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d180c6000 0x7f8d181e9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d17fff700 1 -- 172.21.15.31:0/4143192944 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d180a74e0 msgr2=0x7f8d180a91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d17fff700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d180a74e0 0x7f8d180a91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d17fff700 1 -- 172.21.15.31:0/4143192944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d180c6000 msgr2=0x7f8d181e9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d17fff700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d180c6000 0x7f8d181e9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d17fff700 1 -- 172.21.15.31:0/4143192944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c009550 con 0x7f8d180ac520 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d1ca17700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d180a74e0 0x7f8d180a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d17fff700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 0x7f8d181e95d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400c820 tx=0x7f8d0400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d157fa700 1 -- 172.21.15.31:0/4143192944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04011800 con 0x7f8d180ac520 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d157fa700 1 -- 172.21.15.31:0/4143192944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d04011960 con 0x7f8d180ac520 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d157fa700 1 -- 172.21.15.31:0/4143192944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04011d10 con 0x7f8d180ac520 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.788+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d18156720 con 0x7f8d180ac520 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.789+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d180bd520 con 0x7f8d180ac520 2024-03-30T01:42:49.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.789+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d180bd740 con 0x7f8d180ac520 2024-03-30T01:42:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.790+0000 7f8d157fa700 1 -- 172.21.15.31:0/4143192944 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d04018360 con 0x7f8d180ac520 2024-03-30T01:42:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.790+0000 7f8d157fa700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8cfc06ac90 0x7f8cfc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.790+0000 7f8d157fa700 1 -- 172.21.15.31:0/4143192944 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f8d04089770 con 0x7f8d180ac520 2024-03-30T01:42:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.791+0000 7f8d1ca17700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8cfc06ac90 0x7f8cfc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:49.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.791+0000 7f8d1ca17700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8cfc06ac90 0x7f8cfc06d150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8d1806fd00 tx=0x7f8d0c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:49.795+0000 7f8d157fa700 1 -- 172.21.15.31:0/4143192944 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d040544f0 con 0x7f8d180ac520 2024-03-30T01:42:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.016+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d180abd40 con 0x7f8d180ac520 2024-03-30T01:42:50.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.018+0000 7f8d157fa700 1 -- 172.21.15.31:0/4143192944 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20061 (secure 0 0 0) 0x7f8d04057b10 con 0x7f8d180ac520 2024-03-30T01:42:50.024 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:50.024 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:43.871647+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:50.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8cfc06ac90 msgr2=0x7f8cfc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8cfc06ac90 0x7f8cfc06d150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8d1806fd00 tx=0x7f8d0c008030 comp rx=0 tx=0).stop 2024-03-30T01:42:50.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 msgr2=0x7f8d181e95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 0x7f8d181e95d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400c820 tx=0x7f8d0400cbe0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 shutdown_connections 2024-03-30T01:42:50.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8cfc06ac90 0x7f8cfc06d150 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d180c6000 0x7f8d181e9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d180a74e0 0x7f8d180a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 --2- 172.21.15.31:0/4143192944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d180ac520 0x7f8d181e95d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 >> 172.21.15.31:0/4143192944 conn(0x7f8d1806ad30 msgr2=0x7f8d180c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.020+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 shutdown_connections 2024-03-30T01:42:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.021+0000 7f8d1ec7b700 1 -- 172.21.15.31:0/4143192944 wait complete. 2024-03-30T01:42:50.040 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T01:42:50.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:50.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.171+0000 7f038ac16700 1 Processor -- start 2024-03-30T01:42:50.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.171+0000 7f038ac16700 1 -- start start 2024-03-30T01:42:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f038ac16700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f038414fc30 0x7f038406a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f038ac16700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0384152650 0x7f0384152a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f038ac16700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03840c2f60 con 0x7f03840c1300 2024-03-30T01:42:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f038ac16700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03840c2a40 con 0x7f0384152650 2024-03-30T01:42:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f038ac16700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038406eca0 con 0x7f038414fc30 2024-03-30T01:42:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f03891b3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0384152650 0x7f0384152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f0383fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f03840c1300 0x7f038406ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36954/0 (socket says 172.21.15.31:36954) 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f0383fff700 1 -- 172.21.15.31:0/2025841576 learned_addr learned my addr 172.21.15.31:0/2025841576 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f03889b2700 1 --2- 172.21.15.31:0/2025841576 >> v2:172.21.15.59:3300/0 conn(0x7f038414fc30 0x7f038406a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f03891b3700 1 -- 172.21.15.31:0/2025841576 >> v2:172.21.15.59:3300/0 conn(0x7f038414fc30 msgr2=0x7f038406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f03891b3700 1 --2- 172.21.15.31:0/2025841576 >> v2:172.21.15.59:3300/0 conn(0x7f038414fc30 0x7f038406a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f03891b3700 1 -- 172.21.15.31:0/2025841576 >> v1:172.21.15.31:6789/0 conn(0x7f03840c1300 legacy=0x7f038406ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f03891b3700 1 -- 172.21.15.31:0/2025841576 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038406b350 con 0x7f0384152650 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f03837fe700 1 -- 172.21.15.31:0/2025841576 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36382592 0 0) 0x7f03840c2f60 con 0x7f03840c1300 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.172+0000 7f03889b2700 1 --2- 172.21.15.31:0/2025841576 >> v2:172.21.15.59:3300/0 conn(0x7f038414fc30 0x7f038406a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f03891b3700 1 --2- 172.21.15.31:0/2025841576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 0x7f0384152a80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f038406fe10 tx=0x7f037c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318fe3ef753c27de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f03837fe700 1 -- 172.21.15.31:0/2025841576 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00e070 con 0x7f0384152650 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f03837fe700 1 -- 172.21.15.31:0/2025841576 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c00bbd0 con 0x7f0384152650 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f03837fe700 1 -- 172.21.15.31:0/2025841576 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c012430 con 0x7f0384152650 2024-03-30T01:42:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f038ac16700 1 -- 172.21.15.31:0/2025841576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 msgr2=0x7f0384152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f038ac16700 1 --2- 172.21.15.31:0/2025841576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 0x7f0384152a80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f038406fe10 tx=0x7f037c00b330 comp rx=0 tx=0).stop 2024-03-30T01:42:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f038ac16700 1 -- 172.21.15.31:0/2025841576 shutdown_connections 2024-03-30T01:42:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f038ac16700 1 --2- 172.21.15.31:0/2025841576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 0x7f0384152a80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f038ac16700 1 --2- 172.21.15.31:0/2025841576 >> v2:172.21.15.59:3300/0 conn(0x7f038414fc30 0x7f038406a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f038ac16700 1 -- 172.21.15.31:0/2025841576 >> 172.21.15.31:0/2025841576 conn(0x7f03840c3a60 msgr2=0x7f03840c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f038ac16700 1 -- 172.21.15.31:0/2025841576 shutdown_connections 2024-03-30T01:42:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.173+0000 7f038ac16700 1 -- 172.21.15.31:0/2025841576 wait complete. 2024-03-30T01:42:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.175+0000 7f038ac16700 1 Processor -- start 2024-03-30T01:42:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.175+0000 7f038ac16700 1 -- start start 2024-03-30T01:42:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f038ac16700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03840c1300 0x7f038414e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f038ac16700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f038414fc30 0x7f03841e8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f038ac16700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 0x7f038414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f038ac16700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841e83b0 con 0x7f0384152650 2024-03-30T01:42:50.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f038ac16700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038414eef0 con 0x7f03840c1300 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f038ac16700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038414f1d0 con 0x7f038414fc30 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f03891b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 0x7f038414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f03891b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 0x7f038414b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35200/0 (socket says 172.21.15.31:35200) 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f03891b3700 1 -- 172.21.15.31:0/2611215476 learned_addr learned my addr 172.21.15.31:0/2611215476 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f03889b2700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03840c1300 0x7f038414e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.176+0000 7f0383fff700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f038414fc30 0x7f03841e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03891b3700 1 -- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f038414fc30 msgr2=0x7f03841e8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03891b3700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f038414fc30 0x7f03841e8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03891b3700 1 -- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03840c1300 msgr2=0x7f038414e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03891b3700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03840c1300 0x7f038414e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03891b3700 1 -- 172.21.15.31:0/2611215476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c009a70 con 0x7f0384152650 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03889b2700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03840c1300 0x7f038414e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f0383fff700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f038414fc30 0x7f03841e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03891b3700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 0x7f038414b3c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0384073220 tx=0x7f037c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03817fa700 1 -- 172.21.15.31:0/2611215476 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c012430 con 0x7f0384152650 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03817fa700 1 -- 172.21.15.31:0/2611215476 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c020070 con 0x7f0384152650 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03841e7d60 con 0x7f0384152650 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f03817fa700 1 -- 172.21.15.31:0/2611215476 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c004280 con 0x7f0384152650 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.177+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038414b900 con 0x7f0384152650 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.178+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03841e8560 con 0x7f0384152650 2024-03-30T01:42:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.179+0000 7f03817fa700 1 -- 172.21.15.31:0/2611215476 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f037c0043e0 con 0x7f0384152650 2024-03-30T01:42:50.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.179+0000 7f03817fa700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f036806ac90 0x7f036806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.179+0000 7f03817fa700 1 -- 172.21.15.31:0/2611215476 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f037c022070 con 0x7f0384152650 2024-03-30T01:42:50.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.179+0000 7f03889b2700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f036806ac90 0x7f036806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.180+0000 7f03889b2700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f036806ac90 0x7f036806d150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0384149f80 tx=0x7f0378008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:50.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.184+0000 7f03817fa700 1 -- 172.21.15.31:0/2611215476 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f037c0562d0 con 0x7f0384152650 2024-03-30T01:42:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.449+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f038404ea80 con 0x7f0384152650 2024-03-30T01:42:50.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.449+0000 7f03817fa700 1 -- 172.21.15.31:0/2611215476 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5942 (secure 0 0 0) 0x7f037c0598f0 con 0x7f0384152650 2024-03-30T01:42:50.455 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:50.455 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:10.264695+0000","modified":"2024-03-30T01:42:13.278071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/1616247357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1616247357},{"type":"v1","addr":"172.21.15.31:6837","nonce":1616247357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:24.267501+0000","modified":"2024-03-30T01:42:27.273615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T01:42:50.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f036806ac90 msgr2=0x7f036806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f036806ac90 0x7f036806d150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0384149f80 tx=0x7f0378008040 comp rx=0 tx=0).stop 2024-03-30T01:42:50.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 msgr2=0x7f038414b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 0x7f038414b3c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0384073220 tx=0x7f037c004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 shutdown_connections 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384152650 0x7f038414b3c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f036806ac90 0x7f036806d150 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03840c1300 0x7f038414e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 --2- 172.21.15.31:0/2611215476 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f038414fc30 0x7f03841e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 >> 172.21.15.31:0/2611215476 conn(0x7f03840c3a60 msgr2=0x7f0384143d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 shutdown_connections 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.452+0000 7f038ac16700 1 -- 172.21.15.31:0/2611215476 wait complete. 2024-03-30T01:42:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-03-30T01:42:50.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:50.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.602+0000 7fb1782e2700 1 Processor -- start 2024-03-30T01:42:50.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.602+0000 7fb1782e2700 1 -- start start 2024-03-30T01:42:50.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.603+0000 7fb1782e2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1700722f0 0x7fb170156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.603+0000 7fb1782e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb170152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.603+0000 7fb1782e2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb170153510 con 0x7fb17014fa40 2024-03-30T01:42:50.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.603+0000 7fb1782e2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb170156b50 con 0x7fb1700722f0 2024-03-30T01:42:50.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.603+0000 7fb1782e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb170152680 con 0x7fb170152b60 2024-03-30T01:42:50.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.603+0000 7fb17607e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb17014fa40 0x7fb1700c3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36964/0 (socket says 172.21.15.31:36964) 2024-03-30T01:42:50.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17607e700 1 -- 172.21.15.31:0/3368307495 learned_addr learned my addr 172.21.15.31:0/3368307495 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:50.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17687f700 1 --2- 172.21.15.31:0/3368307495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb170152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17587d700 1 --2- 172.21.15.31:0/3368307495 >> v2:172.21.15.59:3300/0 conn(0x7fb1700722f0 0x7fb170156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17507c700 1 -- 172.21.15.31:0/3368307495 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184516370 0 0) 0x7fb170153510 con 0x7fb17014fa40 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17507c700 1 -- 172.21.15.31:0/3368307495 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb15c0035b0 con 0x7fb17014fa40 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17687f700 1 -- 172.21.15.31:0/3368307495 >> v2:172.21.15.59:3300/0 conn(0x7fb1700722f0 msgr2=0x7fb170156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17687f700 1 --2- 172.21.15.31:0/3368307495 >> v2:172.21.15.59:3300/0 conn(0x7fb1700722f0 0x7fb170156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17687f700 1 -- 172.21.15.31:0/3368307495 >> v1:172.21.15.31:6789/0 conn(0x7fb17014fa40 legacy=0x7fb1700c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17687f700 1 -- 172.21.15.31:0/3368307495 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170071e70 con 0x7fb170152b60 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17587d700 1 --2- 172.21.15.31:0/3368307495 >> v2:172.21.15.59:3300/0 conn(0x7fb1700722f0 0x7fb170156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.604+0000 7fb17687f700 1 --2- 172.21.15.31:0/3368307495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb170152fd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb168009d90 tx=0x7fb16800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32871e093f6e4260 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb17507c700 1 -- 172.21.15.31:0/3368307495 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16800e070 con 0x7fb170152b60 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb17507c700 1 -- 172.21.15.31:0/3368307495 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16800bb20 con 0x7fb170152b60 2024-03-30T01:42:50.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb17507c700 1 -- 172.21.15.31:0/3368307495 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168012430 con 0x7fb170152b60 2024-03-30T01:42:50.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb1782e2700 1 -- 172.21.15.31:0/3368307495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 msgr2=0x7fb170152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb1782e2700 1 --2- 172.21.15.31:0/3368307495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb170152fd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb168009d90 tx=0x7fb16800b330 comp rx=0 tx=0).stop 2024-03-30T01:42:50.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb1782e2700 1 -- 172.21.15.31:0/3368307495 shutdown_connections 2024-03-30T01:42:50.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb1782e2700 1 --2- 172.21.15.31:0/3368307495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb170152fd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb1782e2700 1 --2- 172.21.15.31:0/3368307495 >> v2:172.21.15.59:3300/0 conn(0x7fb1700722f0 0x7fb170156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb1782e2700 1 -- 172.21.15.31:0/3368307495 >> 172.21.15.31:0/3368307495 conn(0x7fb17006ba20 msgr2=0x7fb17006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:50.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb1782e2700 1 -- 172.21.15.31:0/3368307495 shutdown_connections 2024-03-30T01:42:50.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.605+0000 7fb1782e2700 1 -- 172.21.15.31:0/3368307495 wait complete. 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb1782e2700 1 Processor -- start 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb1782e2700 1 -- start start 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb1782e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1700722f0 0x7fb1701dd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb1782e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17014fa40 0x7fb170150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb1782e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb17014b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb1782e2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700c7cc0 con 0x7fb1700722f0 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb1782e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170150000 con 0x7fb17014fa40 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb1782e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb170151aa0 con 0x7fb170152b60 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb17607e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1700722f0 0x7fb1701dd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb17607e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1700722f0 0x7fb1701dd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35216/0 (socket says 172.21.15.31:35216) 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.607+0000 7fb17607e700 1 -- 172.21.15.31:0/1550876227 learned_addr learned my addr 172.21.15.31:0/1550876227 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb17587d700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17014fa40 0x7fb170150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb17687f700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb17014b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb17607e700 1 -- 172.21.15.31:0/1550876227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 msgr2=0x7fb17014b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb17607e700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb17014b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb17607e700 1 -- 172.21.15.31:0/1550876227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17014fa40 msgr2=0x7fb170150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb17607e700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17014fa40 0x7fb170150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb17607e700 1 -- 172.21.15.31:0/1550876227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1640094a0 con 0x7fb1700722f0 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb17687f700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb17014b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb17607e700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1700722f0 0x7fb1701dd740 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c700 tx=0x7fb16c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb162ffd700 1 -- 172.21.15.31:0/1550876227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c005d70 con 0x7fb1700722f0 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb162ffd700 1 -- 172.21.15.31:0/1550876227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c0070d0 con 0x7fb1700722f0 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb168009a70 con 0x7fb1700722f0 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb162ffd700 1 -- 172.21.15.31:0/1550876227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c007460 con 0x7fb1700722f0 2024-03-30T01:42:50.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.608+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1701e84e0 con 0x7fb1700722f0 2024-03-30T01:42:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.609+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1700c2730 con 0x7fb1700722f0 2024-03-30T01:42:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.610+0000 7fb162ffd700 1 -- 172.21.15.31:0/1550876227 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb16c0075c0 con 0x7fb1700722f0 2024-03-30T01:42:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.610+0000 7fb162ffd700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb15406ac90 0x7fb15406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.610+0000 7fb162ffd700 1 -- 172.21.15.31:0/1550876227 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7fb16c088dd0 con 0x7fb1700722f0 2024-03-30T01:42:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.610+0000 7fb17587d700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb15406ac90 0x7fb15406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:50.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.611+0000 7fb17587d700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb15406ac90 0x7fb15406d150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb1701e8130 tx=0x7fb164008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:50.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.615+0000 7fb162ffd700 1 -- 172.21.15.31:0/1550876227 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb16c053ad0 con 0x7fb1700722f0 2024-03-30T01:42:50.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.880+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb17004ea80 con 0x7fb1700722f0 2024-03-30T01:42:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.880+0000 7fb162ffd700 1 -- 172.21.15.31:0/1550876227 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5942 (secure 0 0 0) 0x7fb16c0570f0 con 0x7fb1700722f0 2024-03-30T01:42:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:50.886 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:10.264695+0000","modified":"2024-03-30T01:42:13.278071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.31:6837/1616247357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":1616247357},{"type":"v1","addr":"172.21.15.31:6837","nonce":1616247357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:24.267501+0000","modified":"2024-03-30T01:42:27.273615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb15406ac90 msgr2=0x7fb15406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb15406ac90 0x7fb15406d150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb1701e8130 tx=0x7fb164008040 comp rx=0 tx=0).stop 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1700722f0 msgr2=0x7fb1701dd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1700722f0 0x7fb1701dd740 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c700 tx=0x7fb16c00cac0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 shutdown_connections 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1700722f0 0x7fb1701dd740 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb15406ac90 0x7fb15406d150 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb170152b60 0x7fb17014b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 --2- 172.21.15.31:0/1550876227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb17014fa40 0x7fb170150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 >> 172.21.15.31:0/1550876227 conn(0x7fb17006ba20 msgr2=0x7fb170152f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:50.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 shutdown_connections 2024-03-30T01:42:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:50.883+0000 7fb1782e2700 1 -- 172.21.15.31:0/1550876227 wait complete. 2024-03-30T01:42:50.890 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-03-30T01:42:50.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.034+0000 7fcbb42c2700 1 Processor -- start 2024-03-30T01:42:51.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.034+0000 7fcbb42c2700 1 -- start start 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.035+0000 7fcbb42c2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcbac152100 0x7fcbac0c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.035+0000 7fcbb42c2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcbac0c5dc0 0x7fcbac0c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.035+0000 7fcbb42c2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac070d60 con 0x7fcbac151d20 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.035+0000 7fcbb42c2700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac157e90 con 0x7fcbac152100 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.035+0000 7fcbb42c2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac158170 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb185d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcbac152100 0x7fcbac0c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb205e700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcbac151d20 0x7fcbac0705a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36974/0 (socket says 172.21.15.31:36974) 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb185d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcbac152100 0x7fcbac0c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35218/0 (socket says 172.21.15.31:35218) 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb205e700 1 -- 172.21.15.31:0/1222083401 learned_addr learned my addr 172.21.15.31:0/1222083401 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb285f700 1 --2- 172.21.15.31:0/1222083401 >> v2:172.21.15.59:3300/0 conn(0x7fcbac0c5dc0 0x7fcbac0c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb185d700 1 -- 172.21.15.31:0/1222083401 >> v2:172.21.15.59:3300/0 conn(0x7fcbac0c5dc0 msgr2=0x7fcbac0c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb185d700 1 --2- 172.21.15.31:0/1222083401 >> v2:172.21.15.59:3300/0 conn(0x7fcbac0c5dc0 0x7fcbac0c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb185d700 1 -- 172.21.15.31:0/1222083401 >> v1:172.21.15.31:6789/0 conn(0x7fcbac151d20 legacy=0x7fcbac0705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb185d700 1 -- 172.21.15.31:0/1222083401 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac158450 con 0x7fcbac152100 2024-03-30T01:42:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb105c700 1 -- 172.21.15.31:0/1222083401 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792500442 0 0) 0x7fcbac070d60 con 0x7fcbac151d20 2024-03-30T01:42:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb185d700 1 --2- 172.21.15.31:0/1222083401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac152100 0x7fcbac0c4850 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009840 tx=0x7fcba0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9087f96fdbf4ca2c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb105c700 1 -- 172.21.15.31:0/1222083401 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0011830 con 0x7fcbac152100 2024-03-30T01:42:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb105c700 1 -- 172.21.15.31:0/1222083401 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba0011990 con 0x7fcbac152100 2024-03-30T01:42:51.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.036+0000 7fcbb105c700 1 -- 172.21.15.31:0/1222083401 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0011d20 con 0x7fcbac152100 2024-03-30T01:42:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.037+0000 7fcbb42c2700 1 -- 172.21.15.31:0/1222083401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac152100 msgr2=0x7fcbac0c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.037+0000 7fcbb42c2700 1 --2- 172.21.15.31:0/1222083401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac152100 0x7fcbac0c4850 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009840 tx=0x7fcba0009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.037+0000 7fcbb42c2700 1 -- 172.21.15.31:0/1222083401 shutdown_connections 2024-03-30T01:42:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.037+0000 7fcbb42c2700 1 --2- 172.21.15.31:0/1222083401 >> v2:172.21.15.59:3300/0 conn(0x7fcbac0c5dc0 0x7fcbac0c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.037+0000 7fcbb42c2700 1 --2- 172.21.15.31:0/1222083401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac152100 0x7fcbac0c4850 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.037+0000 7fcbb42c2700 1 -- 172.21.15.31:0/1222083401 >> 172.21.15.31:0/1222083401 conn(0x7fcbac06ad30 msgr2=0x7fcbac0a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.037+0000 7fcbb42c2700 1 -- 172.21.15.31:0/1222083401 shutdown_connections 2024-03-30T01:42:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.037+0000 7fcbb42c2700 1 -- 172.21.15.31:0/1222083401 wait complete. 2024-03-30T01:42:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.038+0000 7fcbb42c2700 1 Processor -- start 2024-03-30T01:42:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.038+0000 7fcbb42c2700 1 -- start start 2024-03-30T01:42:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb42c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac0c5dc0 0x7fcbac147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb42c2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcbac151d20 0x7fcbac0c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb42c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbac152100 0x7fcbac1ed690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb42c2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac1eab50 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb42c2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac157870 con 0x7fcbac152100 2024-03-30T01:42:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb42c2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac06f9a0 con 0x7fcbac151d20 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb205e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac0c5dc0 0x7fcbac147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb205e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac0c5dc0 0x7fcbac147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35234/0 (socket says 172.21.15.31:35234) 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb205e700 1 -- 172.21.15.31:0/967648289 learned_addr learned my addr 172.21.15.31:0/967648289 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb185d700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcbac151d20 0x7fcbac0c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.039+0000 7fcbb285f700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbac152100 0x7fcbac1ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcbb205e700 1 -- 172.21.15.31:0/967648289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcbac151d20 msgr2=0x7fcbac0c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcbb205e700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcbac151d20 0x7fcbac0c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcbb205e700 1 -- 172.21.15.31:0/967648289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbac152100 msgr2=0x7fcbac1ed690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcbb205e700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbac152100 0x7fcbac1ed690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcbb205e700 1 -- 172.21.15.31:0/967648289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00094a0 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcbb205e700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac0c5dc0 0x7fcbac147c80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcba80011e0 tx=0x7fcba800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcb9effd700 1 -- 172.21.15.31:0/967648289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8004c80 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcb9effd700 1 -- 172.21.15.31:0/967648289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba8004e00 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcb9effd700 1 -- 172.21.15.31:0/967648289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8007310 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac06f3e0 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.040+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac1ed170 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.041+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac04ea80 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.042+0000 7fcb9effd700 1 -- 172.21.15.31:0/967648289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcba8007470 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.042+0000 7fcb9effd700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb9006ac90 0x7fcb9006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.042+0000 7fcb9effd700 1 -- 172.21.15.31:0/967648289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7fcba8089450 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.042+0000 7fcbb185d700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb9006ac90 0x7fcb9006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.043+0000 7fcbb185d700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb9006ac90 0x7fcb9006d150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcba00090b0 tx=0x7fcba001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:51.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.047+0000 7fcb9effd700 1 -- 172.21.15.31:0/967648289 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcba8054140 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.259+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbac0c31c0 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.261+0000 7fcb9effd700 1 -- 172.21.15.31:0/967648289 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+20061 (secure 0 0 0) 0x7fcba8057760 con 0x7fcbac0c5dc0 2024-03-30T01:42:51.268 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:51.268 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:43.871647+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:42:07.881738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T01:42:09.236870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:51.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb9006ac90 msgr2=0x7fcb9006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb9006ac90 0x7fcb9006d150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcba00090b0 tx=0x7fcba001c040 comp rx=0 tx=0).stop 2024-03-30T01:42:51.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac0c5dc0 msgr2=0x7fcbac147c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac0c5dc0 0x7fcbac147c80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcba80011e0 tx=0x7fcba800ca30 comp rx=0 tx=0).stop 2024-03-30T01:42:51.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 shutdown_connections 2024-03-30T01:42:51.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac0c5dc0 0x7fcbac147c80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcb9006ac90 0x7fcb9006d150 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcbac152100 0x7fcbac1ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 --2- 172.21.15.31:0/967648289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcbac151d20 0x7fcbac0c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 >> 172.21.15.31:0/967648289 conn(0x7fcbac06ad30 msgr2=0x7fcbac0a9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:51.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.263+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 shutdown_connections 2024-03-30T01:42:51.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.264+0000 7fcbb42c2700 1 -- 172.21.15.31:0/967648289 wait complete. 2024-03-30T01:42:51.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T01:42:51.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.415+0000 7f6be6f35700 1 Processor -- start 2024-03-30T01:42:51.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.415+0000 7f6be6f35700 1 -- start start 2024-03-30T01:42:51.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6be6f35700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6be014fa40 0x7f6be0156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6be6f35700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6be00ab100 0x7f6be00c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6be6f35700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be0071ee0 con 0x7f6be00c4100 2024-03-30T01:42:51.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6be6f35700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be0152570 con 0x7f6be00ab100 2024-03-30T01:42:51.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6be6f35700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be006d5c0 con 0x7f6be014fa40 2024-03-30T01:42:51.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6bdffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6be00ab100 0x7f6be00c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6bdffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6be00ab100 0x7f6be00c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35244/0 (socket says 172.21.15.31:35244) 2024-03-30T01:42:51.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6bdffff700 1 -- 172.21.15.31:0/1327564186 learned_addr learned my addr 172.21.15.31:0/1327564186 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:51.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6be4cd1700 1 --2- 172.21.15.31:0/1327564186 >> v2:172.21.15.59:3300/0 conn(0x7f6be014fa40 0x7f6be0156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6bdffff700 1 -- 172.21.15.31:0/1327564186 >> v2:172.21.15.59:3300/0 conn(0x7f6be014fa40 msgr2=0x7f6be0156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6bdffff700 1 --2- 172.21.15.31:0/1327564186 >> v2:172.21.15.59:3300/0 conn(0x7f6be014fa40 0x7f6be0156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6bdffff700 1 -- 172.21.15.31:0/1327564186 >> v1:172.21.15.59:6789/0 conn(0x7f6be00c4100 legacy=0x7f6be0152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6bdffff700 1 -- 172.21.15.31:0/1327564186 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be006d860 con 0x7f6be00ab100 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.416+0000 7f6bdffff700 1 --2- 172.21.15.31:0/1327564186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 0x7f6be00c3bc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6be00c4720 tx=0x7f6bcc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edfce18f329c624 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6bdf7fe700 1 -- 172.21.15.31:0/1327564186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc004230 con 0x7f6be00ab100 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6bdf7fe700 1 -- 172.21.15.31:0/1327564186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc004390 con 0x7f6be00ab100 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6bdf7fe700 1 -- 172.21.15.31:0/1327564186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0036a0 con 0x7f6be00ab100 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6be6f35700 1 -- 172.21.15.31:0/1327564186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 msgr2=0x7f6be00c3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6be6f35700 1 --2- 172.21.15.31:0/1327564186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 0x7f6be00c3bc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6be00c4720 tx=0x7f6bcc009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6be6f35700 1 -- 172.21.15.31:0/1327564186 shutdown_connections 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6be6f35700 1 --2- 172.21.15.31:0/1327564186 >> v2:172.21.15.59:3300/0 conn(0x7f6be014fa40 0x7f6be0156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6be6f35700 1 --2- 172.21.15.31:0/1327564186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 0x7f6be00c3bc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6be6f35700 1 -- 172.21.15.31:0/1327564186 >> 172.21.15.31:0/1327564186 conn(0x7f6be006ba20 msgr2=0x7f6be006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6be6f35700 1 -- 172.21.15.31:0/1327564186 shutdown_connections 2024-03-30T01:42:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.417+0000 7f6be6f35700 1 -- 172.21.15.31:0/1327564186 wait complete. 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.419+0000 7f6be6f35700 1 Processor -- start 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.419+0000 7f6be6f35700 1 -- start start 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be6f35700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 0x7f6be00c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be6f35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6be00c4100 0x7f6be01517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be6f35700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6be014fa40 0x7f6be0145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be6f35700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be0155080 con 0x7f6be00ab100 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be6f35700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be01eaba0 con 0x7f6be00c4100 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be6f35700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00c64d0 con 0x7f6be014fa40 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be4cd1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 0x7f6be00c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be4cd1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 0x7f6be00c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35246/0 (socket says 172.21.15.31:35246) 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be4cd1700 1 -- 172.21.15.31:0/1972769779 learned_addr learned my addr 172.21.15.31:0/1972769779 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6bdffff700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6be00c4100 0x7f6be01517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.420+0000 7f6be54d2700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6be014fa40 0x7f6be0145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6be4cd1700 1 -- 172.21.15.31:0/1972769779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6be014fa40 msgr2=0x7f6be0145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6be4cd1700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6be014fa40 0x7f6be0145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6be4cd1700 1 -- 172.21.15.31:0/1972769779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6be00c4100 msgr2=0x7f6be01517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6be4cd1700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6be00c4100 0x7f6be01517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6be4cd1700 1 -- 172.21.15.31:0/1972769779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc009550 con 0x7f6be00ab100 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6be4cd1700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 0x7f6be00c5f90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400d830 tx=0x7f6bd400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6bdd7fa700 1 -- 172.21.15.31:0/1972769779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40125f0 con 0x7f6be00ab100 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6bdd7fa700 1 -- 172.21.15.31:0/1972769779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4012750 con 0x7f6be00ab100 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be01eadf0 con 0x7f6be00ab100 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6bdd7fa700 1 -- 172.21.15.31:0/1972769779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4012b00 con 0x7f6be00ab100 2024-03-30T01:42:51.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.421+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be01eb3b0 con 0x7f6be00ab100 2024-03-30T01:42:51.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.422+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be00c15f0 con 0x7f6be00ab100 2024-03-30T01:42:51.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.423+0000 7f6bdd7fa700 1 -- 172.21.15.31:0/1972769779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6bd4012c60 con 0x7f6be00ab100 2024-03-30T01:42:51.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.423+0000 7f6bdd7fa700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6bc406acc0 0x7f6bc406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.423+0000 7f6bdd7fa700 1 -- 172.21.15.31:0/1972769779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6972+0+0 (secure 0 0 0) 0x7f6bd4089cb0 con 0x7f6be00ab100 2024-03-30T01:42:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.423+0000 7f6bdffff700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6bc406acc0 0x7f6bc406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.424+0000 7f6bdffff700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6bc406acc0 0x7f6bc406d180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009130 tx=0x7f6bcc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:51.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.428+0000 7f6bdd7fa700 1 -- 172.21.15.31:0/1972769779 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6bd4054910 con 0x7f6be00ab100 2024-03-30T01:42:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.697+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6be004ea80 con 0x7f6be00ab100 2024-03-30T01:42:51.704 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-03-30T01:42:51.698+0000 7fd8b60ee700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T01:42:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.705+0000 7f6bdd7fa700 1 -- 172.21.15.31:0/1972769779 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6bd4057f30 con 0x7f6be00ab100 2024-03-30T01:42:51.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6bc406acc0 msgr2=0x7f6bc406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6bc406acc0 0x7f6bc406d180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009130 tx=0x7f6bcc008030 comp rx=0 tx=0).stop 2024-03-30T01:42:51.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 msgr2=0x7f6be00c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 0x7f6be00c5f90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400d830 tx=0x7f6bd400dbf0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 shutdown_connections 2024-03-30T01:42:51.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6be00ab100 0x7f6be00c5f90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6bc406acc0 0x7f6bc406d180 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6be014fa40 0x7f6be0145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 --2- 172.21.15.31:0/1972769779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6be00c4100 0x7f6be01517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 >> 172.21.15.31:0/1972769779 conn(0x7f6be006ba20 msgr2=0x7f6be00c76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:51.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 shutdown_connections 2024-03-30T01:42:51.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.708+0000 7f6be6f35700 1 -- 172.21.15.31:0/1972769779 wait complete. 2024-03-30T01:42:51.716 INFO:tasks.ceph.mds.c.smithi031.stderr: -4307> 2024-03-30T01:42:12.786+0000 7fbce7086700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T01:42:51.717 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T01:42:51.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T01:42:51.749 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2024-03-30T01:42:51.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.864+0000 7f6b11a24700 1 Processor -- start 2024-03-30T01:42:51.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.864+0000 7f6b11a24700 1 -- start start 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.864+0000 7f6b11a24700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6b0c072a50 0x7f6b0c071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.864+0000 7f6b11a24700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b0c072410 0x7f6b0c0bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.864+0000 7f6b11a24700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c158af0 con 0x7f6b0c14e940 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.864+0000 7f6b11a24700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c157c90 con 0x7f6b0c072410 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.864+0000 7f6b11a24700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c157ee0 con 0x7f6b0c072a50 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b0b7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b0c072410 0x7f6b0c0bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b0affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6b0c14e940 0x7f6b0c155120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36990/0 (socket says 172.21.15.31:36990) 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b0affd700 1 -- 172.21.15.31:0/3164392266 learned_addr learned my addr 172.21.15.31:0/3164392266 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b0a7fc700 1 --2- 172.21.15.31:0/3164392266 >> v2:172.21.15.59:3300/0 conn(0x7f6b0c072a50 0x7f6b0c071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b0b7fe700 1 -- 172.21.15.31:0/3164392266 >> v2:172.21.15.59:3300/0 conn(0x7f6b0c072a50 msgr2=0x7f6b0c071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b0b7fe700 1 --2- 172.21.15.31:0/3164392266 >> v2:172.21.15.59:3300/0 conn(0x7f6b0c072a50 0x7f6b0c071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b0b7fe700 1 -- 172.21.15.31:0/3164392266 >> v1:172.21.15.31:6789/0 conn(0x7f6b0c14e940 legacy=0x7f6b0c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b0b7fe700 1 -- 172.21.15.31:0/3164392266 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c1581c0 con 0x7f6b0c072410 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b09ffb700 1 -- 172.21.15.31:0/3164392266 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845571293 0 0) 0x7f6b0c158af0 con 0x7f6b0c14e940 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b0b7fe700 1 --2- 172.21.15.31:0/3164392266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c072410 0x7f6b0c0bfa90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0ac820 tx=0x7f6b0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a986448e35e73620 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b09ffb700 1 -- 172.21.15.31:0/3164392266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400e070 con 0x7f6b0c072410 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b09ffb700 1 -- 172.21.15.31:0/3164392266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0400bbd0 con 0x7f6b0c072410 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.865+0000 7f6b09ffb700 1 -- 172.21.15.31:0/3164392266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04012430 con 0x7f6b0c072410 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.866+0000 7f6b11a24700 1 -- 172.21.15.31:0/3164392266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c072410 msgr2=0x7f6b0c0bfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.866+0000 7f6b11a24700 1 --2- 172.21.15.31:0/3164392266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c072410 0x7f6b0c0bfa90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0ac820 tx=0x7f6b0400b330 comp rx=0 tx=0).stop 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.866+0000 7f6b11a24700 1 -- 172.21.15.31:0/3164392266 shutdown_connections 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.866+0000 7f6b11a24700 1 --2- 172.21.15.31:0/3164392266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c072410 0x7f6b0c0bfa90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.866+0000 7f6b11a24700 1 --2- 172.21.15.31:0/3164392266 >> v2:172.21.15.59:3300/0 conn(0x7f6b0c072a50 0x7f6b0c071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.866+0000 7f6b11a24700 1 -- 172.21.15.31:0/3164392266 >> 172.21.15.31:0/3164392266 conn(0x7f6b0c06e4e0 msgr2=0x7f6b0c0c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.866+0000 7f6b11a24700 1 -- 172.21.15.31:0/3164392266 shutdown_connections 2024-03-30T01:42:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.866+0000 7f6b11a24700 1 -- 172.21.15.31:0/3164392266 wait complete. 2024-03-30T01:42:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.867+0000 7f6b11a24700 1 Processor -- start 2024-03-30T01:42:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b11a24700 1 -- start start 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b11a24700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b0c072410 0x7f6b0c153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b11a24700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b0c072a50 0x7f6b0c0a9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b11a24700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c14e940 0x7f6b0c0a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b11a24700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c151950 con 0x7f6b0c14e940 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b11a24700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c147d50 con 0x7f6b0c072410 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b11a24700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c150fb0 con 0x7f6b0c072a50 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c14e940 0x7f6b0c0a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c14e940 0x7f6b0c0a9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35282/0 (socket says 172.21.15.31:35282) 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.868+0000 7f6b0b7fe700 1 -- 172.21.15.31:0/935412360 learned_addr learned my addr 172.21.15.31:0/935412360 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b0affd700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b0c072410 0x7f6b0c153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b0a7fc700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b0c072a50 0x7f6b0c0a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b0b7fe700 1 -- 172.21.15.31:0/935412360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b0c072a50 msgr2=0x7f6b0c0a9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b0b7fe700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b0c072a50 0x7f6b0c0a9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b0b7fe700 1 -- 172.21.15.31:0/935412360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b0c072410 msgr2=0x7f6b0c153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b0b7fe700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b0c072410 0x7f6b0c153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b0b7fe700 1 -- 172.21.15.31:0/935412360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b04009a70 con 0x7f6b0c14e940 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b0b7fe700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c14e940 0x7f6b0c0a9860 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400b730 tx=0x7f6b04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6af3fff700 1 -- 172.21.15.31:0/935412360 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04012430 con 0x7f6b0c14e940 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6af3fff700 1 -- 172.21.15.31:0/935412360 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b04020070 con 0x7f6b0c14e940 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6af3fff700 1 -- 172.21.15.31:0/935412360 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04004280 con 0x7f6b0c14e940 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c151290 con 0x7f6b0c14e940 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.869+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c153a50 con 0x7f6b0c14e940 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.870+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c0747d0 con 0x7f6b0c14e940 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.871+0000 7f6af3fff700 1 -- 172.21.15.31:0/935412360 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6b040043e0 con 0x7f6b0c14e940 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.871+0000 7f6af3fff700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6af806ac90 0x7f6af806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.871+0000 7f6af3fff700 1 -- 172.21.15.31:0/935412360 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7058+0+0 (secure 0 0 0) 0x7f6b04022070 con 0x7f6b0c14e940 2024-03-30T01:42:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.871+0000 7f6b0affd700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6af806ac90 0x7f6af806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:51.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.872+0000 7f6b0affd700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6af806ac90 0x7f6af806d150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6afc000f60 tx=0x7f6afc007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:51.876+0000 7f6af3fff700 1 -- 172.21.15.31:0/935412360 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6b040566e0 con 0x7f6b0c14e940 2024-03-30T01:42:52.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.144+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6b0c06be80 con 0x7f6b0c14e940 2024-03-30T01:42:52.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.712+0000 7f6af3fff700 1 -- 172.21.15.31:0/935412360 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6b04059d00 con 0x7f6b0c14e940 2024-03-30T01:42:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.715+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6af806ac90 msgr2=0x7f6af806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.715+0000 7f6b11a24700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6af806ac90 0x7f6af806d150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6afc000f60 tx=0x7f6afc007c90 comp rx=0 tx=0).stop 2024-03-30T01:42:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.715+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c14e940 msgr2=0x7f6b0c0a9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.715+0000 7f6b11a24700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c14e940 0x7f6b0c0a9860 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400b730 tx=0x7f6b04004ad0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.716+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 shutdown_connections 2024-03-30T01:42:52.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.716+0000 7f6b11a24700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c14e940 0x7f6b0c0a9860 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.716+0000 7f6b11a24700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6af806ac90 0x7f6af806d150 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.716+0000 7f6b11a24700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b0c072410 0x7f6b0c153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.716+0000 7f6b11a24700 1 --2- 172.21.15.31:0/935412360 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b0c072a50 0x7f6b0c0a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.716+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 >> 172.21.15.31:0/935412360 conn(0x7f6b0c06e4e0 msgr2=0x7f6b0c06af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:52.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.716+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 shutdown_connections 2024-03-30T01:42:52.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.716+0000 7f6b11a24700 1 -- 172.21.15.31:0/935412360 wait complete. 2024-03-30T01:42:52.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T01:42:52.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.923+0000 7f8633ccb700 1 Processor -- start 2024-03-30T01:42:52.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.923+0000 7f8633ccb700 1 -- start start 2024-03-30T01:42:52.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8633ccb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f862c0a6e40 0x7f862c06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:52.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8633ccb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c070880 0x7f862c0c1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:52.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8633ccb700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c158660 con 0x7f862c155820 2024-03-30T01:42:52.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8633ccb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862c0c2580 con 0x7f862c0a6e40 2024-03-30T01:42:52.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8633ccb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c070c90 con 0x7f862c070880 2024-03-30T01:42:52.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8631a67700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f862c155820 0x7f862c157f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36998/0 (socket says 172.21.15.31:36998) 2024-03-30T01:42:52.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8631a67700 1 -- 172.21.15.31:0/2914105299 learned_addr learned my addr 172.21.15.31:0/2914105299 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:52.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8631266700 1 --2- 172.21.15.31:0/2914105299 >> v2:172.21.15.59:3300/0 conn(0x7f862c0a6e40 0x7f862c06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:52.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8632268700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c070880 0x7f862c0c1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:52.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092324922 0 0) 0x7f862c158660 con 0x7f862c155820 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86180035b0 con 0x7f862c155820 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 790360745 0 0) 0x7f86180035b0 con 0x7f862c155820 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c070880 msgr2=0x7f862c0c1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8630a65700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c070880 0x7f862c0c1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 >> v2:172.21.15.59:3300/0 conn(0x7f862c0a6e40 msgr2=0x7f862c06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8630a65700 1 --2- 172.21.15.31:0/2914105299 >> v2:172.21.15.59:3300/0 conn(0x7f862c0a6e40 0x7f862c06cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.924+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0c7320 con 0x7f862c155820 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f8620001670 con 0x7f862c155820 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 >> v1:172.21.15.31:6789/0 conn(0x7f862c155820 legacy=0x7f862c157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8618003a00 0x7f8618003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c070880 0x7f8618004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618004d90 0x7f8618005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c070c90 con 0x7f8618003a00 2024-03-30T01:42:52.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861800ba10 con 0x7f8618004d90 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861800bb50 con 0x7f862c070880 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8631266700 1 --2- 172.21.15.31:0/2914105299 >> v2:172.21.15.59:3300/0 conn(0x7f862c0a6e40 0x7f862c06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8631a67700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8618003a00 0x7f8618003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8632268700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618004d90 0x7f8618005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8631266700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c070880 0x7f8618004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8631a67700 1 -- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c070880 msgr2=0x7f8618004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8631a67700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c070880 0x7f8618004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8631a67700 1 -- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618004d90 msgr2=0x7f8618005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8631a67700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618004d90 0x7f8618005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8631a67700 1 -- 172.21.15.31:0/2914105299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180035b0 con 0x7f8618003a00 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8632268700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618004d90 0x7f8618005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8631a67700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8618003a00 0x7f8618003e20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f862000c320 tx=0x7f8618005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f855fb05c8769b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:52.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862000ce60 con 0x7f8618003a00 2024-03-30T01:42:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8620005960 con 0x7f8618003a00 2024-03-30T01:42:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.925+0000 7f8630a65700 1 -- 172.21.15.31:0/2914105299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8620005cf0 con 0x7f8618003a00 2024-03-30T01:42:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.926+0000 7f8633ccb700 1 -- 172.21.15.31:0/2914105299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8618003a00 msgr2=0x7f8618003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.926+0000 7f8633ccb700 1 --2- 172.21.15.31:0/2914105299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8618003a00 0x7f8618003e20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f862000c320 tx=0x7f8618005880 comp rx=0 tx=0).stop 2024-03-30T01:42:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.926+0000 7f8632268700 1 -- 172.21.15.31:0/2914105299 reap_dead start 2024-03-30T01:42:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.926+0000 7f8633ccb700 1 -- 172.21.15.31:0/2914105299 shutdown_connections 2024-03-30T01:42:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.926+0000 7f8633ccb700 1 -- 172.21.15.31:0/2914105299 >> 172.21.15.31:0/2914105299 conn(0x7f862c0c48d0 msgr2=0x7f862c0c4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.926+0000 7f8633ccb700 1 -- 172.21.15.31:0/2914105299 shutdown_connections 2024-03-30T01:42:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.926+0000 7f8633ccb700 1 -- 172.21.15.31:0/2914105299 wait complete. 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.928+0000 7f8633ccb700 1 Processor -- start 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.928+0000 7f8633ccb700 1 -- start start 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8633ccb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618003a00 0x7f862c06fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8633ccb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c0a7870 0x7f862c1ea410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8633ccb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f862c1ea950 0x7f862c14eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8633ccb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0a8600 con 0x7f862c1ea950 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8633ccb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1ee390 con 0x7f8618003a00 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8633ccb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1ee670 con 0x7f862c0a7870 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8632268700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f862c1ea950 0x7f862c14eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8632268700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f862c1ea950 0x7f862c14eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35292/0 (socket says 172.21.15.31:35292) 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8632268700 1 -- 172.21.15.31:0/1777078596 learned_addr learned my addr 172.21.15.31:0/1777078596 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:52.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8631266700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c0a7870 0x7f862c1ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8631a67700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618003a00 0x7f862c06fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8632268700 1 -- 172.21.15.31:0/1777078596 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c0a7870 msgr2=0x7f862c1ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8632268700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c0a7870 0x7f862c1ea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8632268700 1 -- 172.21.15.31:0/1777078596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618003a00 msgr2=0x7f862c06fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8632268700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618003a00 0x7f862c06fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8632268700 1 -- 172.21.15.31:0/1777078596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0a8290 con 0x7f862c1ea950 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8631a67700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618003a00 0x7f862c06fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.929+0000 7f8632268700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f862c1ea950 0x7f862c14eb00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f862c0731e0 tx=0x7f86240104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.930+0000 7f861e7fc700 1 -- 172.21.15.31:0/1777078596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862401b070 con 0x7f862c1ea950 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.930+0000 7f861e7fc700 1 -- 172.21.15.31:0/1777078596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862400f040 con 0x7f862c1ea950 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.930+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c149250 con 0x7f862c1ea950 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.930+0000 7f861e7fc700 1 -- 172.21.15.31:0/1777078596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862400e050 con 0x7f862c1ea950 2024-03-30T01:42:52.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.930+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c149980 con 0x7f862c1ea950 2024-03-30T01:42:52.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.931+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862c04ea80 con 0x7f862c1ea950 2024-03-30T01:42:52.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.931+0000 7f861e7fc700 1 -- 172.21.15.31:0/1777078596 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8624015440 con 0x7f862c1ea950 2024-03-30T01:42:52.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.931+0000 7f861e7fc700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f861006ac90 0x7f861006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:52.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.932+0000 7f861e7fc700 1 -- 172.21.15.31:0/1777078596 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7058+0+0 (secure 0 0 0) 0x7f8624089930 con 0x7f862c1ea950 2024-03-30T01:42:52.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.932+0000 7f8631a67700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f861006ac90 0x7f861006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:52.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.932+0000 7f8631a67700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f861006ac90 0x7f861006d150 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8620015040 tx=0x7f862000a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:52.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:52.936+0000 7f861e7fc700 1 -- 172.21.15.31:0/1777078596 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8624054670 con 0x7f862c1ea950 2024-03-30T01:42:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.151+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f862c0a8420 con 0x7f862c1ea950 2024-03-30T01:42:53.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.902+0000 7f861e7fc700 1 -- 172.21.15.31:0/1777078596 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8624057c90 con 0x7f862c1ea950 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.907+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f861006ac90 msgr2=0x7f861006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.907+0000 7f8633ccb700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f861006ac90 0x7f861006d150 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8620015040 tx=0x7f862000a000 comp rx=0 tx=0).stop 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.907+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f862c1ea950 msgr2=0x7f862c14eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.907+0000 7f8633ccb700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f862c1ea950 0x7f862c14eb00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f862c0731e0 tx=0x7f86240104a0 comp rx=0 tx=0).stop 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.908+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 shutdown_connections 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.908+0000 7f8633ccb700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f862c1ea950 0x7f862c14eb00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.908+0000 7f8633ccb700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f861006ac90 0x7f861006d150 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.908+0000 7f8633ccb700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8618003a00 0x7f862c06fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.908+0000 7f8633ccb700 1 --2- 172.21.15.31:0/1777078596 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f862c0a7870 0x7f862c1ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.908+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 >> 172.21.15.31:0/1777078596 conn(0x7f862c0c48d0 msgr2=0x7f862c0c3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.908+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 shutdown_connections 2024-03-30T01:42:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:53.908+0000 7f8633ccb700 1 -- 172.21.15.31:0/1777078596 wait complete. 2024-03-30T01:42:53.916 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-03-30T01:42:53.931 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T01:42:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.069+0000 7f5f87908700 1 Processor -- start 2024-03-30T01:42:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.070+0000 7f5f87908700 1 -- start start 2024-03-30T01:42:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.070+0000 7f5f87908700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f8013dba0 0x7f5f8006e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.070+0000 7f5f87908700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f8006ecb0 0x7f5f80157e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.070+0000 7f5f87908700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f800c0c50 con 0x7f5f80158370 2024-03-30T01:42:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.070+0000 7f5f87908700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f801556a0 con 0x7f5f8006ecb0 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.070+0000 7f5f87908700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800c0320 con 0x7f5f8013dba0 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f84ea3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f8006ecb0 0x7f5f80157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f84ea3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f8006ecb0 0x7f5f80157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35296/0 (socket says 172.21.15.31:35296) 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f84ea3700 1 -- 172.21.15.31:0/1132887272 learned_addr learned my addr 172.21.15.31:0/1132887272 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f856a4700 1 --2- 172.21.15.31:0/1132887272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f8013dba0 0x7f5f8006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f84ea3700 1 -- 172.21.15.31:0/1132887272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f8013dba0 msgr2=0x7f5f8006e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f84ea3700 1 --2- 172.21.15.31:0/1132887272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f8013dba0 0x7f5f8006e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f84ea3700 1 -- 172.21.15.31:0/1132887272 >> v1:172.21.15.59:6789/0 conn(0x7f5f80158370 legacy=0x7f5f8006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f84ea3700 1 -- 172.21.15.31:0/1132887272 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f800c0600 con 0x7f5f8006ecb0 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f84ea3700 1 --2- 172.21.15.31:0/1132887272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 0x7f5f80157e30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0098f0 tx=0x7f5f7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4f858620544153 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f77fff700 1 -- 172.21.15.31:0/1132887272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c00b8a0 con 0x7f5f8006ecb0 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f77fff700 1 -- 172.21.15.31:0/1132887272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c00ba00 con 0x7f5f8006ecb0 2024-03-30T01:42:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.071+0000 7f5f77fff700 1 -- 172.21.15.31:0/1132887272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c00bd90 con 0x7f5f8006ecb0 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.072+0000 7f5f87908700 1 -- 172.21.15.31:0/1132887272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 msgr2=0x7f5f80157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.072+0000 7f5f87908700 1 --2- 172.21.15.31:0/1132887272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 0x7f5f80157e30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0098f0 tx=0x7f5f7c009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.072+0000 7f5f87908700 1 -- 172.21.15.31:0/1132887272 shutdown_connections 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.072+0000 7f5f87908700 1 --2- 172.21.15.31:0/1132887272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f8013dba0 0x7f5f8006e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.072+0000 7f5f87908700 1 --2- 172.21.15.31:0/1132887272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 0x7f5f80157e30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.072+0000 7f5f87908700 1 -- 172.21.15.31:0/1132887272 >> 172.21.15.31:0/1132887272 conn(0x7f5f800c39f0 msgr2=0x7f5f800c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.072+0000 7f5f87908700 1 -- 172.21.15.31:0/1132887272 shutdown_connections 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.072+0000 7f5f87908700 1 -- 172.21.15.31:0/1132887272 wait complete. 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.073+0000 7f5f87908700 1 Processor -- start 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.073+0000 7f5f87908700 1 -- start start 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f87908700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 0x7f5f80156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f87908700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f8013dba0 0x7f5f800c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f87908700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f80158370 0x7f5f80150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f87908700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f801eccc0 con 0x7f5f8006ecb0 2024-03-30T01:42:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f87908700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f801ecfa0 con 0x7f5f8013dba0 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f87908700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f801ed280 con 0x7f5f80158370 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f856a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 0x7f5f80156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f856a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 0x7f5f80156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35312/0 (socket says 172.21.15.31:35312) 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f856a4700 1 -- 172.21.15.31:0/2893397190 learned_addr learned my addr 172.21.15.31:0/2893397190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f84ea3700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f8013dba0 0x7f5f800c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f85ea5700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f80158370 0x7f5f80150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f856a4700 1 -- 172.21.15.31:0/2893397190 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f80158370 msgr2=0x7f5f80150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f856a4700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f80158370 0x7f5f80150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f856a4700 1 -- 172.21.15.31:0/2893397190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f8013dba0 msgr2=0x7f5f800c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f856a4700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f8013dba0 0x7f5f800c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.074+0000 7f5f856a4700 1 -- 172.21.15.31:0/2893397190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c009550 con 0x7f5f8006ecb0 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.075+0000 7f5f84ea3700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f8013dba0 0x7f5f800c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.075+0000 7f5f856a4700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 0x7f5f80156f10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000e810 tx=0x7f5f7000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.075+0000 7f5f75ffb700 1 -- 172.21.15.31:0/2893397190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f70013610 con 0x7f5f8006ecb0 2024-03-30T01:42:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.075+0000 7f5f75ffb700 1 -- 172.21.15.31:0/2893397190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f70013770 con 0x7f5f8006ecb0 2024-03-30T01:42:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.075+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f801ed560 con 0x7f5f8006ecb0 2024-03-30T01:42:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.075+0000 7f5f75ffb700 1 -- 172.21.15.31:0/2893397190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f70013b20 con 0x7f5f8006ecb0 2024-03-30T01:42:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.075+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f80157450 con 0x7f5f8006ecb0 2024-03-30T01:42:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.076+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8004ea80 con 0x7f5f8006ecb0 2024-03-30T01:42:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.077+0000 7f5f75ffb700 1 -- 172.21.15.31:0/2893397190 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5f70013c80 con 0x7f5f8006ecb0 2024-03-30T01:42:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.077+0000 7f5f75ffb700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5f6406ac90 0x7f5f6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.077+0000 7f5f75ffb700 1 -- 172.21.15.31:0/2893397190 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6620+0+0 (secure 0 0 0) 0x7f5f700893f0 con 0x7f5f8006ecb0 2024-03-30T01:42:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.077+0000 7f5f84ea3700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5f6406ac90 0x7f5f6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:54.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.078+0000 7f5f84ea3700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5f6406ac90 0x7f5f6406d150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c000c00 tx=0x7f5f7c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.082+0000 7f5f75ffb700 1 -- 172.21.15.31:0/2893397190 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5f70054250 con 0x7f5f8006ecb0 2024-03-30T01:42:54.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.302+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5f800008d0 con 0x7f5f8006ecb0 2024-03-30T01:42:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.906+0000 7f5f75ffb700 1 -- 172.21.15.31:0/2893397190 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5f70057870 con 0x7f5f8006ecb0 2024-03-30T01:42:54.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.913+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5f6406ac90 msgr2=0x7f5f6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:54.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.913+0000 7f5f87908700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5f6406ac90 0x7f5f6406d150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c000c00 tx=0x7f5f7c008030 comp rx=0 tx=0).stop 2024-03-30T01:42:54.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 msgr2=0x7f5f80156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:54.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 0x7f5f80156f10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000e810 tx=0x7f5f7000ebd0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 shutdown_connections 2024-03-30T01:42:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f8006ecb0 0x7f5f80156f10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5f6406ac90 0x7f5f6406d150 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f80158370 0x7f5f80150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 --2- 172.21.15.31:0/2893397190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f8013dba0 0x7f5f800c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 >> 172.21.15.31:0/2893397190 conn(0x7f5f800c39f0 msgr2=0x7f5f8013c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 shutdown_connections 2024-03-30T01:42:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:54.914+0000 7f5f87908700 1 -- 172.21.15.31:0/2893397190 wait complete. 2024-03-30T01:42:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-03-30T01:42:54.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.071+0000 7ff029a3a700 1 Processor -- start 2024-03-30T01:42:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.071+0000 7ff029a3a700 1 -- start start 2024-03-30T01:42:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff029a3a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0240c7270 0x7ff0240c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff029a3a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff0241521d0 0x7ff024157da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff029a3a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02406bf10 con 0x7ff0240c6330 2024-03-30T01:42:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff029a3a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0240c5990 con 0x7ff0240c7270 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff029a3a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02406b300 con 0x7ff0241521d0 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff022ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0240c7270 0x7ff0240c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff0227fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff0240c6330 0x7ff024151ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:37006/0 (socket says 172.21.15.31:37006) 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff022ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0240c7270 0x7ff0240c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35328/0 (socket says 172.21.15.31:35328) 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff0227fc700 1 -- 172.21.15.31:0/2473613189 learned_addr learned my addr 172.21.15.31:0/2473613189 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff0237fe700 1 --2- 172.21.15.31:0/2473613189 >> v2:172.21.15.59:3300/0 conn(0x7ff0241521d0 0x7ff024157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff022ffd700 1 -- 172.21.15.31:0/2473613189 >> v2:172.21.15.59:3300/0 conn(0x7ff0241521d0 msgr2=0x7ff024157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff022ffd700 1 --2- 172.21.15.31:0/2473613189 >> v2:172.21.15.59:3300/0 conn(0x7ff0241521d0 0x7ff024157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff022ffd700 1 -- 172.21.15.31:0/2473613189 >> v1:172.21.15.31:6789/0 conn(0x7ff0240c6330 legacy=0x7ff024151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff022ffd700 1 -- 172.21.15.31:0/2473613189 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02406b5e0 con 0x7ff0240c7270 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.072+0000 7ff021ffb700 1 -- 172.21.15.31:0/2473613189 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491021470 0 0) 0x7ff02406bf10 con 0x7ff0240c6330 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff022ffd700 1 --2- 172.21.15.31:0/2473613189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c7270 0x7ff0240c5df0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff0240c9540 tx=0x7ff014009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fffccec05d05f78 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff021ffb700 1 -- 172.21.15.31:0/2473613189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014011830 con 0x7ff0240c7270 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff021ffb700 1 -- 172.21.15.31:0/2473613189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff014011990 con 0x7ff0240c7270 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff021ffb700 1 -- 172.21.15.31:0/2473613189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014011d20 con 0x7ff0240c7270 2024-03-30T01:42:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff029a3a700 1 -- 172.21.15.31:0/2473613189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c7270 msgr2=0x7ff0240c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff029a3a700 1 --2- 172.21.15.31:0/2473613189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c7270 0x7ff0240c5df0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff0240c9540 tx=0x7ff014009b20 comp rx=0 tx=0).stop 2024-03-30T01:42:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff029a3a700 1 -- 172.21.15.31:0/2473613189 shutdown_connections 2024-03-30T01:42:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff029a3a700 1 --2- 172.21.15.31:0/2473613189 >> v2:172.21.15.59:3300/0 conn(0x7ff0241521d0 0x7ff024157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff029a3a700 1 --2- 172.21.15.31:0/2473613189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c7270 0x7ff0240c5df0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff029a3a700 1 -- 172.21.15.31:0/2473613189 >> 172.21.15.31:0/2473613189 conn(0x7ff0240abf60 msgr2=0x7ff0240ac370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.073+0000 7ff029a3a700 1 -- 172.21.15.31:0/2473613189 shutdown_connections 2024-03-30T01:42:55.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.074+0000 7ff029a3a700 1 -- 172.21.15.31:0/2473613189 wait complete. 2024-03-30T01:42:55.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.075+0000 7ff029a3a700 1 Processor -- start 2024-03-30T01:42:55.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.075+0000 7ff029a3a700 1 -- start start 2024-03-30T01:42:55.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff029a3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c6330 0x7ff0240c0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff029a3a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0240c7270 0x7ff0240c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff029a3a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0241521d0 0x7ff0240c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff029a3a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0240c28f0 con 0x7ff0240c6330 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff029a3a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0241e9670 con 0x7ff0240c7270 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff029a3a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0240c4020 con 0x7ff0241521d0 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff022ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c6330 0x7ff0240c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff022ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c6330 0x7ff0240c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35338/0 (socket says 172.21.15.31:35338) 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff022ffd700 1 -- 172.21.15.31:0/2371198620 learned_addr learned my addr 172.21.15.31:0/2371198620 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff0237fe700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0241521d0 0x7ff0240c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff0227fc700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0240c7270 0x7ff0240c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff022ffd700 1 -- 172.21.15.31:0/2371198620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0241521d0 msgr2=0x7ff0240c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff022ffd700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0241521d0 0x7ff0240c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff022ffd700 1 -- 172.21.15.31:0/2371198620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0240c7270 msgr2=0x7ff0240c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff022ffd700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0240c7270 0x7ff0240c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff022ffd700 1 -- 172.21.15.31:0/2371198620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0140094a0 con 0x7ff0240c6330 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff022ffd700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c6330 0x7ff0240c0830 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff014009b60 tx=0x7ff014004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.076+0000 7ff01bfff700 1 -- 172.21.15.31:0/2371198620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0140036a0 con 0x7ff0240c6330 2024-03-30T01:42:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.077+0000 7ff01bfff700 1 -- 172.21.15.31:0/2371198620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01401f070 con 0x7ff0240c6330 2024-03-30T01:42:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.077+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0241ea180 con 0x7ff0240c6330 2024-03-30T01:42:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.077+0000 7ff01bfff700 1 -- 172.21.15.31:0/2371198620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014003a60 con 0x7ff0240c6330 2024-03-30T01:42:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.077+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0241ea660 con 0x7ff0240c6330 2024-03-30T01:42:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.078+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02404ea80 con 0x7ff0240c6330 2024-03-30T01:42:55.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.078+0000 7ff01bfff700 1 -- 172.21.15.31:0/2371198620 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff014003bc0 con 0x7ff0240c6330 2024-03-30T01:42:55.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.079+0000 7ff01bfff700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00406ac90 0x7ff00406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.079+0000 7ff01bfff700 1 -- 172.21.15.31:0/2371198620 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7ff014021070 con 0x7ff0240c6330 2024-03-30T01:42:55.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.079+0000 7ff0227fc700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00406ac90 0x7ff00406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.079+0000 7ff0227fc700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00406ac90 0x7ff00406d150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff0240cb010 tx=0x7ff00c007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:55.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.084+0000 7ff01bfff700 1 -- 172.21.15.31:0/2371198620 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff014055f30 con 0x7ff0240c6330 2024-03-30T01:42:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.299+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff02414c730 con 0x7ff0240c6330 2024-03-30T01:42:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.300+0000 7ff01bfff700 1 -- 172.21.15.31:0/2371198620 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16431 (secure 0 0 0) 0x7ff014015020 con 0x7ff0240c6330 2024-03-30T01:42:55.308 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:55.309 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:54.899321+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6837/1616247357":"2024-03-31T01:42:51.698949+0000","172.21.15.31:6835/1616247357":"2024-03-31T01:42:51.698949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00406ac90 msgr2=0x7ff00406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00406ac90 0x7ff00406d150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff0240cb010 tx=0x7ff00c007be0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c6330 msgr2=0x7ff0240c0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c6330 0x7ff0240c0830 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff014009b60 tx=0x7ff014004750 comp rx=0 tx=0).stop 2024-03-30T01:42:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 shutdown_connections 2024-03-30T01:42:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0240c6330 0x7ff0240c0830 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff00406ac90 0x7ff00406d150 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0241521d0 0x7ff0240c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 --2- 172.21.15.31:0/2371198620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0240c7270 0x7ff0240c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.302+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 >> 172.21.15.31:0/2371198620 conn(0x7ff0240abf60 msgr2=0x7ff02414fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.303+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 shutdown_connections 2024-03-30T01:42:55.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.303+0000 7ff029a3a700 1 -- 172.21.15.31:0/2371198620 wait complete. 2024-03-30T01:42:55.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-30T01:42:21.874769+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-30T01:42:23.236623+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:42:55.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:42:55.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.455+0000 7f75f5e97700 1 Processor -- start 2024-03-30T01:42:55.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.455+0000 7f75f5e97700 1 -- start start 2024-03-30T01:42:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75f5e97700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75f0151b60 0x7f75f0070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75f5e97700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f00c5d40 0x7f75f00c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75f5e97700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f0154990 con 0x7f75f0151f40 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75f5e97700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f0155610 con 0x7f75f0151b60 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75f5e97700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01547a0 con 0x7f75f00c5d40 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75eeffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f00c5d40 0x7f75f00c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75ef7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75f0151b60 0x7f75f0070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75effff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f75f0151f40 0x7f75f00c6610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:50836/0 (socket says 172.21.15.31:50836) 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75effff700 1 -- 172.21.15.31:0/1888011997 learned_addr learned my addr 172.21.15.31:0/1888011997 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75eeffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f00c5d40 0x7f75f00c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:49004/0 (socket says 172.21.15.31:49004) 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.456+0000 7f75ef7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75f0151b60 0x7f75f0070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:46762/0 (socket says 172.21.15.31:46762) 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ee7fc700 1 -- 172.21.15.31:0/1888011997 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198913395 0 0) 0x7f75f0154990 con 0x7f75f0151f40 2024-03-30T01:42:55.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ee7fc700 1 -- 172.21.15.31:0/1888011997 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d80035b0 con 0x7f75f0151f40 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ef7fe700 1 -- 172.21.15.31:0/1888011997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f00c5d40 msgr2=0x7f75f00c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ef7fe700 1 --2- 172.21.15.31:0/1888011997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f00c5d40 0x7f75f00c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ef7fe700 1 -- 172.21.15.31:0/1888011997 >> v1:172.21.15.59:6789/0 conn(0x7f75f0151f40 legacy=0x7f75f00c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ef7fe700 1 -- 172.21.15.31:0/1888011997 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f0154c70 con 0x7f75f0151b60 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75eeffd700 1 --2- 172.21.15.31:0/1888011997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f00c5d40 0x7f75f00c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ef7fe700 1 --2- 172.21.15.31:0/1888011997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f0151b60 0x7f75f0070680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009870 tx=0x7f75e0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be9da71fc623bb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ee7fc700 1 -- 172.21.15.31:0/1888011997 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0011830 con 0x7f75f0151b60 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ee7fc700 1 -- 172.21.15.31:0/1888011997 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e0011990 con 0x7f75f0151b60 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.457+0000 7f75ee7fc700 1 -- 172.21.15.31:0/1888011997 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0011d20 con 0x7f75f0151b60 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.458+0000 7f75f5e97700 1 -- 172.21.15.31:0/1888011997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f0151b60 msgr2=0x7f75f0070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.458+0000 7f75f5e97700 1 --2- 172.21.15.31:0/1888011997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f0151b60 0x7f75f0070680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009870 tx=0x7f75e0009b80 comp rx=0 tx=0).stop 2024-03-30T01:42:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.458+0000 7f75f5e97700 1 -- 172.21.15.31:0/1888011997 shutdown_connections 2024-03-30T01:42:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.458+0000 7f75f5e97700 1 --2- 172.21.15.31:0/1888011997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f0151b60 0x7f75f0070680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.458+0000 7f75f5e97700 1 --2- 172.21.15.31:0/1888011997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f00c5d40 0x7f75f00c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.458+0000 7f75f5e97700 1 -- 172.21.15.31:0/1888011997 >> 172.21.15.31:0/1888011997 conn(0x7f75f00c7d60 msgr2=0x7f75f00c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.458+0000 7f75f5e97700 1 -- 172.21.15.31:0/1888011997 shutdown_connections 2024-03-30T01:42:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.458+0000 7f75f5e97700 1 -- 172.21.15.31:0/1888011997 wait complete. 2024-03-30T01:42:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.459+0000 7f75f5e97700 1 Processor -- start 2024-03-30T01:42:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.459+0000 7f75f5e97700 1 -- start start 2024-03-30T01:42:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75f5e97700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f00c5d40 0x7f75f01ed1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75f5e97700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f0151b60 0x7f75f00a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75f5e97700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f0151f40 0x7f75f00a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75f5e97700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01569a0 con 0x7f75f00c5d40 2024-03-30T01:42:55.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75f5e97700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01edeb0 con 0x7f75f0151b60 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75f5e97700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f00a7a40 con 0x7f75f0151f40 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f00c5d40 0x7f75f01ed1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f00c5d40 0x7f75f01ed1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51342/0 (socket says 172.21.15.31:51342) 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75ef7fe700 1 -- 172.21.15.31:0/3173762257 learned_addr learned my addr 172.21.15.31:0/3173762257 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75eeffd700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f0151b60 0x7f75f00a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.460+0000 7f75effff700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f0151f40 0x7f75f00a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75ef7fe700 1 -- 172.21.15.31:0/3173762257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f0151f40 msgr2=0x7f75f00a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75ef7fe700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f0151f40 0x7f75f00a9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75ef7fe700 1 -- 172.21.15.31:0/3173762257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f0151b60 msgr2=0x7f75f00a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75ef7fe700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f0151b60 0x7f75f00a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75ef7fe700 1 -- 172.21.15.31:0/3173762257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e40094a0 con 0x7f75f00c5d40 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75ef7fe700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f00c5d40 0x7f75f01ed1e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f75e00097c0 tx=0x7f75e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75d7fff700 1 -- 172.21.15.31:0/3173762257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0004590 con 0x7f75f00c5d40 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75d7fff700 1 -- 172.21.15.31:0/3173762257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e001f070 con 0x7f75f00c5d40 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75d7fff700 1 -- 172.21.15.31:0/3173762257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e00038e0 con 0x7f75f00c5d40 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e00094a0 con 0x7f75f00c5d40 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.461+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f00a7c90 con 0x7f75f00c5d40 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.462+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f00747d0 con 0x7f75f00c5d40 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.462+0000 7f75d7fff700 1 -- 172.21.15.31:0/3173762257 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f75e0003a40 con 0x7f75f00c5d40 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.463+0000 7f75d7fff700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f75dc06acc0 0x7f75dc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.463+0000 7f75d7fff700 1 -- 172.21.15.31:0/3173762257 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f75e0021070 con 0x7f75f00c5d40 2024-03-30T01:42:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.463+0000 7f75eeffd700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f75dc06acc0 0x7f75dc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.464+0000 7f75eeffd700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f75dc06acc0 0x7f75dc06d180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f75f00a6e60 tx=0x7f75e4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:55.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.468+0000 7f75d7fff700 1 -- 172.21.15.31:0/3173762257 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f75e0055c90 con 0x7f75f00c5d40 2024-03-30T01:42:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.682+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75f01edc90 con 0x7f75f00c5d40 2024-03-30T01:42:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.682+0000 7f75d7fff700 1 -- 172.21.15.31:0/3173762257 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75e00150a0 con 0x7f75f00c5d40 2024-03-30T01:42:55.689 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f75dc06acc0 msgr2=0x7f75dc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f75dc06acc0 0x7f75dc06d180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f75f00a6e60 tx=0x7f75e4009210 comp rx=0 tx=0).stop 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f00c5d40 msgr2=0x7f75f01ed1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f00c5d40 0x7f75f01ed1e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f75e00097c0 tx=0x7f75e0004750 comp rx=0 tx=0).stop 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 shutdown_connections 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75f00c5d40 0x7f75f01ed1e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f75dc06acc0 0x7f75dc06d180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75f0151f40 0x7f75f00a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 --2- 172.21.15.31:0/3173762257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75f0151b60 0x7f75f00a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 >> 172.21.15.31:0/3173762257 conn(0x7f75f00c7d60 msgr2=0x7f75f006bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:55.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 shutdown_connections 2024-03-30T01:42:55.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.684+0000 7f75f5e97700 1 -- 172.21.15.31:0/3173762257 wait complete. 2024-03-30T01:42:55.705 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-30T01:42:55.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.839+0000 7f1701118700 1 Processor -- start 2024-03-30T01:42:55.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.839+0000 7f1701118700 1 -- start start 2024-03-30T01:42:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f1701118700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16fc151c90 0x7f16fc06a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f1701118700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f16fc0c1250 0x7f16fc154680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f1701118700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 0x7f16fc154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f1701118700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc0acaa0 con 0x7f16fc151c90 2024-03-30T01:42:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f1701118700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc0c3790 con 0x7f16fc0c1250 2024-03-30T01:42:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f1701118700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc152070 con 0x7f16fc06d340 2024-03-30T01:42:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16fc151c90 0x7f16fc06a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fad9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16fc151c90 0x7f16fc06a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51346/0 (socket says 172.21.15.31:51346) 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fad9d700 1 -- 172.21.15.31:0/3558475598 learned_addr learned my addr 172.21.15.31:0/3558475598 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fa59c700 1 --2- 172.21.15.31:0/3558475598 >> v2:172.21.15.59:3300/0 conn(0x7f16fc0c1250 0x7f16fc154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fb59e700 1 --2- 172.21.15.31:0/3558475598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 0x7f16fc154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fad9d700 1 -- 172.21.15.31:0/3558475598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 msgr2=0x7f16fc154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fad9d700 1 --2- 172.21.15.31:0/3558475598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 0x7f16fc154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fad9d700 1 -- 172.21.15.31:0/3558475598 >> v2:172.21.15.59:3300/0 conn(0x7f16fc0c1250 msgr2=0x7f16fc154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fad9d700 1 --2- 172.21.15.31:0/3558475598 >> v2:172.21.15.59:3300/0 conn(0x7f16fc0c1250 0x7f16fc154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fad9d700 1 -- 172.21.15.31:0/3558475598 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc0acef0 con 0x7f16fc151c90 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fa59c700 1 --2- 172.21.15.31:0/3558475598 >> v2:172.21.15.59:3300/0 conn(0x7f16fc0c1250 0x7f16fc154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16fad9d700 1 --2- 172.21.15.31:0/3558475598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 0x7f16fc06a700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0c0ba0 tx=0x7f16ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a0ba45d5c9180d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16f9d9b700 1 -- 172.21.15.31:0/3558475598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00b870 con 0x7f16fc151c90 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16f9d9b700 1 -- 172.21.15.31:0/3558475598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec00b9d0 con 0x7f16fc151c90 2024-03-30T01:42:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.840+0000 7f16f9d9b700 1 -- 172.21.15.31:0/3558475598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00bd60 con 0x7f16fc151c90 2024-03-30T01:42:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.841+0000 7f1701118700 1 -- 172.21.15.31:0/3558475598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 msgr2=0x7f16fc06a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.841+0000 7f1701118700 1 --2- 172.21.15.31:0/3558475598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 0x7f16fc06a700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0c0ba0 tx=0x7f16ec009bd0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.841+0000 7f1701118700 1 -- 172.21.15.31:0/3558475598 shutdown_connections 2024-03-30T01:42:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.841+0000 7f1701118700 1 --2- 172.21.15.31:0/3558475598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 0x7f16fc154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.841+0000 7f1701118700 1 --2- 172.21.15.31:0/3558475598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 0x7f16fc06a700 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.841+0000 7f1701118700 1 --2- 172.21.15.31:0/3558475598 >> v2:172.21.15.59:3300/0 conn(0x7f16fc0c1250 0x7f16fc154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.841+0000 7f1701118700 1 -- 172.21.15.31:0/3558475598 >> 172.21.15.31:0/3558475598 conn(0x7f16fc0c3910 msgr2=0x7f16fc0c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.841+0000 7f1701118700 1 -- 172.21.15.31:0/3558475598 shutdown_connections 2024-03-30T01:42:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.841+0000 7f1701118700 1 -- 172.21.15.31:0/3558475598 wait complete. 2024-03-30T01:42:55.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.843+0000 7f1701118700 1 Processor -- start 2024-03-30T01:42:55.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.843+0000 7f1701118700 1 -- start start 2024-03-30T01:42:55.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f1701118700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 0x7f16fc0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f1701118700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16fc0c1250 0x7f16fc14b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f1701118700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 0x7f16fc14b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f1701118700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc156d40 con 0x7f16fc151c90 2024-03-30T01:42:55.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f1701118700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc1ea410 con 0x7f16fc0c1250 2024-03-30T01:42:55.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f1701118700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc1ea940 con 0x7f16fc06d340 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fb59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 0x7f16fc14b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fb59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 0x7f16fc14b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51358/0 (socket says 172.21.15.31:51358) 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fb59e700 1 -- 172.21.15.31:0/1037530032 learned_addr learned my addr 172.21.15.31:0/1037530032 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fad9d700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 0x7f16fc0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fa59c700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16fc0c1250 0x7f16fc14b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fb59e700 1 -- 172.21.15.31:0/1037530032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 msgr2=0x7f16fc0c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fb59e700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 0x7f16fc0c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fb59e700 1 -- 172.21.15.31:0/1037530032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16fc0c1250 msgr2=0x7f16fc14b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fb59e700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16fc0c1250 0x7f16fc14b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fb59e700 1 -- 172.21.15.31:0/1037530032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec009550 con 0x7f16fc151c90 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fad9d700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 0x7f16fc0c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16fb59e700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 0x7f16fc14b780 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f16f400ccf0 tx=0x7f16f400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.844+0000 7f16e37fe700 1 -- 172.21.15.31:0/1037530032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4003a20 con 0x7f16fc151c90 2024-03-30T01:42:55.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.845+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16fc14bcc0 con 0x7f16fc151c90 2024-03-30T01:42:55.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.845+0000 7f16e37fe700 1 -- 172.21.15.31:0/1037530032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f4003b80 con 0x7f16fc151c90 2024-03-30T01:42:55.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.845+0000 7f16e37fe700 1 -- 172.21.15.31:0/1037530032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4004020 con 0x7f16fc151c90 2024-03-30T01:42:55.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.845+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16fc1579a0 con 0x7f16fc151c90 2024-03-30T01:42:55.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.845+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16fc066f90 con 0x7f16fc151c90 2024-03-30T01:42:55.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.846+0000 7f16e37fe700 1 -- 172.21.15.31:0/1037530032 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f16f4004180 con 0x7f16fc151c90 2024-03-30T01:42:55.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.847+0000 7f16e37fe700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f16e806ac90 0x7f16e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:55.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.847+0000 7f16e37fe700 1 -- 172.21.15.31:0/1037530032 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f16f4012070 con 0x7f16fc151c90 2024-03-30T01:42:55.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.847+0000 7f16fad9d700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f16e806ac90 0x7f16e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:55.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.847+0000 7f16fad9d700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f16e806ac90 0x7f16e806d150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f16ec014040 tx=0x7f16ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:55.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:55.851+0000 7f16e37fe700 1 -- 172.21.15.31:0/1037530032 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f16f4054d10 con 0x7f16fc151c90 2024-03-30T01:42:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.069+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16fc066950 con 0x7f16fc151c90 2024-03-30T01:42:56.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.070+0000 7f16e37fe700 1 -- 172.21.15.31:0/1037530032 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v48) v1 ==== 105+0+11 (secure 0 0 0) 0x7f16f4089e60 con 0x7f16fc151c90 2024-03-30T01:42:56.076 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f16e806ac90 msgr2=0x7f16e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f16e806ac90 0x7f16e806d150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f16ec014040 tx=0x7f16ec008030 comp rx=0 tx=0).stop 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 msgr2=0x7f16fc14b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 0x7f16fc14b780 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f16f400ccf0 tx=0x7f16f400a570 comp rx=0 tx=0).stop 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 shutdown_connections 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16fc151c90 0x7f16fc14b780 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f16e806ac90 0x7f16e806d150 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16fc06d340 0x7f16fc0c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 --2- 172.21.15.31:0/1037530032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16fc0c1250 0x7f16fc14b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 >> 172.21.15.31:0/1037530032 conn(0x7f16fc0c3910 msgr2=0x7f16fc0c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 shutdown_connections 2024-03-30T01:42:56.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.072+0000 7f1701118700 1 -- 172.21.15.31:0/1037530032 wait complete. 2024-03-30T01:42:56.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-30T01:42:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.225+0000 7fc9788b3700 1 Processor -- start 2024-03-30T01:42:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.225+0000 7fc9788b3700 1 -- start start 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.226+0000 7fc9788b3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc97414ff20 0x7fc9740c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.226+0000 7fc9788b3700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9740c4620 con 0x7fc97414fb40 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.226+0000 7fc9788b3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc974152ad0 con 0x7fc9740a7600 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.226+0000 7fc9788b3700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc974152570 con 0x7fc97414ff20 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.226+0000 7fc972d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc97414ff20 0x7fc9740c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.227+0000 7fc97259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc97414fb40 0x7fc9740c2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33310/0 (socket says 172.21.15.31:33310) 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.227+0000 7fc97259c700 1 -- 172.21.15.31:0/113373109 learned_addr learned my addr 172.21.15.31:0/113373109 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.227+0000 7fc972d9d700 1 -- 172.21.15.31:0/113373109 >> v1:172.21.15.59:6789/0 conn(0x7fc9740a7600 legacy=0x7fc974152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.227+0000 7fc972d9d700 1 -- 172.21.15.31:0/113373109 >> v1:172.21.15.31:6789/0 conn(0x7fc97414fb40 legacy=0x7fc9740c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.227+0000 7fc972d9d700 1 -- 172.21.15.31:0/113373109 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740c4060 con 0x7fc97414ff20 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.227+0000 7fc972d9d700 1 --2- 172.21.15.31:0/113373109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414ff20 0x7fc9740c47f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009d90 tx=0x7fc96c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5740e247ec8da0e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:56.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.227+0000 7fc97159a700 1 -- 172.21.15.31:0/113373109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00e070 con 0x7fc97414ff20 2024-03-30T01:42:56.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.227+0000 7fc97159a700 1 -- 172.21.15.31:0/113373109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c00bbd0 con 0x7fc97414ff20 2024-03-30T01:42:56.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.227+0000 7fc97159a700 1 -- 172.21.15.31:0/113373109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c012430 con 0x7fc97414ff20 2024-03-30T01:42:56.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.228+0000 7fc9788b3700 1 -- 172.21.15.31:0/113373109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414ff20 msgr2=0x7fc9740c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.228+0000 7fc9788b3700 1 --2- 172.21.15.31:0/113373109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414ff20 0x7fc9740c47f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009d90 tx=0x7fc96c00b330 comp rx=0 tx=0).stop 2024-03-30T01:42:56.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.228+0000 7fc9788b3700 1 -- 172.21.15.31:0/113373109 shutdown_connections 2024-03-30T01:42:56.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.228+0000 7fc9788b3700 1 --2- 172.21.15.31:0/113373109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414ff20 0x7fc9740c47f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.228+0000 7fc9788b3700 1 -- 172.21.15.31:0/113373109 >> 172.21.15.31:0/113373109 conn(0x7fc9740c7d60 msgr2=0x7fc9740c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:56.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.228+0000 7fc9788b3700 1 -- 172.21.15.31:0/113373109 shutdown_connections 2024-03-30T01:42:56.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.228+0000 7fc9788b3700 1 -- 172.21.15.31:0/113373109 wait complete. 2024-03-30T01:42:56.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.230+0000 7fc9788b3700 1 Processor -- start 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.230+0000 7fc9788b3700 1 -- start start 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc9788b3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9740a7600 0x7fc9740c1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc9788b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414fb40 0x7fc9740a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc9788b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc97414ff20 0x7fc9740a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc9788b3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740bff80 con 0x7fc97414fb40 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc9788b3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97414ba00 con 0x7fc97414ff20 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc9788b3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740a9510 con 0x7fc9740a7600 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc971d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414fb40 0x7fc9740a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc971d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414fb40 0x7fc9740a8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51374/0 (socket says 172.21.15.31:51374) 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc971d9b700 1 -- 172.21.15.31:0/2095349431 learned_addr learned my addr 172.21.15.31:0/2095349431 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc972d9d700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc97414ff20 0x7fc9740a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc97259c700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9740a7600 0x7fc9740c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc971d9b700 1 -- 172.21.15.31:0/2095349431 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9740a7600 msgr2=0x7fc9740c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc971d9b700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9740a7600 0x7fc9740c1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc971d9b700 1 -- 172.21.15.31:0/2095349431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc97414ff20 msgr2=0x7fc9740a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc971d9b700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc97414ff20 0x7fc9740a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc971d9b700 1 -- 172.21.15.31:0/2095349431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c009a70 con 0x7fc97414fb40 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc97259c700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9740a7600 0x7fc9740c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.231+0000 7fc971d9b700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414fb40 0x7fc9740a8a90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc96400c700 tx=0x7fc96400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.232+0000 7fc962ffd700 1 -- 172.21.15.31:0/2095349431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9640052d0 con 0x7fc97414fb40 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.232+0000 7fc962ffd700 1 -- 172.21.15.31:0/2095349431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964005430 con 0x7fc97414fb40 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.232+0000 7fc962ffd700 1 -- 172.21.15.31:0/2095349431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964003880 con 0x7fc97414fb40 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.232+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9740bd110 con 0x7fc97414fb40 2024-03-30T01:42:56.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.232+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9741e8760 con 0x7fc97414fb40 2024-03-30T01:42:56.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.233+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9740747d0 con 0x7fc97414fb40 2024-03-30T01:42:56.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.233+0000 7fc962ffd700 1 -- 172.21.15.31:0/2095349431 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc9640039e0 con 0x7fc97414fb40 2024-03-30T01:42:56.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.234+0000 7fc962ffd700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc95406ac90 0x7fc95406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.234+0000 7fc962ffd700 1 -- 172.21.15.31:0/2095349431 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fc964088820 con 0x7fc97414fb40 2024-03-30T01:42:56.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.234+0000 7fc97259c700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc95406ac90 0x7fc95406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.234+0000 7fc97259c700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc95406ac90 0x7fc95406d150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc968007fd0 tx=0x7fc968009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:56.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.238+0000 7fc962ffd700 1 -- 172.21.15.31:0/2095349431 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc9640538e0 con 0x7fc97414fb40 2024-03-30T01:42:56.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.454+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fc9740bd2a0 con 0x7fc97414fb40 2024-03-30T01:42:56.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.454+0000 7fc962ffd700 1 -- 172.21.15.31:0/2095349431 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v48) v1 ==== 101+0+11 (secure 0 0 0) 0x7fc964056f00 con 0x7fc97414fb40 2024-03-30T01:42:56.461 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:42:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc95406ac90 msgr2=0x7fc95406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc95406ac90 0x7fc95406d150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc968007fd0 tx=0x7fc968009040 comp rx=0 tx=0).stop 2024-03-30T01:42:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414fb40 msgr2=0x7fc9740a8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414fb40 0x7fc9740a8a90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc96400c700 tx=0x7fc96400ca10 comp rx=0 tx=0).stop 2024-03-30T01:42:56.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 shutdown_connections 2024-03-30T01:42:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc95406ac90 0x7fc95406d150 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc97414ff20 0x7fc9740a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9740a7600 0x7fc9740c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 --2- 172.21.15.31:0/2095349431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc97414fb40 0x7fc9740a8a90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 >> 172.21.15.31:0/2095349431 conn(0x7fc9740c7d60 msgr2=0x7fc9741503e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.456+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 shutdown_connections 2024-03-30T01:42:56.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.457+0000 7fc9788b3700 1 -- 172.21.15.31:0/2095349431 wait complete. 2024-03-30T01:42:56.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:56.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.607+0000 7f98e41ea700 1 Processor -- start 2024-03-30T01:42:56.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.607+0000 7f98e41ea700 1 -- start start 2024-03-30T01:42:56.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e41ea700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f98dc0722b0 0x7f98dc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e41ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc1479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e41ea700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc0a88c0 con 0x7f98dc154d20 2024-03-30T01:42:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e41ea700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc0c3110 con 0x7f98dc0722b0 2024-03-30T01:42:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e41ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc155810 con 0x7f98dc06db20 2024-03-30T01:42:56.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e2787700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc1479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e1785700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f98dc0722b0 0x7f98dc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e1f86700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f98dc154d20 0x7f98dc155100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:50848/0 (socket says 172.21.15.31:50848) 2024-03-30T01:42:56.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e1f86700 1 -- 172.21.15.31:0/2948856677 learned_addr learned my addr 172.21.15.31:0/2948856677 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:56.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e1785700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f98dc0722b0 0x7f98dc06d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:46808/0 (socket says 172.21.15.31:46808) 2024-03-30T01:42:56.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.608+0000 7f98e2787700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc1479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:49034/0 (socket says 172.21.15.31:49034) 2024-03-30T01:42:56.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.609+0000 7f98e1785700 1 -- 172.21.15.31:0/2948856677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 msgr2=0x7f98dc1479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:56.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.609+0000 7f98e1785700 1 --2- 172.21.15.31:0/2948856677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc1479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.609+0000 7f98e1785700 1 -- 172.21.15.31:0/2948856677 >> v1:172.21.15.59:6789/0 conn(0x7f98dc154d20 legacy=0x7f98dc155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.609+0000 7f98e1785700 1 -- 172.21.15.31:0/2948856677 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc0a85e0 con 0x7f98dc0722b0 2024-03-30T01:42:56.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.609+0000 7f98e2787700 1 --2- 172.21.15.31:0/2948856677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc1479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:42:56.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.609+0000 7f98e1785700 1 --2- 172.21.15.31:0/2948856677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98dc0722b0 0x7f98dc06d5e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009840 tx=0x7f98cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d52ca393769f2e4a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:56.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.609+0000 7f98e0f84700 1 -- 172.21.15.31:0/2948856677 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc011830 con 0x7f98dc0722b0 2024-03-30T01:42:56.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.609+0000 7f98e0f84700 1 -- 172.21.15.31:0/2948856677 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc011990 con 0x7f98dc0722b0 2024-03-30T01:42:56.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.609+0000 7f98e0f84700 1 -- 172.21.15.31:0/2948856677 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc011d20 con 0x7f98dc0722b0 2024-03-30T01:42:56.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.610+0000 7f98e41ea700 1 -- 172.21.15.31:0/2948856677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98dc0722b0 msgr2=0x7f98dc06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.610+0000 7f98e41ea700 1 --2- 172.21.15.31:0/2948856677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98dc0722b0 0x7f98dc06d5e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009840 tx=0x7f98cc009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:56.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.610+0000 7f98e41ea700 1 -- 172.21.15.31:0/2948856677 shutdown_connections 2024-03-30T01:42:56.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.610+0000 7f98e41ea700 1 --2- 172.21.15.31:0/2948856677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc1479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.610+0000 7f98e41ea700 1 --2- 172.21.15.31:0/2948856677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98dc0722b0 0x7f98dc06d5e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.610+0000 7f98e41ea700 1 -- 172.21.15.31:0/2948856677 >> 172.21.15.31:0/2948856677 conn(0x7f98dc0c3a60 msgr2=0x7f98dc0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:56.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.610+0000 7f98e41ea700 1 -- 172.21.15.31:0/2948856677 shutdown_connections 2024-03-30T01:42:56.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.610+0000 7f98e41ea700 1 -- 172.21.15.31:0/2948856677 wait complete. 2024-03-30T01:42:56.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.612+0000 7f98e41ea700 1 Processor -- start 2024-03-30T01:42:56.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.612+0000 7f98e41ea700 1 -- start start 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e41ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e41ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98dc0722b0 0x7f98dc135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e41ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98dc154d20 0x7f98dc135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e41ea700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc14b8b0 con 0x7f98dc0722b0 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e41ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0ab850 con 0x7f98dc154d20 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e41ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0ac6b0 con 0x7f98dc06db20 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1785700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98dc0722b0 0x7f98dc135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1785700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98dc0722b0 0x7f98dc135560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51382/0 (socket says 172.21.15.31:51382) 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1785700 1 -- 172.21.15.31:0/3192701424 learned_addr learned my addr 172.21.15.31:0/3192701424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1f86700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e2787700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98dc154d20 0x7f98dc135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1785700 1 -- 172.21.15.31:0/3192701424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 msgr2=0x7f98dc147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1785700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1785700 1 -- 172.21.15.31:0/3192701424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98dc154d20 msgr2=0x7f98dc135aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1785700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98dc154d20 0x7f98dc135aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1785700 1 -- 172.21.15.31:0/3192701424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4009a70 con 0x7f98dc0722b0 2024-03-30T01:42:56.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.613+0000 7f98e1785700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98dc0722b0 0x7f98dc135560 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0090b0 tx=0x7f98cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:56.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.614+0000 7f98d27fc700 1 -- 172.21.15.31:0/3192701424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0045d0 con 0x7f98dc0722b0 2024-03-30T01:42:56.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.614+0000 7f98d27fc700 1 -- 172.21.15.31:0/3192701424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc01f070 con 0x7f98dc0722b0 2024-03-30T01:42:56.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.614+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc0094a0 con 0x7f98dc0722b0 2024-03-30T01:42:56.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.614+0000 7f98d27fc700 1 -- 172.21.15.31:0/3192701424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0038e0 con 0x7f98dc0722b0 2024-03-30T01:42:56.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.614+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc0abd80 con 0x7f98dc0722b0 2024-03-30T01:42:56.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.615+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc06d940 con 0x7f98dc0722b0 2024-03-30T01:42:56.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.615+0000 7f98d27fc700 1 -- 172.21.15.31:0/3192701424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f98cc003a40 con 0x7f98dc0722b0 2024-03-30T01:42:56.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.616+0000 7f98d27fc700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98c006ac90 0x7f98c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:56.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.616+0000 7f98d27fc700 1 -- 172.21.15.31:0/3192701424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f98cc021070 con 0x7f98dc0722b0 2024-03-30T01:42:56.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.616+0000 7f98e1f86700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98c006ac90 0x7f98c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:56.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.616+0000 7f98e1f86700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98c006ac90 0x7f98c006d150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005910 tx=0x7f98d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:56.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.620+0000 7f98d27fc700 1 -- 172.21.15.31:0/3192701424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f98cc055d40 con 0x7f98dc0722b0 2024-03-30T01:42:56.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.883+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98dc04ea80 con 0x7f98dc0722b0 2024-03-30T01:42:56.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.883+0000 7f98d27fc700 1 -- 172.21.15.31:0/3192701424 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f98cc015090 con 0x7f98dc0722b0 2024-03-30T01:42:56.890 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:56.890 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4017049280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":4017049280},{"type":"v1","addr":"172.21.15.31:6837","nonce":4017049280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:24.267501+0000","modified":"2024-03-30T01:42:27.273615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T01:42:56.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98c006ac90 msgr2=0x7f98c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98c006ac90 0x7f98c006d150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005910 tx=0x7f98d8009040 comp rx=0 tx=0).stop 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98dc0722b0 msgr2=0x7f98dc135560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98dc0722b0 0x7f98dc135560 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0090b0 tx=0x7f98cc004750 comp rx=0 tx=0).stop 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 shutdown_connections 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f98c006ac90 0x7f98c006d150 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98dc154d20 0x7f98dc135aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f98dc06db20 0x7f98dc147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 --2- 172.21.15.31:0/3192701424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98dc0722b0 0x7f98dc135560 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 >> 172.21.15.31:0/3192701424 conn(0x7f98dc0c3a60 msgr2=0x7f98dc155100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 shutdown_connections 2024-03-30T01:42:56.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:56.886+0000 7f98e41ea700 1 -- 172.21.15.31:0/3192701424 wait complete. 2024-03-30T01:42:56.894 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-03-30T01:42:56.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:57.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.036+0000 7ffbf0fda700 1 Processor -- start 2024-03-30T01:42:57.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.036+0000 7ffbf0fda700 1 -- start start 2024-03-30T01:42:57.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbf0fda700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0ac520 0x7ffbec0ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbf0fda700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbec0a74e0 0x7ffbec0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbf0fda700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbec153590 con 0x7ffbec0c6000 2024-03-30T01:42:57.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbf0fda700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbec0ab680 con 0x7ffbec0a74e0 2024-03-30T01:42:57.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbf0fda700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec1500c0 con 0x7ffbec0ac520 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbe9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbec0a74e0 0x7ffbec0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbead9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffbec0c6000 0x7ffbec0c6490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33312/0 (socket says 172.21.15.31:33312) 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbe9d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbec0a74e0 0x7ffbec0ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51388/0 (socket says 172.21.15.31:51388) 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbead9d700 1 -- 172.21.15.31:0/3027257566 learned_addr learned my addr 172.21.15.31:0/3027257566 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbea59c700 1 --2- 172.21.15.31:0/3027257566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0ac520 0x7ffbec0ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbe9d9b700 1 -- 172.21.15.31:0/3027257566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0ac520 msgr2=0x7ffbec0ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbe9d9b700 1 --2- 172.21.15.31:0/3027257566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0ac520 0x7ffbec0ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbe9d9b700 1 -- 172.21.15.31:0/3027257566 >> v1:172.21.15.31:6789/0 conn(0x7ffbec0c6000 legacy=0x7ffbec0c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbe9d9b700 1 -- 172.21.15.31:0/3027257566 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec0c7990 con 0x7ffbec0a74e0 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbe959a700 1 -- 172.21.15.31:0/3027257566 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094661559 0 0) 0x7ffbec153590 con 0x7ffbec0c6000 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.037+0000 7ffbe9d9b700 1 --2- 172.21.15.31:0/3027257566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0a74e0 0x7ffbec0ace40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009840 tx=0x7ffbdc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fc5d6eeac3253d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbe959a700 1 -- 172.21.15.31:0/3027257566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc011830 con 0x7ffbec0a74e0 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbe959a700 1 -- 172.21.15.31:0/3027257566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc011990 con 0x7ffbec0a74e0 2024-03-30T01:42:57.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbe959a700 1 -- 172.21.15.31:0/3027257566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc011d20 con 0x7ffbec0a74e0 2024-03-30T01:42:57.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbf0fda700 1 -- 172.21.15.31:0/3027257566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0a74e0 msgr2=0x7ffbec0ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbf0fda700 1 --2- 172.21.15.31:0/3027257566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0a74e0 0x7ffbec0ace40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009840 tx=0x7ffbdc009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:57.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbf0fda700 1 -- 172.21.15.31:0/3027257566 shutdown_connections 2024-03-30T01:42:57.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbf0fda700 1 --2- 172.21.15.31:0/3027257566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0ac520 0x7ffbec0ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbf0fda700 1 --2- 172.21.15.31:0/3027257566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0a74e0 0x7ffbec0ace40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbf0fda700 1 -- 172.21.15.31:0/3027257566 >> 172.21.15.31:0/3027257566 conn(0x7ffbec06ad30 msgr2=0x7ffbec0a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:57.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbf0fda700 1 -- 172.21.15.31:0/3027257566 shutdown_connections 2024-03-30T01:42:57.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.038+0000 7ffbf0fda700 1 -- 172.21.15.31:0/3027257566 wait complete. 2024-03-30T01:42:57.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.040+0000 7ffbf0fda700 1 Processor -- start 2024-03-30T01:42:57.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.040+0000 7ffbf0fda700 1 -- start start 2024-03-30T01:42:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbf0fda700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0a74e0 0x7ffbec151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbf0fda700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbec0ac520 0x7ffbec0c1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbf0fda700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0c6000 0x7ffbec14c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbf0fda700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec154010 con 0x7ffbec0c6000 2024-03-30T01:42:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbf0fda700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec1523f0 con 0x7ffbec0ac520 2024-03-30T01:42:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbf0fda700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec06d1f0 con 0x7ffbec0a74e0 2024-03-30T01:42:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbead9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0c6000 0x7ffbec14c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbead9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0c6000 0x7ffbec14c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51390/0 (socket says 172.21.15.31:51390) 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbead9d700 1 -- 172.21.15.31:0/2605162924 learned_addr learned my addr 172.21.15.31:0/2605162924 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbea59c700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0a74e0 0x7ffbec151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbe9d9b700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbec0ac520 0x7ffbec0c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbead9d700 1 -- 172.21.15.31:0/2605162924 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0a74e0 msgr2=0x7ffbec151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbead9d700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0a74e0 0x7ffbec151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbead9d700 1 -- 172.21.15.31:0/2605162924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbec0ac520 msgr2=0x7ffbec0c1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbead9d700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbec0ac520 0x7ffbec0c1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbead9d700 1 -- 172.21.15.31:0/2605162924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc0094a0 con 0x7ffbec0c6000 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbead9d700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0c6000 0x7ffbec14c560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400cba0 tx=0x7ffbe400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbe2ffd700 1 -- 172.21.15.31:0/2605162924 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe400ab80 con 0x7ffbec0c6000 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbe2ffd700 1 -- 172.21.15.31:0/2605162924 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe400ad00 con 0x7ffbec0c6000 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbe2ffd700 1 -- 172.21.15.31:0/2605162924 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4007310 con 0x7ffbec0c6000 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.041+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbec06d4d0 con 0x7ffbec0c6000 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.042+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbec153c30 con 0x7ffbec0c6000 2024-03-30T01:42:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.042+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbec154150 con 0x7ffbec0c6000 2024-03-30T01:42:57.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.043+0000 7ffbe2ffd700 1 -- 172.21.15.31:0/2605162924 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffbe4007470 con 0x7ffbec0c6000 2024-03-30T01:42:57.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.044+0000 7ffbe2ffd700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffbcc06ac90 0x7ffbcc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.044+0000 7ffbe2ffd700 1 -- 172.21.15.31:0/2605162924 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7ffbe4088fe0 con 0x7ffbec0c6000 2024-03-30T01:42:57.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.044+0000 7ffbea59c700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffbcc06ac90 0x7ffbcc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.044+0000 7ffbea59c700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffbcc06ac90 0x7ffbcc06d150 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009b70 tx=0x7ffbd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:57.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.048+0000 7ffbe2ffd700 1 -- 172.21.15.31:0/2605162924 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ffbec154150 con 0x7ffbec0c6000 2024-03-30T01:42:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.266+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbec154150 con 0x7ffbec0c6000 2024-03-30T01:42:57.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.267+0000 7ffbe2ffd700 1 -- 172.21.15.31:0/2605162924 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16431 (secure 0 0 0) 0x7ffbec154150 con 0x7ffbec0c6000 2024-03-30T01:42:57.274 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:57.274 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:54.899321+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6837/1616247357":"2024-03-31T01:42:51.698949+0000","172.21.15.31:6835/1616247357":"2024-03-31T01:42:51.698949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffbcc06ac90 msgr2=0x7ffbcc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffbcc06ac90 0x7ffbcc06d150 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009b70 tx=0x7ffbd4008040 comp rx=0 tx=0).stop 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0c6000 msgr2=0x7ffbec14c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0c6000 0x7ffbec14c560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400cba0 tx=0x7ffbe400cf60 comp rx=0 tx=0).stop 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 shutdown_connections 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbec0c6000 0x7ffbec14c560 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffbcc06ac90 0x7ffbcc06d150 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbec0a74e0 0x7ffbec151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 --2- 172.21.15.31:0/2605162924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbec0ac520 0x7ffbec0c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 >> 172.21.15.31:0/2605162924 conn(0x7ffbec06ad30 msgr2=0x7ffbec0c35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 shutdown_connections 2024-03-30T01:42:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.270+0000 7ffbf0fda700 1 -- 172.21.15.31:0/2605162924 wait complete. 2024-03-30T01:42:57.290 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-30T01:42:57.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:57.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.421+0000 7f0661bd0700 1 Processor -- start 2024-03-30T01:42:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.421+0000 7f0661bd0700 1 -- start start 2024-03-30T01:42:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f0661bd0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f065c0ac8b0 0x7f065c153190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f0661bd0700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c155b00 con 0x7f065c153c00 2024-03-30T01:42:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f0661bd0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c0abff0 con 0x7f065c0ad0c0 2024-03-30T01:42:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f0661bd0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c1536d0 con 0x7f065c0ac8b0 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f065c153c00 0x7f065c148cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33326/0 (socket says 172.21.15.31:33326) 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065bfff700 1 -- 172.21.15.31:0/1702033207 learned_addr learned my addr 172.21.15.31:0/1702033207 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065b7fe700 1 --2- 172.21.15.31:0/1702033207 >> v2:172.21.15.59:3300/0 conn(0x7f065c0ac8b0 0x7f065c153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065a7fc700 1 -- 172.21.15.31:0/1702033207 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926455513 0 0) 0x7f065c155b00 con 0x7f065c153c00 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065a7fc700 1 -- 172.21.15.31:0/1702033207 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06440035b0 con 0x7f065c153c00 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065a7fc700 1 -- 172.21.15.31:0/1702033207 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342364151 0 0) 0x7f0650001110 con 0x7f065c0ad0c0 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065a7fc700 1 -- 172.21.15.31:0/1702033207 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f065c155b00 con 0x7f065c0ad0c0 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065a7fc700 1 -- 172.21.15.31:0/1702033207 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3021821753 0 0) 0x7f06440035b0 con 0x7f065c153c00 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065b7fe700 1 -- 172.21.15.31:0/1702033207 >> v1:172.21.15.59:6789/0 conn(0x7f065c0ad0c0 legacy=0x7f065c0c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065b7fe700 1 -- 172.21.15.31:0/1702033207 >> v1:172.21.15.31:6789/0 conn(0x7f065c153c00 legacy=0x7f065c148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.422+0000 7f065b7fe700 1 -- 172.21.15.31:0/1702033207 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0c2980 con 0x7f065c0ac8b0 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.423+0000 7f065b7fe700 1 --2- 172.21.15.31:0/1702033207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065c0ac8b0 0x7f065c153190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f064c009840 tx=0x7f064c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65704f87cb2b03f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.423+0000 7f065a7fc700 1 -- 172.21.15.31:0/1702033207 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06540019f0 con 0x7f065c0ac8b0 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.423+0000 7f065a7fc700 1 -- 172.21.15.31:0/1702033207 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c004030 con 0x7f065c0ac8b0 2024-03-30T01:42:57.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.423+0000 7f065a7fc700 1 -- 172.21.15.31:0/1702033207 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0043c0 con 0x7f065c0ac8b0 2024-03-30T01:42:57.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.423+0000 7f0661bd0700 1 -- 172.21.15.31:0/1702033207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065c0ac8b0 msgr2=0x7f065c153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.423+0000 7f0661bd0700 1 --2- 172.21.15.31:0/1702033207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065c0ac8b0 0x7f065c153190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f064c009840 tx=0x7f064c009b50 comp rx=0 tx=0).stop 2024-03-30T01:42:57.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.424+0000 7f0661bd0700 1 -- 172.21.15.31:0/1702033207 shutdown_connections 2024-03-30T01:42:57.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.424+0000 7f0661bd0700 1 --2- 172.21.15.31:0/1702033207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065c0ac8b0 0x7f065c153190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.424+0000 7f0661bd0700 1 -- 172.21.15.31:0/1702033207 >> 172.21.15.31:0/1702033207 conn(0x7f065c0a9c90 msgr2=0x7f065c0aa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:57.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.424+0000 7f0661bd0700 1 -- 172.21.15.31:0/1702033207 shutdown_connections 2024-03-30T01:42:57.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.424+0000 7f0661bd0700 1 -- 172.21.15.31:0/1702033207 wait complete. 2024-03-30T01:42:57.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.425+0000 7f0661bd0700 1 Processor -- start 2024-03-30T01:42:57.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.425+0000 7f0661bd0700 1 -- start start 2024-03-30T01:42:57.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f0661bd0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f065c0ac8b0 0x7f065c14c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f0661bd0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065c0ad0c0 0x7f065c1ddd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f0661bd0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c153c00 0x7f065c1eba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f0661bd0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c1de9f0 con 0x7f065c153c00 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f0661bd0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c149970 con 0x7f065c0ad0c0 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f0661bd0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c070790 con 0x7f065c0ac8b0 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c153c00 0x7f065c1eba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c153c00 0x7f065c1eba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51396/0 (socket says 172.21.15.31:51396) 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065bfff700 1 -- 172.21.15.31:0/3162477687 learned_addr learned my addr 172.21.15.31:0/3162477687 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065affd700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065c0ad0c0 0x7f065c1ddd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065b7fe700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f065c0ac8b0 0x7f065c14c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065bfff700 1 -- 172.21.15.31:0/3162477687 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f065c0ac8b0 msgr2=0x7f065c14c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065bfff700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f065c0ac8b0 0x7f065c14c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065bfff700 1 -- 172.21.15.31:0/3162477687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065c0ad0c0 msgr2=0x7f065c1ddd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065bfff700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065c0ad0c0 0x7f065c1ddd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065bfff700 1 -- 172.21.15.31:0/3162477687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0094a0 con 0x7f065c153c00 2024-03-30T01:42:57.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.426+0000 7f065bfff700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c153c00 0x7f065c1eba50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0654001700 tx=0x7f065400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:57.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.427+0000 7f0643fff700 1 -- 172.21.15.31:0/3162477687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065400ad80 con 0x7f065c153c00 2024-03-30T01:42:57.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.427+0000 7f0643fff700 1 -- 172.21.15.31:0/3162477687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06540070d0 con 0x7f065c153c00 2024-03-30T01:42:57.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.427+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c0704b0 con 0x7f065c153c00 2024-03-30T01:42:57.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.427+0000 7f0643fff700 1 -- 172.21.15.31:0/3162477687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654007480 con 0x7f065c153c00 2024-03-30T01:42:57.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.427+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c1de2c0 con 0x7f065c153c00 2024-03-30T01:42:57.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.428+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c0ab070 con 0x7f065c153c00 2024-03-30T01:42:57.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.428+0000 7f0643fff700 1 -- 172.21.15.31:0/3162477687 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f06540075e0 con 0x7f065c153c00 2024-03-30T01:42:57.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.429+0000 7f0643fff700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f064806ac90 0x7f064806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.429+0000 7f0643fff700 1 -- 172.21.15.31:0/3162477687 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f06540893d0 con 0x7f065c153c00 2024-03-30T01:42:57.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.429+0000 7f065b7fe700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f064806ac90 0x7f064806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.429+0000 7f065b7fe700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f064806ac90 0x7f064806d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f064c000c00 tx=0x7f064c013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:57.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.433+0000 7f0643fff700 1 -- 172.21.15.31:0/3162477687 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0654054460 con 0x7f065c153c00 2024-03-30T01:42:57.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.699+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f065c04ea80 con 0x7f065c153c00 2024-03-30T01:42:57.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.700+0000 7f0643fff700 1 -- 172.21.15.31:0/3162477687 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f0654057a80 con 0x7f065c153c00 2024-03-30T01:42:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:57.706 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4017049280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":4017049280},{"type":"v1","addr":"172.21.15.31:6837","nonce":4017049280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:24.267501+0000","modified":"2024-03-30T01:42:27.273615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f064806ac90 msgr2=0x7f064806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f064806ac90 0x7f064806d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f064c000c00 tx=0x7f064c013040 comp rx=0 tx=0).stop 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c153c00 msgr2=0x7f065c1eba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c153c00 0x7f065c1eba50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0654001700 tx=0x7f065400a340 comp rx=0 tx=0).stop 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 shutdown_connections 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c153c00 0x7f065c1eba50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f064806ac90 0x7f064806d150 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f065c0ac8b0 0x7f065c14c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 --2- 172.21.15.31:0/3162477687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065c0ad0c0 0x7f065c1ddd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 >> 172.21.15.31:0/3162477687 conn(0x7f065c0a9c90 msgr2=0x7f065c0c6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 shutdown_connections 2024-03-30T01:42:57.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.702+0000 7f0661bd0700 1 -- 172.21.15.31:0/3162477687 wait complete. 2024-03-30T01:42:57.710 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-03-30T01:42:57.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:42:57.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.854+0000 7f4b3dfc1700 1 Processor -- start 2024-03-30T01:42:57.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.854+0000 7f4b3dfc1700 1 -- start start 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.854+0000 7f4b3dfc1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380ac520 0x7f4b380ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.854+0000 7f4b3dfc1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b380a74e0 0x7f4b380ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.854+0000 7f4b3dfc1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b38153590 con 0x7f4b380c6000 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b3dfc1700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b380ab680 con 0x7f4b380a74e0 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b3dfc1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b381500c0 con 0x7f4b380ac520 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b36ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b380a74e0 0x7f4b380ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b36ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b380a74e0 0x7f4b380ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51412/0 (socket says 172.21.15.31:51412) 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b36ffd700 1 -- 172.21.15.31:0/4022338272 learned_addr learned my addr 172.21.15.31:0/4022338272 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b377fe700 1 --2- 172.21.15.31:0/4022338272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380ac520 0x7f4b380ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b36ffd700 1 -- 172.21.15.31:0/4022338272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380ac520 msgr2=0x7f4b380ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b36ffd700 1 --2- 172.21.15.31:0/4022338272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380ac520 0x7f4b380ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b36ffd700 1 -- 172.21.15.31:0/4022338272 >> v1:172.21.15.59:6789/0 conn(0x7f4b380c6000 legacy=0x7f4b380c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b36ffd700 1 -- 172.21.15.31:0/4022338272 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b380c7990 con 0x7f4b380a74e0 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b36ffd700 1 --2- 172.21.15.31:0/4022338272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380a74e0 0x7f4b380ace40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0098f0 tx=0x7f4b2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d5833165de1129 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b367fc700 1 -- 172.21.15.31:0/4022338272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c00b8a0 con 0x7f4b380a74e0 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b367fc700 1 -- 172.21.15.31:0/4022338272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c00ba00 con 0x7f4b380a74e0 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.855+0000 7f4b367fc700 1 -- 172.21.15.31:0/4022338272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c00bd90 con 0x7f4b380a74e0 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.856+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/4022338272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380a74e0 msgr2=0x7f4b380ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.856+0000 7f4b3dfc1700 1 --2- 172.21.15.31:0/4022338272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380a74e0 0x7f4b380ace40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0098f0 tx=0x7f4b2c009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.856+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/4022338272 shutdown_connections 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.856+0000 7f4b3dfc1700 1 --2- 172.21.15.31:0/4022338272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380ac520 0x7f4b380ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.856+0000 7f4b3dfc1700 1 --2- 172.21.15.31:0/4022338272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380a74e0 0x7f4b380ace40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.856+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/4022338272 >> 172.21.15.31:0/4022338272 conn(0x7f4b3806ad30 msgr2=0x7f4b380a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.856+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/4022338272 shutdown_connections 2024-03-30T01:42:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.856+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/4022338272 wait complete. 2024-03-30T01:42:57.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.857+0000 7f4b3dfc1700 1 Processor -- start 2024-03-30T01:42:57.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.857+0000 7f4b3dfc1700 1 -- start start 2024-03-30T01:42:57.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b3dfc1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b380a74e0 0x7f4b3814f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b3dfc1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380ac520 0x7f4b3814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b3dfc1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380c6000 0x7f4b381eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b3dfc1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b381dc510 con 0x7f4b380ac520 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b3dfc1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b381e8ec0 con 0x7f4b380a74e0 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b3dfc1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b381e91a0 con 0x7f4b380c6000 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b36ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380ac520 0x7f4b3814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b36ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380ac520 0x7f4b3814f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51414/0 (socket says 172.21.15.31:51414) 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b36ffd700 1 -- 172.21.15.31:0/936131022 learned_addr learned my addr 172.21.15.31:0/936131022 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b377fe700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b380a74e0 0x7f4b3814f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.858+0000 7f4b37fff700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380c6000 0x7f4b381eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b36ffd700 1 -- 172.21.15.31:0/936131022 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380c6000 msgr2=0x7f4b381eb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b36ffd700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380c6000 0x7f4b381eb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b36ffd700 1 -- 172.21.15.31:0/936131022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b380a74e0 msgr2=0x7f4b3814f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b36ffd700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b380a74e0 0x7f4b3814f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b36ffd700 1 -- 172.21.15.31:0/936131022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c009550 con 0x7f4b380ac520 2024-03-30T01:42:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b36ffd700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380ac520 0x7f4b3814f710 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009c40 tx=0x7f4b2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b1ffff700 1 -- 172.21.15.31:0/936131022 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0042c0 con 0x7f4b380ac520 2024-03-30T01:42:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b1ffff700 1 -- 172.21.15.31:0/936131022 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c01f070 con 0x7f4b380ac520 2024-03-30T01:42:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b381e9480 con 0x7f4b380ac520 2024-03-30T01:42:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b1ffff700 1 -- 172.21.15.31:0/936131022 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0036a0 con 0x7f4b380ac520 2024-03-30T01:42:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.859+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b381dbf50 con 0x7f4b380ac520 2024-03-30T01:42:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.860+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3804ea80 con 0x7f4b380ac520 2024-03-30T01:42:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.861+0000 7f4b1ffff700 1 -- 172.21.15.31:0/936131022 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4b2c021030 con 0x7f4b380ac520 2024-03-30T01:42:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.861+0000 7f4b1ffff700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4b2406ac90 0x7f4b2406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.861+0000 7f4b1ffff700 1 -- 172.21.15.31:0/936131022 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f4b2c08a790 con 0x7f4b380ac520 2024-03-30T01:42:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.861+0000 7f4b377fe700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4b2406ac90 0x7f4b2406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.862+0000 7f4b377fe700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4b2406ac90 0x7f4b2406d150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b28006b20 tx=0x7f4b28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:57.866+0000 7f4b1ffff700 1 -- 172.21.15.31:0/936131022 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4b2c055940 con 0x7f4b380ac520 2024-03-30T01:42:58.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.136+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b380c29f0 con 0x7f4b380ac520 2024-03-30T01:42:58.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.137+0000 7f4b1ffff700 1 -- 172.21.15.31:0/936131022 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f4b2c058f60 con 0x7f4b380ac520 2024-03-30T01:42:58.143 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:58.143 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4017049280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":4017049280},{"type":"v1","addr":"172.21.15.31:6837","nonce":4017049280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:42:24.267501+0000","modified":"2024-03-30T01:42:27.273615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5235":{"gid":5235,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.31:6836/2419893471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2419893471},{"type":"v1","addr":"172.21.15.31:6836","nonce":2419893471}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-30T01:42:58.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4b2406ac90 msgr2=0x7f4b2406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4b2406ac90 0x7f4b2406d150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b28006b20 tx=0x7f4b28008040 comp rx=0 tx=0).stop 2024-03-30T01:42:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380ac520 msgr2=0x7f4b3814f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380ac520 0x7f4b3814f710 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009c40 tx=0x7f4b2c004750 comp rx=0 tx=0).stop 2024-03-30T01:42:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 shutdown_connections 2024-03-30T01:42:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f4b2406ac90 0x7f4b2406d150 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b380c6000 0x7f4b381eb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b380a74e0 0x7f4b3814f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 --2- 172.21.15.31:0/936131022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b380ac520 0x7f4b3814f710 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 >> 172.21.15.31:0/936131022 conn(0x7f4b3806ad30 msgr2=0x7f4b38152460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 shutdown_connections 2024-03-30T01:42:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.139+0000 7f4b3dfc1700 1 -- 172.21.15.31:0/936131022 wait complete. 2024-03-30T01:42:58.148 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-03-30T01:42:58.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:42:58.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.290+0000 7fc82052a700 1 Processor -- start 2024-03-30T01:42:58.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.290+0000 7fc82052a700 1 -- start start 2024-03-30T01:42:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.291+0000 7fc82052a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8180c46a0 0x7fc8180c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc82052a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8180a8460 con 0x7fc818072270 2024-03-30T01:42:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc82052a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8180c2e40 con 0x7fc818154cf0 2024-03-30T01:42:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc82052a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806dcb0 con 0x7fc8180c46a0 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81eac7700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc818072270 0x7fc818151490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33330/0 (socket says 172.21.15.31:33330) 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81eac7700 1 -- 172.21.15.31:0/1151893126 learned_addr learned my addr 172.21.15.31:0/1151893126 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81dac5700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8180c46a0 0x7fc8180c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540827320 0 0) 0x7fc8180a8460 con 0x7fc818072270 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8040057c0 con 0x7fc818072270 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181997141 0 0) 0x7fc8180c2e40 con 0x7fc818154cf0 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8180a8460 con 0x7fc818154cf0 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3683868868 0 0) 0x7fc8040057c0 con 0x7fc818072270 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8180c46a0 msgr2=0x7fc8180c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8180c46a0 0x7fc8180c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 >> v1:172.21.15.59:6789/0 conn(0x7fc818154cf0 legacy=0x7fc81806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818155630 con 0x7fc818072270 2024-03-30T01:42:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7fc8100019f0 con 0x7fc818072270 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 >> v1:172.21.15.31:6789/0 conn(0x7fc818072270 legacy=0x7fc818151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8180c46a0 0x7fc804005dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc804006330 0x7fc804006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc804006fa0 0x7fc804007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806dcb0 con 0x7fc804006330 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80400ddb0 con 0x7fc804006fa0 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.292+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80400df40 con 0x7fc8180c46a0 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81dac5700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc804006330 0x7fc804006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81e2c6700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8180c46a0 0x7fc804005dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81eac7700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc804006fa0 0x7fc804007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81dac5700 1 -- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8180c46a0 msgr2=0x7fc804005dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81dac5700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8180c46a0 0x7fc804005dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81dac5700 1 -- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc804006fa0 msgr2=0x7fc804007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81dac5700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc804006fa0 0x7fc804007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81dac5700 1 -- 172.21.15.31:0/1151893126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8040057c0 con 0x7fc804006330 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81e2c6700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8180c46a0 0x7fc804005dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81dac5700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc804006330 0x7fc804006a60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc808009b30 tx=0x7fc804007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45900d99d29ac8d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800e6c0 con 0x7fc804006330 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80800e820 con 0x7fc804006330 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.293+0000 7fc81d2c4700 1 -- 172.21.15.31:0/1151893126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800eb90 con 0x7fc804006330 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.294+0000 7fc82052a700 1 -- 172.21.15.31:0/1151893126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc804006330 msgr2=0x7fc804006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.294+0000 7fc82052a700 1 --2- 172.21.15.31:0/1151893126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc804006330 0x7fc804006a60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc808009b30 tx=0x7fc804007a90 comp rx=0 tx=0).stop 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.294+0000 7fc81eac7700 1 -- 172.21.15.31:0/1151893126 reap_dead start 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.294+0000 7fc82052a700 1 -- 172.21.15.31:0/1151893126 shutdown_connections 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.294+0000 7fc82052a700 1 -- 172.21.15.31:0/1151893126 >> 172.21.15.31:0/1151893126 conn(0x7fc8180c3a60 msgr2=0x7fc8180c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.294+0000 7fc82052a700 1 -- 172.21.15.31:0/1151893126 shutdown_connections 2024-03-30T01:42:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.294+0000 7fc82052a700 1 -- 172.21.15.31:0/1151893126 wait complete. 2024-03-30T01:42:58.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.296+0000 7fc82052a700 1 Processor -- start 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.296+0000 7fc82052a700 1 -- start start 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc82052a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc804006330 0x7fc8180c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc82052a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8181e5ce0 0x7fc8180c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc82052a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc818144950 0x7fc818144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc82052a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818145310 con 0x7fc8181e5ce0 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc82052a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180c6c70 con 0x7fc818144950 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc82052a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc818143b90 con 0x7fc804006330 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81dac5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8181e5ce0 0x7fc8180c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81dac5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8181e5ce0 0x7fc8180c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51422/0 (socket says 172.21.15.31:51422) 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81dac5700 1 -- 172.21.15.31:0/2253613983 learned_addr learned my addr 172.21.15.31:0/2253613983 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81e2c6700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc804006330 0x7fc8180c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81eac7700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc818144950 0x7fc818144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81dac5700 1 -- 172.21.15.31:0/2253613983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc804006330 msgr2=0x7fc8180c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81dac5700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc804006330 0x7fc8180c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81dac5700 1 -- 172.21.15.31:0/2253613983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc818144950 msgr2=0x7fc818144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81dac5700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc818144950 0x7fc818144dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81dac5700 1 -- 172.21.15.31:0/2253613983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818143e70 con 0x7fc8181e5ce0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.297+0000 7fc81dac5700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8181e5ce0 0x7fc8180c5bc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc8040069b0 tx=0x7fc8080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.298+0000 7fc80effd700 1 -- 172.21.15.31:0/2253613983 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800e6c0 con 0x7fc8181e5ce0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.298+0000 7fc80effd700 1 -- 172.21.15.31:0/2253613983 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8080038a0 con 0x7fc8181e5ce0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.298+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc818145310 con 0x7fc8181e5ce0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.298+0000 7fc80effd700 1 -- 172.21.15.31:0/2253613983 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808003c50 con 0x7fc8181e5ce0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.298+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc818144430 con 0x7fc8181e5ce0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.299+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8181447a0 con 0x7fc8181e5ce0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.299+0000 7fc80effd700 1 -- 172.21.15.31:0/2253613983 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc808013470 con 0x7fc8181e5ce0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.300+0000 7fc80effd700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7fc06ac90 0x7fc7fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.300+0000 7fc80effd700 1 -- 172.21.15.31:0/2253613983 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fc80808ac70 con 0x7fc8181e5ce0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.300+0000 7fc81e2c6700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7fc06ac90 0x7fc7fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.300+0000 7fc81e2c6700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7fc06ac90 0x7fc7fc06d150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc814007fd0 tx=0x7fc814009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.304+0000 7fc80effd700 1 -- 172.21.15.31:0/2253613983 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc808055d80 con 0x7fc8181e5ce0 2024-03-30T01:42:58.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:42:58.366 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:42:58.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.522+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc818143cd0 con 0x7fc8181e5ce0 2024-03-30T01:42:58.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.523+0000 7fc80effd700 1 -- 172.21.15.31:0/2253613983 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16431 (secure 0 0 0) 0x7fc818143cd0 con 0x7fc8181e5ce0 2024-03-30T01:42:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:42:58.530 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:42:54.899321+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:42:21.874769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:42:23.236623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6837/1616247357":"2024-03-31T01:42:51.698949+0000","172.21.15.31:6835/1616247357":"2024-03-31T01:42:51.698949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:42:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7fc06ac90 msgr2=0x7fc7fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7fc06ac90 0x7fc7fc06d150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc814007fd0 tx=0x7fc814009040 comp rx=0 tx=0).stop 2024-03-30T01:42:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8181e5ce0 msgr2=0x7fc8180c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8181e5ce0 0x7fc8180c5bc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc8040069b0 tx=0x7fc8080047c0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 shutdown_connections 2024-03-30T01:42:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc7fc06ac90 0x7fc7fc06d150 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc818144950 0x7fc818144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc804006330 0x7fc8180c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 --2- 172.21.15.31:0/2253613983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8181e5ce0 0x7fc8180c5bc0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 >> 172.21.15.31:0/2253613983 conn(0x7fc8180c3a60 msgr2=0x7fc81806de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 shutdown_connections 2024-03-30T01:42:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.525+0000 7fc82052a700 1 -- 172.21.15.31:0/2253613983 wait complete. 2024-03-30T01:42:58.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-30T01:42:58.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.677+0000 7fc8fab19700 1 Processor -- start 2024-03-30T01:42:58.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.677+0000 7fc8fab19700 1 -- start start 2024-03-30T01:42:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.677+0000 7fc8fab19700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8f40c3180 0x7fc8f406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.677+0000 7fc8fab19700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 0x7fc8f406f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.677+0000 7fc8fab19700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f41547d0 con 0x7fc8f406a700 2024-03-30T01:42:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.677+0000 7fc8fab19700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f4154a90 con 0x7fc8f40c3180 2024-03-30T01:42:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.677+0000 7fc8fab19700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4154d70 con 0x7fc8f40c4710 2024-03-30T01:42:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8f88b5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8f40c3180 0x7fc8f406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8f88b5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8f40c3180 0x7fc8f406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51438/0 (socket says 172.21.15.31:51438) 2024-03-30T01:42:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8f88b5700 1 -- 172.21.15.31:0/1294116409 learned_addr learned my addr 172.21.15.31:0/1294116409 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:58.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8f90b6700 1 --2- 172.21.15.31:0/1294116409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 0x7fc8f406f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8f88b5700 1 -- 172.21.15.31:0/1294116409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 msgr2=0x7fc8f406f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8f88b5700 1 --2- 172.21.15.31:0/1294116409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 0x7fc8f406f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8f88b5700 1 -- 172.21.15.31:0/1294116409 >> v1:172.21.15.59:6789/0 conn(0x7fc8f406a700 legacy=0x7fc8f40c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8f88b5700 1 -- 172.21.15.31:0/1294116409 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f4155050 con 0x7fc8f40c3180 2024-03-30T01:42:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8f88b5700 1 --2- 172.21.15.31:0/1294116409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f40c3180 0x7fc8f406eb00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00098f0 tx=0x7fc8f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945e48ed2294bf3d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8eb7fe700 1 -- 172.21.15.31:0/1294116409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0004230 con 0x7fc8f40c3180 2024-03-30T01:42:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8eb7fe700 1 -- 172.21.15.31:0/1294116409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0004390 con 0x7fc8f40c3180 2024-03-30T01:42:58.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.678+0000 7fc8eb7fe700 1 -- 172.21.15.31:0/1294116409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00036a0 con 0x7fc8f40c3180 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.679+0000 7fc8fab19700 1 -- 172.21.15.31:0/1294116409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f40c3180 msgr2=0x7fc8f406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.679+0000 7fc8fab19700 1 --2- 172.21.15.31:0/1294116409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f40c3180 0x7fc8f406eb00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00098f0 tx=0x7fc8f0009c00 comp rx=0 tx=0).stop 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.679+0000 7fc8fab19700 1 -- 172.21.15.31:0/1294116409 shutdown_connections 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.679+0000 7fc8fab19700 1 --2- 172.21.15.31:0/1294116409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f40c3180 0x7fc8f406eb00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.679+0000 7fc8fab19700 1 --2- 172.21.15.31:0/1294116409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 0x7fc8f406f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.679+0000 7fc8fab19700 1 -- 172.21.15.31:0/1294116409 >> 172.21.15.31:0/1294116409 conn(0x7fc8f40c3a60 msgr2=0x7fc8f40c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.679+0000 7fc8fab19700 1 -- 172.21.15.31:0/1294116409 shutdown_connections 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.679+0000 7fc8fab19700 1 -- 172.21.15.31:0/1294116409 wait complete. 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.680+0000 7fc8fab19700 1 Processor -- start 2024-03-30T01:42:58.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.680+0000 7fc8fab19700 1 -- start start 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8fab19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f406a700 0x7fc8f4155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8fab19700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8f40c3180 0x7fc8f41eb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8fab19700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 0x7fc8f41eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8fab19700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4148300 con 0x7fc8f406a700 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8fab19700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40c5f30 con 0x7fc8f40c3180 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8fab19700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40c5600 con 0x7fc8f40c4710 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8f88b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f406a700 0x7fc8f4155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8f88b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f406a700 0x7fc8f4155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51452/0 (socket says 172.21.15.31:51452) 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8f88b5700 1 -- 172.21.15.31:0/1913595080 learned_addr learned my addr 172.21.15.31:0/1913595080 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8ebfff700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8f40c3180 0x7fc8f41eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.681+0000 7fc8f90b6700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 0x7fc8f41eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8f88b5700 1 -- 172.21.15.31:0/1913595080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 msgr2=0x7fc8f41eb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8f88b5700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 0x7fc8f41eb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8f88b5700 1 -- 172.21.15.31:0/1913595080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8f40c3180 msgr2=0x7fc8f41eb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8f88b5700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8f40c3180 0x7fc8f41eb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8f88b5700 1 -- 172.21.15.31:0/1913595080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f0009550 con 0x7fc8f406a700 2024-03-30T01:42:58.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8f88b5700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f406a700 0x7fc8f4155b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4073220 tx=0x7fc8f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8e97fa700 1 -- 172.21.15.31:0/1913595080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0003cd0 con 0x7fc8f406a700 2024-03-30T01:42:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8e97fa700 1 -- 172.21.15.31:0/1913595080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f001f070 con 0x7fc8f406a700 2024-03-30T01:42:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f40c58e0 con 0x7fc8f406a700 2024-03-30T01:42:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8e97fa700 1 -- 172.21.15.31:0/1913595080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000b8a0 con 0x7fc8f406a700 2024-03-30T01:42:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.682+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f40c63c0 con 0x7fc8f406a700 2024-03-30T01:42:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.683+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f40c6210 con 0x7fc8f406a700 2024-03-30T01:42:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.684+0000 7fc8e97fa700 1 -- 172.21.15.31:0/1913595080 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc8f0021030 con 0x7fc8f406a700 2024-03-30T01:42:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.684+0000 7fc8e97fa700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc8e006acc0 0x7fc8e006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.684+0000 7fc8e97fa700 1 -- 172.21.15.31:0/1913595080 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fc8f008a9c0 con 0x7fc8f406a700 2024-03-30T01:42:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.684+0000 7fc8ebfff700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc8e006acc0 0x7fc8e006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.685+0000 7fc8ebfff700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc8e006acc0 0x7fc8e006d180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc8f41e7a40 tx=0x7fc8e4007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:58.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.689+0000 7fc8e97fa700 1 -- 172.21.15.31:0/1913595080 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc8f0055a50 con 0x7fc8f406a700 2024-03-30T01:42:58.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.953+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7fc8f40c6210 con 0x7fc8f406a700 2024-03-30T01:42:58.960 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-03-30T01:42:58.954+0000 7fd8b60ee700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T01:42:58.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.968+0000 7fc8e97fa700 1 -- 172.21.15.31:0/1913595080 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 163+0+0 (secure 0 0 0) 0x7fc8f0059070 con 0x7fc8f406a700 2024-03-30T01:42:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc8e006acc0 msgr2=0x7fc8e006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc8e006acc0 0x7fc8e006d180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc8f41e7a40 tx=0x7fc8e4007ec0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f406a700 msgr2=0x7fc8f4155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:58.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f406a700 0x7fc8f4155b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4073220 tx=0x7fc8f0004750 comp rx=0 tx=0).stop 2024-03-30T01:42:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 shutdown_connections 2024-03-30T01:42:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f406a700 0x7fc8f4155b50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc8e006acc0 0x7fc8e006d180 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc8f40c4710 0x7fc8f41eb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 --2- 172.21.15.31:0/1913595080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8f40c3180 0x7fc8f41eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 >> 172.21.15.31:0/1913595080 conn(0x7fc8f40c3a60 msgr2=0x7fc8f414d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 shutdown_connections 2024-03-30T01:42:58.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:58.970+0000 7fc8fab19700 1 -- 172.21.15.31:0/1913595080 wait complete. 2024-03-30T01:42:58.980 INFO:tasks.ceph.mds.a.smithi031.stderr: -3432> 2024-03-30T01:42:27.785+0000 7fe21e364700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T01:42:58.981 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T01:42:58.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-30T01:42:59.008 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-03-30T01:42:59.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.125+0000 7f07ee9be700 1 Processor -- start 2024-03-30T01:42:59.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.125+0000 7f07ee9be700 1 -- start start 2024-03-30T01:42:59.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07ee9be700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07e806d040 0x7f07e8072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:59.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07ee9be700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f07e806d900 0x7f07e806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:59.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07ee9be700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e814ee60 con 0x7f07e81549a0 2024-03-30T01:42:59.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07ee9be700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80c2290 con 0x7f07e806d040 2024-03-30T01:42:59.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07ee9be700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e8155490 con 0x7f07e806d900 2024-03-30T01:42:59.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07e77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07e806d040 0x7f07e8072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:59.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07e7fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f07e81549a0 0x7f07e8154d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33344/0 (socket says 172.21.15.31:33344) 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07e7fff700 1 -- 172.21.15.31:0/4240382771 learned_addr learned my addr 172.21.15.31:0/4240382771 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07e77fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07e806d040 0x7f07e8072410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51482/0 (socket says 172.21.15.31:51482) 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07ecf5b700 1 --2- 172.21.15.31:0/4240382771 >> v2:172.21.15.59:3300/0 conn(0x7f07e806d900 0x7f07e806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07e77fe700 1 -- 172.21.15.31:0/4240382771 >> v2:172.21.15.59:3300/0 conn(0x7f07e806d900 msgr2=0x7f07e806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07e77fe700 1 --2- 172.21.15.31:0/4240382771 >> v2:172.21.15.59:3300/0 conn(0x7f07e806d900 0x7f07e806dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07e77fe700 1 -- 172.21.15.31:0/4240382771 >> v1:172.21.15.31:6789/0 conn(0x7f07e81549a0 legacy=0x7f07e8154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07e77fe700 1 -- 172.21.15.31:0/4240382771 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e806e270 con 0x7f07e806d040 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.126+0000 7f07ecf5b700 1 --2- 172.21.15.31:0/4240382771 >> v2:172.21.15.59:3300/0 conn(0x7f07e806d900 0x7f07e806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07e77fe700 1 --2- 172.21.15.31:0/4240382771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d040 0x7f07e8072410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f07e80c6e30 tx=0x7f07dc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef1786f426ea298 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07e6ffd700 1 -- 172.21.15.31:0/4240382771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc011830 con 0x7f07e806d040 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07e6ffd700 1 -- 172.21.15.31:0/4240382771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc011990 con 0x7f07e806d040 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07e6ffd700 1 -- 172.21.15.31:0/4240382771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc011d20 con 0x7f07e806d040 2024-03-30T01:42:59.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07ee9be700 1 -- 172.21.15.31:0/4240382771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d040 msgr2=0x7f07e8072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:59.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07ee9be700 1 --2- 172.21.15.31:0/4240382771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d040 0x7f07e8072410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f07e80c6e30 tx=0x7f07dc009b20 comp rx=0 tx=0).stop 2024-03-30T01:42:59.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07ee9be700 1 -- 172.21.15.31:0/4240382771 shutdown_connections 2024-03-30T01:42:59.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07ee9be700 1 --2- 172.21.15.31:0/4240382771 >> v2:172.21.15.59:3300/0 conn(0x7f07e806d900 0x7f07e806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:59.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07ee9be700 1 --2- 172.21.15.31:0/4240382771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d040 0x7f07e8072410 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:59.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07ee9be700 1 -- 172.21.15.31:0/4240382771 >> 172.21.15.31:0/4240382771 conn(0x7f07e80c3980 msgr2=0x7f07e80c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:59.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07ee9be700 1 -- 172.21.15.31:0/4240382771 shutdown_connections 2024-03-30T01:42:59.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.127+0000 7f07ee9be700 1 -- 172.21.15.31:0/4240382771 wait complete. 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.129+0000 7f07ee9be700 1 Processor -- start 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.129+0000 7f07ee9be700 1 -- start start 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07ee9be700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07e806d040 0x7f07e814a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07ee9be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d900 0x7f07e8140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07ee9be700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07e81549a0 0x7f07e81412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07ee9be700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80aa850 con 0x7f07e806d900 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07ee9be700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e81417e0 con 0x7f07e806d040 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07ee9be700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8147090 con 0x7f07e81549a0 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d900 0x7f07e8140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d900 0x7f07e8140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51484/0 (socket says 172.21.15.31:51484) 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e77fe700 1 -- 172.21.15.31:0/3960945409 learned_addr learned my addr 172.21.15.31:0/3960945409 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e7fff700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07e806d040 0x7f07e814a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07ecf5b700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07e81549a0 0x7f07e81412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e77fe700 1 -- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07e81549a0 msgr2=0x7f07e81412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e77fe700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07e81549a0 0x7f07e81412a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e77fe700 1 -- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07e806d040 msgr2=0x7f07e814a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e77fe700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07e806d040 0x7f07e814a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e77fe700 1 -- 172.21.15.31:0/3960945409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc0094a0 con 0x7f07e806d900 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07ecf5b700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07e81549a0 0x7f07e81412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e7fff700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07e806d040 0x7f07e814a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e77fe700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d900 0x7f07e8140d60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0090b0 tx=0x7f07dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e4ff9700 1 -- 172.21.15.31:0/3960945409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0036a0 con 0x7f07e806d900 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e4ff9700 1 -- 172.21.15.31:0/3960945409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc01f070 con 0x7f07e806d900 2024-03-30T01:42:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e81ea550 con 0x7f07e806d900 2024-03-30T01:42:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.130+0000 7f07e4ff9700 1 -- 172.21.15.31:0/3960945409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc003a60 con 0x7f07e806d900 2024-03-30T01:42:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.131+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e81eab10 con 0x7f07e806d900 2024-03-30T01:42:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.131+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e80c4870 con 0x7f07e806d900 2024-03-30T01:42:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.132+0000 7f07e4ff9700 1 -- 172.21.15.31:0/3960945409 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f07dc003bc0 con 0x7f07e806d900 2024-03-30T01:42:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.132+0000 7f07e4ff9700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f07d406ac90 0x7f07d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:42:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.132+0000 7f07e4ff9700 1 -- 172.21.15.31:0/3960945409 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6263+0+0 (secure 0 0 0) 0x7f07dc021070 con 0x7f07e806d900 2024-03-30T01:42:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.133+0000 7f07e7fff700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f07d406ac90 0x7f07d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:42:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.133+0000 7f07e7fff700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f07d406ac90 0x7f07d406d150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f07d8005910 tx=0x7f07d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:42:59.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.137+0000 7f07e4ff9700 1 -- 172.21.15.31:0/3960945409 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f07dc08f070 con 0x7f07e806d900 2024-03-30T01:42:59.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.400+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f07e80c2a60 con 0x7f07e806d900 2024-03-30T01:42:59.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.969+0000 7f07e4ff9700 1 -- 172.21.15.31:0/3960945409 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 105+0+0 (secure 0 0 0) 0x7f07dc015020 con 0x7f07e806d900 2024-03-30T01:42:59.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.972+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f07d406ac90 msgr2=0x7f07d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.972+0000 7f07ee9be700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f07d406ac90 0x7f07d406d150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f07d8005910 tx=0x7f07d8009040 comp rx=0 tx=0).stop 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d900 msgr2=0x7f07e8140d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d900 0x7f07e8140d60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0090b0 tx=0x7f07dc004750 comp rx=0 tx=0).stop 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 shutdown_connections 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f07d406ac90 0x7f07d406d150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07e81549a0 0x7f07e81412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07e806d040 0x7f07e814a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 --2- 172.21.15.31:0/3960945409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e806d900 0x7f07e8140d60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 >> 172.21.15.31:0/3960945409 conn(0x7f07e80c3980 msgr2=0x7f07e806f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 shutdown_connections 2024-03-30T01:42:59.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:42:59.973+0000 7f07ee9be700 1 -- 172.21.15.31:0/3960945409 wait complete. 2024-03-30T01:42:59.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-30T01:43:00.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.182+0000 7f08254cd700 1 Processor -- start 2024-03-30T01:43:00.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.182+0000 7f08254cd700 1 -- start start 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.182+0000 7f08254cd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f08200a9900 0x7f0820072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.182+0000 7f08254cd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08200c21b0 0x7f082015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.182+0000 7f08254cd700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08200a5450 con 0x7f08200c0b50 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.182+0000 7f08254cd700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08200c2620 con 0x7f08200c21b0 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.182+0000 7f08254cd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08200c1d20 con 0x7f08200a9900 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08200c21b0 0x7f082015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081e7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f08200c0b50 0x7f0820070070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33350/0 (socket says 172.21.15.31:33350) 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081e7fc700 1 -- 172.21.15.31:0/1486269518 learned_addr learned my addr 172.21.15.31:0/1486269518 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081effd700 1 --2- 172.21.15.31:0/1486269518 >> v2:172.21.15.59:3300/0 conn(0x7f08200a9900 0x7f0820072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081dffb700 1 -- 172.21.15.31:0/1486269518 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549096778 0 0) 0x7f08200a5450 con 0x7f08200c0b50 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081dffb700 1 -- 172.21.15.31:0/1486269518 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08080035b0 con 0x7f08200c0b50 2024-03-30T01:43:00.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081f7fe700 1 -- 172.21.15.31:0/1486269518 >> v2:172.21.15.59:3300/0 conn(0x7f08200a9900 msgr2=0x7f0820072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081f7fe700 1 --2- 172.21.15.31:0/1486269518 >> v2:172.21.15.59:3300/0 conn(0x7f08200a9900 0x7f0820072aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081f7fe700 1 -- 172.21.15.31:0/1486269518 >> v1:172.21.15.31:6789/0 conn(0x7f08200c0b50 legacy=0x7f0820070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081f7fe700 1 -- 172.21.15.31:0/1486269518 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0820070780 con 0x7f08200c21b0 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081f7fe700 1 --2- 172.21.15.31:0/1486269518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200c21b0 0x7f082015aae0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f08200c6b80 tx=0x7f081800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec23da969b5495a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081dffb700 1 -- 172.21.15.31:0/1486269518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800f070 con 0x7f08200c21b0 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081dffb700 1 -- 172.21.15.31:0/1486269518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081800a2e0 con 0x7f08200c21b0 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.183+0000 7f081dffb700 1 -- 172.21.15.31:0/1486269518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800cb10 con 0x7f08200c21b0 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.184+0000 7f08254cd700 1 -- 172.21.15.31:0/1486269518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200c21b0 msgr2=0x7f082015aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.184+0000 7f08254cd700 1 --2- 172.21.15.31:0/1486269518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200c21b0 0x7f082015aae0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f08200c6b80 tx=0x7f081800c330 comp rx=0 tx=0).stop 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.184+0000 7f08254cd700 1 -- 172.21.15.31:0/1486269518 shutdown_connections 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.184+0000 7f08254cd700 1 --2- 172.21.15.31:0/1486269518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200c21b0 0x7f082015aae0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.184+0000 7f08254cd700 1 --2- 172.21.15.31:0/1486269518 >> v2:172.21.15.59:3300/0 conn(0x7f08200a9900 0x7f0820072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.184+0000 7f08254cd700 1 -- 172.21.15.31:0/1486269518 >> 172.21.15.31:0/1486269518 conn(0x7f08200c5c60 msgr2=0x7f08200c6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:00.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.184+0000 7f08254cd700 1 -- 172.21.15.31:0/1486269518 shutdown_connections 2024-03-30T01:43:00.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.184+0000 7f08254cd700 1 -- 172.21.15.31:0/1486269518 wait complete. 2024-03-30T01:43:00.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.185+0000 7f08254cd700 1 Processor -- start 2024-03-30T01:43:00.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f08254cd700 1 -- start start 2024-03-30T01:43:00.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f08254cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200a9900 0x7f08201d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:00.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f08254cd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200c0b50 0x7f08201e5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:00.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f08254cd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08200c21b0 0x7f08201e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:00.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f08254cd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08201e60f0 con 0x7f08200a9900 2024-03-30T01:43:00.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f08254cd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08201e8be0 con 0x7f08200c0b50 2024-03-30T01:43:00.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f08254cd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08200a8080 con 0x7f08200c21b0 2024-03-30T01:43:00.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f081effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200a9900 0x7f08201d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f081effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200a9900 0x7f08201d7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51500/0 (socket says 172.21.15.31:51500) 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.186+0000 7f081effd700 1 -- 172.21.15.31:0/2612626121 learned_addr learned my addr 172.21.15.31:0/2612626121 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f081e7fc700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200c0b50 0x7f08201e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f081f7fe700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08200c21b0 0x7f08201e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f081effd700 1 -- 172.21.15.31:0/2612626121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08200c21b0 msgr2=0x7f08201e5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f081effd700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08200c21b0 0x7f08201e5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f081effd700 1 -- 172.21.15.31:0/2612626121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200c0b50 msgr2=0x7f08201e5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f081effd700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200c0b50 0x7f08201e5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f081effd700 1 -- 172.21.15.31:0/2612626121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081800ab80 con 0x7f08200a9900 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f081e7fc700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200c0b50 0x7f08201e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f081effd700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200a9900 0x7f08201d7a80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f081000e810 tx=0x7f081000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f08077fe700 1 -- 172.21.15.31:0/2612626121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810013610 con 0x7f08200a9900 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f08077fe700 1 -- 172.21.15.31:0/2612626121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0810013770 con 0x7f08200a9900 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08201e8ea0 con 0x7f08200a9900 2024-03-30T01:43:00.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f08077fe700 1 -- 172.21.15.31:0/2612626121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810013b20 con 0x7f08200a9900 2024-03-30T01:43:00.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.187+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08201d7fc0 con 0x7f08200a9900 2024-03-30T01:43:00.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.188+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08201e97d0 con 0x7f08200a9900 2024-03-30T01:43:00.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.189+0000 7f08077fe700 1 -- 172.21.15.31:0/2612626121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0810013c80 con 0x7f08200a9900 2024-03-30T01:43:00.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.189+0000 7f08077fe700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f080c06ac90 0x7f080c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:00.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.189+0000 7f08077fe700 1 -- 172.21.15.31:0/2612626121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6263+0+0 (secure 0 0 0) 0x7f08100891c0 con 0x7f08200a9900 2024-03-30T01:43:00.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.189+0000 7f081e7fc700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f080c06ac90 0x7f080c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:00.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.190+0000 7f081e7fc700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f080c06ac90 0x7f080c06d150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f08200a4c40 tx=0x7f0814009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:00.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.194+0000 7f08077fe700 1 -- 172.21.15.31:0/2612626121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0810054180 con 0x7f08200a9900 2024-03-30T01:43:00.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.408+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f082004ea80 con 0x7f08200a9900 2024-03-30T01:43:00.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.972+0000 7f08077fe700 1 -- 172.21.15.31:0/2612626121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v50) v1 ==== 184+0+0 (secure 0 0 0) 0x7f08100577a0 con 0x7f08200a9900 2024-03-30T01:43:00.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f080c06ac90 msgr2=0x7f080c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f080c06ac90 0x7f080c06d150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f08200a4c40 tx=0x7f0814009040 comp rx=0 tx=0).stop 2024-03-30T01:43:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200a9900 msgr2=0x7f08201d7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200a9900 0x7f08201d7a80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f081000e810 tx=0x7f081000ebd0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 shutdown_connections 2024-03-30T01:43:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08200a9900 0x7f08201d7a80 secure :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f081000e810 tx=0x7f081000ebd0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f080c06ac90 0x7f080c06d150 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08200c21b0 0x7f08201e5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 --2- 172.21.15.31:0/2612626121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200c0b50 0x7f08201e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:00.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.978+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 >> 172.21.15.31:0/2612626121 conn(0x7f08200c5c60 msgr2=0x7f08200c6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.979+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 shutdown_connections 2024-03-30T01:43:00.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:00.979+0000 7f08254cd700 1 -- 172.21.15.31:0/2612626121 wait complete. 2024-03-30T01:43:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs2_metadata' removed 2024-03-30T01:43:01.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-30T01:43:01.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.140+0000 7f0d26289700 1 Processor -- start 2024-03-30T01:43:01.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.140+0000 7f0d26289700 1 -- start start 2024-03-30T01:43:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d26289700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d20151f00 0x7f0d200c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d26289700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d201522e0 con 0x7f0d20066870 2024-03-30T01:43:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d26289700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d20155290 con 0x7f0d200bf9f0 2024-03-30T01:43:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d26289700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d200c2760 con 0x7f0d20151f00 2024-03-30T01:43:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d1ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0d20151f00 0x7f0d200c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d1f7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0d20066870 0x7f0d2006b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33358/0 (socket says 172.21.15.31:33358) 2024-03-30T01:43:01.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d1f7fe700 1 -- 172.21.15.31:0/1018702268 learned_addr learned my addr 172.21.15.31:0/1018702268 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d1ffff700 1 --2- 172.21.15.31:0/1018702268 >> v2:172.21.15.31:3300/0 conn(0x7f0d20151f00 0x7f0d200c12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51514/0 (socket says 172.21.15.31:51514) 2024-03-30T01:43:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d1ffff700 1 -- 172.21.15.31:0/1018702268 >> v1:172.21.15.59:6789/0 conn(0x7f0d200bf9f0 legacy=0x7f0d200c19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d1ffff700 1 -- 172.21.15.31:0/1018702268 >> v1:172.21.15.31:6789/0 conn(0x7f0d20066870 legacy=0x7f0d2006b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.141+0000 7f0d1ffff700 1 -- 172.21.15.31:0/1018702268 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d201557c0 con 0x7f0d20151f00 2024-03-30T01:43:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d1ffff700 1 --2- 172.21.15.31:0/1018702268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 0x7f0d200c12f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009840 tx=0x7f0d10009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a78b4fccb83e011 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d1effd700 1 -- 172.21.15.31:0/1018702268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10011830 con 0x7f0d20151f00 2024-03-30T01:43:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d1effd700 1 -- 172.21.15.31:0/1018702268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10011990 con 0x7f0d20151f00 2024-03-30T01:43:01.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d1effd700 1 -- 172.21.15.31:0/1018702268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10011d20 con 0x7f0d20151f00 2024-03-30T01:43:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d26289700 1 -- 172.21.15.31:0/1018702268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 msgr2=0x7f0d200c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d26289700 1 --2- 172.21.15.31:0/1018702268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 0x7f0d200c12f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009840 tx=0x7f0d10009b50 comp rx=0 tx=0).stop 2024-03-30T01:43:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d26289700 1 -- 172.21.15.31:0/1018702268 shutdown_connections 2024-03-30T01:43:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d26289700 1 --2- 172.21.15.31:0/1018702268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 0x7f0d200c12f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d26289700 1 -- 172.21.15.31:0/1018702268 >> 172.21.15.31:0/1018702268 conn(0x7f0d200c30c0 msgr2=0x7f0d200c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:01.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d26289700 1 -- 172.21.15.31:0/1018702268 shutdown_connections 2024-03-30T01:43:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.142+0000 7f0d26289700 1 -- 172.21.15.31:0/1018702268 wait complete. 2024-03-30T01:43:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.144+0000 7f0d26289700 1 Processor -- start 2024-03-30T01:43:01.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.144+0000 7f0d26289700 1 -- start start 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.144+0000 7f0d26289700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d20066870 0x7f0d201e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.144+0000 7f0d26289700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d200bf9f0 0x7f0d201469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d26289700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 0x7f0d201538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d26289700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2014b310 con 0x7f0d20151f00 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d26289700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20158b60 con 0x7f0d200bf9f0 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d26289700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d201ea190 con 0x7f0d20066870 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d24826700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 0x7f0d201538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d24826700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 0x7f0d201538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51526/0 (socket says 172.21.15.31:51526) 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d24826700 1 -- 172.21.15.31:0/1931273191 learned_addr learned my addr 172.21.15.31:0/1931273191 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d1f7fe700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d200bf9f0 0x7f0d201469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:01.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d1ffff700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d20066870 0x7f0d201e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d24826700 1 -- 172.21.15.31:0/1931273191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d20066870 msgr2=0x7f0d201e9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d24826700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d20066870 0x7f0d201e9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d24826700 1 -- 172.21.15.31:0/1931273191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d200bf9f0 msgr2=0x7f0d201469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d24826700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d200bf9f0 0x7f0d201469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d24826700 1 -- 172.21.15.31:0/1931273191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100094a0 con 0x7f0d20151f00 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d24826700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 0x7f0d201538d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800cc50 tx=0x7f0d1800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d1cff9700 1 -- 172.21.15.31:0/1931273191 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800ad80 con 0x7f0d20151f00 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d1cff9700 1 -- 172.21.15.31:0/1931273191 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1800ce50 con 0x7f0d20151f00 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d201ea470 con 0x7f0d20151f00 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d1cff9700 1 -- 172.21.15.31:0/1931273191 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18003c40 con 0x7f0d20151f00 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.145+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d201eaa30 con 0x7f0d20151f00 2024-03-30T01:43:01.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.146+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2014b450 con 0x7f0d20151f00 2024-03-30T01:43:01.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.147+0000 7f0d1cff9700 1 -- 172.21.15.31:0/1931273191 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0d180050d0 con 0x7f0d20151f00 2024-03-30T01:43:01.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.148+0000 7f0d1cff9700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d0c06ac90 0x7f0d0c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:01.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.148+0000 7f0d1ffff700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d0c06ac90 0x7f0d0c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:01.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.148+0000 7f0d1cff9700 1 -- 172.21.15.31:0/1931273191 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f0d180893e0 con 0x7f0d20151f00 2024-03-30T01:43:01.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.148+0000 7f0d1ffff700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d0c06ac90 0x7f0d0c06d150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d100090b0 tx=0x7f0d1001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:01.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.152+0000 7f0d1cff9700 1 -- 172.21.15.31:0/1931273191 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0d2014b450 con 0x7f0d20151f00 2024-03-30T01:43:01.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.371+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0d2014b450 con 0x7f0d20151f00 2024-03-30T01:43:01.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.983+0000 7f0d1cff9700 1 -- 172.21.15.31:0/1931273191 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v51) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0d2014b450 con 0x7f0d20151f00 2024-03-30T01:43:01.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.989+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d0c06ac90 msgr2=0x7f0d0c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:01.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.989+0000 7f0d26289700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d0c06ac90 0x7f0d0c06d150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d100090b0 tx=0x7f0d1001c040 comp rx=0 tx=0).stop 2024-03-30T01:43:01.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.989+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 msgr2=0x7f0d201538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:01.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.989+0000 7f0d26289700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 0x7f0d201538d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800cc50 tx=0x7f0d1800a340 comp rx=0 tx=0).stop 2024-03-30T01:43:01.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.990+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 shutdown_connections 2024-03-30T01:43:01.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.990+0000 7f0d26289700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d20151f00 0x7f0d201538d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:01.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.990+0000 7f0d26289700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0d0c06ac90 0x7f0d0c06d150 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:01.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.990+0000 7f0d26289700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d20066870 0x7f0d201e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:01.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.990+0000 7f0d26289700 1 --2- 172.21.15.31:0/1931273191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d200bf9f0 0x7f0d201469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:01.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.990+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 >> 172.21.15.31:0/1931273191 conn(0x7f0d200c30c0 msgr2=0x7f0d200c1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:01.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.990+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 shutdown_connections 2024-03-30T01:43:01.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:01.990+0000 7f0d26289700 1 -- 172.21.15.31:0/1931273191 wait complete. 2024-03-30T01:43:01.997 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs2_data' removed 2024-03-30T01:43:02.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T01:43:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.147+0000 7f73a9cc8700 1 Processor -- start 2024-03-30T01:43:02.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.147+0000 7f73a9cc8700 1 -- start start 2024-03-30T01:43:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.148+0000 7f73a9cc8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a406f310 0x7f73a406f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.148+0000 7f73a9cc8700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a4153590 con 0x7f73a40c24f0 2024-03-30T01:43:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.148+0000 7f73a9cc8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a40a8840 con 0x7f73a414fc70 2024-03-30T01:43:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.148+0000 7f73a9cc8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40a9650 con 0x7f73a406f310 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a2ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f73a40c24f0 0x7f73a4152e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33374/0 (socket says 172.21.15.31:33374) 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a2ffd700 1 -- 172.21.15.31:0/2560143661 learned_addr learned my addr 172.21.15.31:0/2560143661 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a3fff700 1 --2- 172.21.15.31:0/2560143661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a406f310 0x7f73a406f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a27fc700 1 -- 172.21.15.31:0/2560143661 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325504563 0 0) 0x7f7398001110 con 0x7f73a40c24f0 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a27fc700 1 -- 172.21.15.31:0/2560143661 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f738c0035b0 con 0x7f73a40c24f0 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a27fc700 1 -- 172.21.15.31:0/2560143661 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655186569 0 0) 0x7f73a40a8840 con 0x7f73a414fc70 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a27fc700 1 -- 172.21.15.31:0/2560143661 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7398001110 con 0x7f73a414fc70 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a3fff700 1 -- 172.21.15.31:0/2560143661 >> v1:172.21.15.59:6789/0 conn(0x7f73a414fc70 legacy=0x7f73a4156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a3fff700 1 -- 172.21.15.31:0/2560143661 >> v1:172.21.15.31:6789/0 conn(0x7f73a40c24f0 legacy=0x7f73a4152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a3fff700 1 -- 172.21.15.31:0/2560143661 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a40a8b20 con 0x7f73a406f310 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.149+0000 7f73a3fff700 1 --2- 172.21.15.31:0/2560143661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a406f310 0x7f73a406f760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f739c009df0 tx=0x7f739c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2cc9810e12032a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a27fc700 1 -- 172.21.15.31:0/2560143661 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00e070 con 0x7f73a406f310 2024-03-30T01:43:02.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a27fc700 1 -- 172.21.15.31:0/2560143661 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c00bae0 con 0x7f73a406f310 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a27fc700 1 -- 172.21.15.31:0/2560143661 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00be50 con 0x7f73a406f310 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a9cc8700 1 -- 172.21.15.31:0/2560143661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a406f310 msgr2=0x7f73a406f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a9cc8700 1 --2- 172.21.15.31:0/2560143661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a406f310 0x7f73a406f760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f739c009df0 tx=0x7f739c00b330 comp rx=0 tx=0).stop 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a9cc8700 1 -- 172.21.15.31:0/2560143661 shutdown_connections 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a9cc8700 1 --2- 172.21.15.31:0/2560143661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a406f310 0x7f73a406f760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a9cc8700 1 -- 172.21.15.31:0/2560143661 >> 172.21.15.31:0/2560143661 conn(0x7f73a40c3980 msgr2=0x7f73a40c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a9cc8700 1 -- 172.21.15.31:0/2560143661 shutdown_connections 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.150+0000 7f73a9cc8700 1 -- 172.21.15.31:0/2560143661 wait complete. 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.151+0000 7f73a9cc8700 1 Processor -- start 2024-03-30T01:43:02.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.152+0000 7f73a9cc8700 1 -- start start 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.152+0000 7f73a9cc8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a406f310 0x7f73a40bcee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.152+0000 7f73a9cc8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a40c24f0 0x7f73a414ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.152+0000 7f73a9cc8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73a414fc70 0x7f73a40c09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.152+0000 7f73a9cc8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a413f380 con 0x7f73a414fc70 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.152+0000 7f73a9cc8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a41eae10 con 0x7f73a406f310 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.152+0000 7f73a9cc8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a41eb6b0 con 0x7f73a40c24f0 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73a414fc70 0x7f73a40c09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73a414fc70 0x7f73a40c09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51530/0 (socket says 172.21.15.31:51530) 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a3fff700 1 -- 172.21.15.31:0/1173040765 learned_addr learned my addr 172.21.15.31:0/1173040765 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a37fe700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a406f310 0x7f73a40bcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a2ffd700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a40c24f0 0x7f73a414ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a3fff700 1 -- 172.21.15.31:0/1173040765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a40c24f0 msgr2=0x7f73a414ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a3fff700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a40c24f0 0x7f73a414ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a3fff700 1 -- 172.21.15.31:0/1173040765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a406f310 msgr2=0x7f73a40bcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a3fff700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a406f310 0x7f73a40bcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a3fff700 1 -- 172.21.15.31:0/1173040765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c009a70 con 0x7f73a414fc70 2024-03-30T01:43:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a3fff700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73a414fc70 0x7f73a40c09c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f739c014040 tx=0x7f739c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f738bfff700 1 -- 172.21.15.31:0/1173040765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c004900 con 0x7f73a414fc70 2024-03-30T01:43:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f738bfff700 1 -- 172.21.15.31:0/1173040765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c020070 con 0x7f73a414fc70 2024-03-30T01:43:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a40c12a0 con 0x7f73a414fc70 2024-03-30T01:43:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.153+0000 7f738bfff700 1 -- 172.21.15.31:0/1173040765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c0276e0 con 0x7f73a414fc70 2024-03-30T01:43:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.154+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a41eb340 con 0x7f73a414fc70 2024-03-30T01:43:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.155+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a40c15d0 con 0x7f73a414fc70 2024-03-30T01:43:02.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.155+0000 7f738bfff700 1 -- 172.21.15.31:0/1173040765 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f739c027840 con 0x7f73a414fc70 2024-03-30T01:43:02.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.156+0000 7f738bfff700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f739006ac90 0x7f739006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.156+0000 7f738bfff700 1 -- 172.21.15.31:0/1173040765 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f739c022070 con 0x7f73a414fc70 2024-03-30T01:43:02.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.156+0000 7f73a37fe700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f739006ac90 0x7f739006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.156+0000 7f73a37fe700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f739006ac90 0x7f739006d150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f73940011e0 tx=0x7f7394009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:02.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.160+0000 7f738bfff700 1 -- 172.21.15.31:0/1173040765 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f739c056630 con 0x7f73a414fc70 2024-03-30T01:43:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.385+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f73a41eb820 con 0x7f73a414fc70 2024-03-30T01:43:02.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.389+0000 7f738bfff700 1 -- 172.21.15.31:0/1173040765 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f739c059c50 con 0x7f73a414fc70 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f739006ac90 msgr2=0x7f739006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f739006ac90 0x7f739006d150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f73940011e0 tx=0x7f7394009040 comp rx=0 tx=0).stop 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73a414fc70 msgr2=0x7f73a40c09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73a414fc70 0x7f73a40c09c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f739c014040 tx=0x7f739c004ad0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 shutdown_connections 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73a414fc70 0x7f73a40c09c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f739006ac90 0x7f739006d150 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a406f310 0x7f73a40bcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 --2- 172.21.15.31:0/1173040765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73a40c24f0 0x7f73a414ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 >> 172.21.15.31:0/1173040765 conn(0x7f73a40c3980 msgr2=0x7f73a40c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 shutdown_connections 2024-03-30T01:43:02.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.392+0000 7f73a9cc8700 1 -- 172.21.15.31:0/1173040765 wait complete. 2024-03-30T01:43:02.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent' 2024-03-30T01:43:02.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.542+0000 7fc28155e700 1 Processor -- start 2024-03-30T01:43:02.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.542+0000 7fc28155e700 1 -- start start 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.543+0000 7fc28155e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc27c0722f0 0x7fc27c156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.543+0000 7fc28155e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27c071ee0 con 0x7fc27c14fa40 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.543+0000 7fc28155e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27c06d5c0 con 0x7fc27c0ab100 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.543+0000 7fc28155e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27c152570 con 0x7fc27c0722f0 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.543+0000 7fc27affd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc27c14fa40 0x7fc27c0c3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33386/0 (socket says 172.21.15.31:33386) 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.543+0000 7fc27affd700 1 -- 172.21.15.31:0/3016111752 learned_addr learned my addr 172.21.15.31:0/3016111752 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.543+0000 7fc27b7fe700 1 --2- 172.21.15.31:0/3016111752 >> v2:172.21.15.59:3300/0 conn(0x7fc27c0722f0 0x7fc27c156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc279ffb700 1 -- 172.21.15.31:0/3016111752 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597081039 0 0) 0x7fc27c071ee0 con 0x7fc27c14fa40 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc279ffb700 1 -- 172.21.15.31:0/3016111752 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2680035b0 con 0x7fc27c14fa40 2024-03-30T01:43:02.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc279ffb700 1 -- 172.21.15.31:0/3016111752 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279238794 0 0) 0x7fc27c06d5c0 con 0x7fc27c0ab100 2024-03-30T01:43:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc279ffb700 1 -- 172.21.15.31:0/3016111752 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc27c071ee0 con 0x7fc27c0ab100 2024-03-30T01:43:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc27b7fe700 1 -- 172.21.15.31:0/3016111752 >> v1:172.21.15.59:6789/0 conn(0x7fc27c0ab100 legacy=0x7fc27c152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc279ffb700 1 -- 172.21.15.31:0/3016111752 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3692941593 0 0) 0x7fc2680035b0 con 0x7fc27c14fa40 2024-03-30T01:43:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc27b7fe700 1 -- 172.21.15.31:0/3016111752 >> v1:172.21.15.31:6789/0 conn(0x7fc27c14fa40 legacy=0x7fc27c0c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc27b7fe700 1 -- 172.21.15.31:0/3016111752 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c06d860 con 0x7fc27c0722f0 2024-03-30T01:43:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc27b7fe700 1 --2- 172.21.15.31:0/3016111752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc27c0722f0 0x7fc27c156570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc274009d90 tx=0x7fc27400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d1bdd450c70d6e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc279ffb700 1 -- 172.21.15.31:0/3016111752 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400e070 con 0x7fc27c0722f0 2024-03-30T01:43:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc279ffb700 1 -- 172.21.15.31:0/3016111752 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc274003a20 con 0x7fc27c0722f0 2024-03-30T01:43:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.544+0000 7fc279ffb700 1 -- 172.21.15.31:0/3016111752 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264001670 con 0x7fc27c0722f0 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.545+0000 7fc28155e700 1 -- 172.21.15.31:0/3016111752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc27c0722f0 msgr2=0x7fc27c156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.545+0000 7fc28155e700 1 --2- 172.21.15.31:0/3016111752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc27c0722f0 0x7fc27c156570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc274009d90 tx=0x7fc27400b330 comp rx=0 tx=0).stop 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.545+0000 7fc28155e700 1 -- 172.21.15.31:0/3016111752 shutdown_connections 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.545+0000 7fc28155e700 1 --2- 172.21.15.31:0/3016111752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc27c0722f0 0x7fc27c156570 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.545+0000 7fc28155e700 1 -- 172.21.15.31:0/3016111752 >> 172.21.15.31:0/3016111752 conn(0x7fc27c06ba20 msgr2=0x7fc27c06be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.545+0000 7fc28155e700 1 -- 172.21.15.31:0/3016111752 shutdown_connections 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.545+0000 7fc28155e700 1 -- 172.21.15.31:0/3016111752 wait complete. 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.546+0000 7fc28155e700 1 Processor -- start 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc28155e700 1 -- start start 2024-03-30T01:43:02.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc28155e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27c0722f0 0x7fc27c0bc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc28155e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc27c0ab100 0x7fc27c0bd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc28155e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc27c14fa40 0x7fc27c0c0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc28155e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c154f30 con 0x7fc27c0722f0 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc28155e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0bc580 con 0x7fc27c14fa40 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc28155e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c155180 con 0x7fc27c0ab100 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc27affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27c0722f0 0x7fc27c0bc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc27affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27c0722f0 0x7fc27c0bc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51534/0 (socket says 172.21.15.31:51534) 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc27affd700 1 -- 172.21.15.31:0/3511083178 learned_addr learned my addr 172.21.15.31:0/3511083178 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc27b7fe700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc27c14fa40 0x7fc27c0c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.547+0000 7fc27a7fc700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc27c0ab100 0x7fc27c0bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc27affd700 1 -- 172.21.15.31:0/3511083178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc27c0ab100 msgr2=0x7fc27c0bd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc27affd700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc27c0ab100 0x7fc27c0bd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc27affd700 1 -- 172.21.15.31:0/3511083178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc27c14fa40 msgr2=0x7fc27c0c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc27affd700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc27c14fa40 0x7fc27c0c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc27affd700 1 -- 172.21.15.31:0/3511083178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274009a70 con 0x7fc27c0722f0 2024-03-30T01:43:02.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc27affd700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27c0722f0 0x7fc27c0bc880 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc26400c6b0 tx=0x7fc26400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:02.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc2737fe700 1 -- 172.21.15.31:0/3511083178 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264004c80 con 0x7fc27c0722f0 2024-03-30T01:43:02.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc2737fe700 1 -- 172.21.15.31:0/3511083178 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc264004de0 con 0x7fc27c0722f0 2024-03-30T01:43:02.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc2737fe700 1 -- 172.21.15.31:0/3511083178 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2640072f0 con 0x7fc27c0722f0 2024-03-30T01:43:02.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27c1e8110 con 0x7fc27c0722f0 2024-03-30T01:43:02.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.548+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c0bc060 con 0x7fc27c0722f0 2024-03-30T01:43:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.549+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27c1e8630 con 0x7fc27c0722f0 2024-03-30T01:43:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.550+0000 7fc2737fe700 1 -- 172.21.15.31:0/3511083178 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc264007450 con 0x7fc27c0722f0 2024-03-30T01:43:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.550+0000 7fc2737fe700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc25c06ac90 0x7fc25c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.550+0000 7fc2737fe700 1 -- 172.21.15.31:0/3511083178 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fc264088ee0 con 0x7fc27c0722f0 2024-03-30T01:43:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.550+0000 7fc27a7fc700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc25c06ac90 0x7fc25c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.551+0000 7fc27a7fc700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc25c06ac90 0x7fc25c06d150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0c2cb0 tx=0x7fc26c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.555+0000 7fc2737fe700 1 -- 172.21.15.31:0/3511083178 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc2640542b0 con 0x7fc27c0722f0 2024-03-30T01:43:02.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.860+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]} v 0) v1 -- 0x7fc27c06e300 con 0x7fc27c0722f0 2024-03-30T01:43:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.982+0000 7fc2737fe700 1 -- 172.21.15.31:0/3511083178 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]}]=0 v138) v1 ==== 159+0+0 (secure 0 0 0) 0x7fc264007700 con 0x7fc27c0722f0 2024-03-30T01:43:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc25c06ac90 msgr2=0x7fc25c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc25c06ac90 0x7fc25c06d150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0c2cb0 tx=0x7fc26c009040 comp rx=0 tx=0).stop 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27c0722f0 msgr2=0x7fc27c0bc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27c0722f0 0x7fc27c0bc880 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc26400c6b0 tx=0x7fc26400c9c0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 shutdown_connections 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc27c0722f0 0x7fc27c0bc880 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc25c06ac90 0x7fc25c06d150 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc27c14fa40 0x7fc27c0c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 --2- 172.21.15.31:0/3511083178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc27c0ab100 0x7fc27c0bd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 >> 172.21.15.31:0/3511083178 conn(0x7fc27c06ba20 msgr2=0x7fc27c0c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 shutdown_connections 2024-03-30T01:43:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:02.985+0000 7fc28155e700 1 -- 172.21.15.31:0/3511083178 wait complete. 2024-03-30T01:43:03.007 INFO:tasks.cephfs_test_runner:test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) ... ok 2024-03-30T01:43:03.007 INFO:tasks.cephfs_test_runner:Starting test: test_mount_mon_and_osd_caps_present_mds_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) 2024-03-30T01:43:03.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:03.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.141+0000 7f2279bf9700 1 Processor -- start 2024-03-30T01:43:03.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.141+0000 7f2279bf9700 1 -- start start 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.141+0000 7f2279bf9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2274150110 0x7f2274156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.141+0000 7f2279bf9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22740c47c0 0x7f22740ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.141+0000 7f2279bf9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740c4d60 con 0x7f227414fd30 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.141+0000 7f2279bf9700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22740c5040 con 0x7f2274150110 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.141+0000 7f2279bf9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22740c5320 con 0x7f22740c47c0 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f2272ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2274150110 0x7f2274156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f2272ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2274150110 0x7f2274156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51546/0 (socket says 172.21.15.31:51546) 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f2272ffd700 1 -- 172.21.15.31:0/999127796 learned_addr learned my addr 172.21.15.31:0/999127796 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f2273fff700 1 --2- 172.21.15.31:0/999127796 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22740c47c0 0x7f22740ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f2272ffd700 1 -- 172.21.15.31:0/999127796 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22740c47c0 msgr2=0x7f22740ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:03.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f2272ffd700 1 --2- 172.21.15.31:0/999127796 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22740c47c0 0x7f22740ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f2272ffd700 1 -- 172.21.15.31:0/999127796 >> v1:172.21.15.59:6789/0 conn(0x7f227414fd30 legacy=0x7f22740c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f2272ffd700 1 -- 172.21.15.31:0/999127796 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22740c5600 con 0x7f2274150110 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f2272ffd700 1 --2- 172.21.15.31:0/999127796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2274150110 0x7f2274156800 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f22680098f0 tx=0x7f2268009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be598cf1d9056465 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f22727fc700 1 -- 172.21.15.31:0/999127796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268004230 con 0x7f2274150110 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f22727fc700 1 -- 172.21.15.31:0/999127796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268004390 con 0x7f2274150110 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.142+0000 7f22727fc700 1 -- 172.21.15.31:0/999127796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22680036a0 con 0x7f2274150110 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.143+0000 7f2279bf9700 1 -- 172.21.15.31:0/999127796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2274150110 msgr2=0x7f2274156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.143+0000 7f2279bf9700 1 --2- 172.21.15.31:0/999127796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2274150110 0x7f2274156800 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f22680098f0 tx=0x7f2268009c00 comp rx=0 tx=0).stop 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.143+0000 7f2279bf9700 1 -- 172.21.15.31:0/999127796 shutdown_connections 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.143+0000 7f2279bf9700 1 --2- 172.21.15.31:0/999127796 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22740c47c0 0x7f22740ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.143+0000 7f2279bf9700 1 --2- 172.21.15.31:0/999127796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2274150110 0x7f2274156800 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.143+0000 7f2279bf9700 1 -- 172.21.15.31:0/999127796 >> 172.21.15.31:0/999127796 conn(0x7f227406ab20 msgr2=0x7f227406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:03.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.143+0000 7f2279bf9700 1 -- 172.21.15.31:0/999127796 shutdown_connections 2024-03-30T01:43:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.143+0000 7f2279bf9700 1 -- 172.21.15.31:0/999127796 wait complete. 2024-03-30T01:43:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.144+0000 7f2279bf9700 1 Processor -- start 2024-03-30T01:43:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2279bf9700 1 -- start start 2024-03-30T01:43:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2279bf9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22740c47c0 0x7f22741533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2279bf9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f227414fd30 0x7f2274154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2279bf9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2274150110 0x7f22741e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2279bf9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741dc1d0 con 0x7f227414fd30 2024-03-30T01:43:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2279bf9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741dc4b0 con 0x7f22740c47c0 2024-03-30T01:43:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2279bf9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22741dc790 con 0x7f2274150110 2024-03-30T01:43:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2272ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f227414fd30 0x7f2274154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2272ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f227414fd30 0x7f2274154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51558/0 (socket says 172.21.15.31:51558) 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.145+0000 7f2272ffd700 1 -- 172.21.15.31:0/2077172275 learned_addr learned my addr 172.21.15.31:0/2077172275 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f22737fe700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22740c47c0 0x7f22741533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2273fff700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2274150110 0x7f22741e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2272ffd700 1 -- 172.21.15.31:0/2077172275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2274150110 msgr2=0x7f22741e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2272ffd700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2274150110 0x7f22741e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2272ffd700 1 -- 172.21.15.31:0/2077172275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22740c47c0 msgr2=0x7f22741533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2272ffd700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22740c47c0 0x7f22741533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2272ffd700 1 -- 172.21.15.31:0/2077172275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2268009550 con 0x7f227414fd30 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2273fff700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2274150110 0x7f22741e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2272ffd700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f227414fd30 0x7f2274154350 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2268009840 tx=0x7f2268004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f225bfff700 1 -- 172.21.15.31:0/2077172275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226800b660 con 0x7f227414fd30 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f225bfff700 1 -- 172.21.15.31:0/2077172275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226800b7c0 con 0x7f227414fd30 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22741dca70 con 0x7f227414fd30 2024-03-30T01:43:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f225bfff700 1 -- 172.21.15.31:0/2077172275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226800bb30 con 0x7f227414fd30 2024-03-30T01:43:03.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.146+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2274152a40 con 0x7f227414fd30 2024-03-30T01:43:03.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.147+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2274073760 con 0x7f227414fd30 2024-03-30T01:43:03.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.148+0000 7f225bfff700 1 -- 172.21.15.31:0/2077172275 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f226800bc90 con 0x7f227414fd30 2024-03-30T01:43:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.148+0000 7f225bfff700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f226006ac90 0x7f226006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.148+0000 7f225bfff700 1 -- 172.21.15.31:0/2077172275 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f226808a900 con 0x7f227414fd30 2024-03-30T01:43:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.148+0000 7f22737fe700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f226006ac90 0x7f226006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.149+0000 7f22737fe700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f226006ac90 0x7f226006d150 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f22740c3b60 tx=0x7f2264007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:03.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.153+0000 7f225bfff700 1 -- 172.21.15.31:0/2077172275 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2268055cd0 con 0x7f227414fd30 2024-03-30T01:43:03.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.368+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22741e9030 con 0x7f227414fd30 2024-03-30T01:43:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.369+0000 7f225bfff700 1 -- 172.21.15.31:0/2077172275 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+12805 (secure 0 0 0) 0x7f22741e9030 con 0x7f227414fd30 2024-03-30T01:43:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:03.376 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":51,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:01.971052+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6836/2419893471":"2024-03-31T01:42:58.955270+0000","172.21.15.31:6834/2419893471":"2024-03-31T01:42:58.955270+0000","172.21.15.31:6835/1616247357":"2024-03-31T01:42:51.698949+0000","172.21.15.31:6837/1616247357":"2024-03-31T01:42:51.698949+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f226006ac90 msgr2=0x7f226006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f226006ac90 0x7f226006d150 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f22740c3b60 tx=0x7f2264007ec0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f227414fd30 msgr2=0x7f2274154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f227414fd30 0x7f2274154350 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2268009840 tx=0x7f2268004750 comp rx=0 tx=0).stop 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 shutdown_connections 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f226006ac90 0x7f226006d150 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2274150110 0x7f22741e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22740c47c0 0x7f22741533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 --2- 172.21.15.31:0/2077172275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f227414fd30 0x7f2274154350 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 >> 172.21.15.31:0/2077172275 conn(0x7f227406ab20 msgr2=0x7f22740c8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 shutdown_connections 2024-03-30T01:43:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.371+0000 7f2279bf9700 1 -- 172.21.15.31:0/2077172275 wait complete. 2024-03-30T01:43:03.392 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:43:03.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:43:03.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.524+0000 7f7ff647e700 1 Processor -- start 2024-03-30T01:43:03.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.524+0000 7f7ff647e700 1 -- start start 2024-03-30T01:43:03.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7ff647e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7ff00a7600 0x7f7ff0152f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7ff647e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ff014ff20 0x7f7ff00c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7ff647e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff00c4620 con 0x7f7ff014fb40 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7ff647e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff0152570 con 0x7f7ff014ff20 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7ff647e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff0152ad0 con 0x7f7ff00a7600 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7ff4a1b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ff014ff20 0x7f7ff00c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7ff4a1b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ff014ff20 0x7f7ff00c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51572/0 (socket says 172.21.15.31:51572) 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7ff4a1b700 1 -- 172.21.15.31:0/1923606199 learned_addr learned my addr 172.21.15.31:0/1923606199 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7feffff700 1 --1- 172.21.15.31:0/1923606199 >> v1:172.21.15.59:6789/0 conn(0x7f7ff014fb40 0x7f7ff00c2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:50926/0 (socket says 172.21.15.31:50926) 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.525+0000 7f7fef7fe700 1 --2- 172.21.15.31:0/1923606199 >> v2:172.21.15.59:3300/0 conn(0x7f7ff00a7600 0x7f7ff0152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.526+0000 7f7ff4a1b700 1 -- 172.21.15.31:0/1923606199 >> v2:172.21.15.59:3300/0 conn(0x7f7ff00a7600 msgr2=0x7f7ff0152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.526+0000 7f7ff4a1b700 1 --2- 172.21.15.31:0/1923606199 >> v2:172.21.15.59:3300/0 conn(0x7f7ff00a7600 0x7f7ff0152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.526+0000 7f7ff4a1b700 1 -- 172.21.15.31:0/1923606199 >> v1:172.21.15.59:6789/0 conn(0x7f7ff014fb40 legacy=0x7f7ff00c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.526+0000 7f7ff4a1b700 1 -- 172.21.15.31:0/1923606199 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff01534a0 con 0x7f7ff014ff20 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.526+0000 7f7ff4a1b700 1 --2- 172.21.15.31:0/1923606199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014ff20 0x7f7ff00c47f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00ab010 tx=0x7f7fe800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca896f35dfa627da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:03.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.526+0000 7f7feeffd700 1 -- 172.21.15.31:0/1923606199 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800e070 con 0x7f7ff014ff20 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.526+0000 7f7feeffd700 1 -- 172.21.15.31:0/1923606199 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe800b910 con 0x7f7ff014ff20 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.526+0000 7f7feeffd700 1 -- 172.21.15.31:0/1923606199 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800bcc0 con 0x7f7ff014ff20 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.527+0000 7f7ff647e700 1 -- 172.21.15.31:0/1923606199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014ff20 msgr2=0x7f7ff00c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.527+0000 7f7ff647e700 1 --2- 172.21.15.31:0/1923606199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014ff20 0x7f7ff00c47f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00ab010 tx=0x7f7fe800b330 comp rx=0 tx=0).stop 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.527+0000 7f7ff647e700 1 -- 172.21.15.31:0/1923606199 shutdown_connections 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.527+0000 7f7ff647e700 1 --2- 172.21.15.31:0/1923606199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014ff20 0x7f7ff00c47f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.527+0000 7f7ff647e700 1 --2- 172.21.15.31:0/1923606199 >> v2:172.21.15.59:3300/0 conn(0x7f7ff00a7600 0x7f7ff0152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.527+0000 7f7ff647e700 1 -- 172.21.15.31:0/1923606199 >> 172.21.15.31:0/1923606199 conn(0x7f7ff00c7d60 msgr2=0x7f7ff00c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.527+0000 7f7ff647e700 1 -- 172.21.15.31:0/1923606199 shutdown_connections 2024-03-30T01:43:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.527+0000 7f7ff647e700 1 -- 172.21.15.31:0/1923606199 wait complete. 2024-03-30T01:43:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.528+0000 7f7ff647e700 1 Processor -- start 2024-03-30T01:43:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.528+0000 7f7ff647e700 1 -- start start 2024-03-30T01:43:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7ff647e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ff00a7600 0x7f7ff00c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7ff647e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014fb40 0x7f7ff0099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7ff647e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ff014ff20 0x7f7ff01e4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7ff647e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff009a780 con 0x7f7ff014fb40 2024-03-30T01:43:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7ff647e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff01e85a0 con 0x7f7ff00a7600 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7ff647e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff01e77e0 con 0x7f7ff014ff20 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014fb40 0x7f7ff0099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014fb40 0x7f7ff0099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51582/0 (socket says 172.21.15.31:51582) 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7fef7fe700 1 -- 172.21.15.31:0/1084355197 learned_addr learned my addr 172.21.15.31:0/1084355197 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7feffff700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ff00a7600 0x7f7ff00c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7ff4a1b700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ff014ff20 0x7f7ff01e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7fef7fe700 1 -- 172.21.15.31:0/1084355197 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ff014ff20 msgr2=0x7f7ff01e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7fef7fe700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ff014ff20 0x7f7ff01e4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7fef7fe700 1 -- 172.21.15.31:0/1084355197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ff00a7600 msgr2=0x7f7ff00c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7fef7fe700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ff00a7600 0x7f7ff00c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.529+0000 7f7fef7fe700 1 -- 172.21.15.31:0/1084355197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8009b30 con 0x7f7ff014fb40 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.530+0000 7f7fef7fe700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014fb40 0x7f7ff0099a00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e870 tx=0x7f7fe400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:03.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.530+0000 7f7fecff9700 1 -- 172.21.15.31:0/1084355197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4009590 con 0x7f7ff014fb40 2024-03-30T01:43:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.530+0000 7f7fecff9700 1 -- 172.21.15.31:0/1084355197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe400ee70 con 0x7f7ff014fb40 2024-03-30T01:43:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.530+0000 7f7fecff9700 1 -- 172.21.15.31:0/1084355197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400ca50 con 0x7f7ff014fb40 2024-03-30T01:43:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.530+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff01e7ac0 con 0x7f7ff014fb40 2024-03-30T01:43:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.530+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff01e8080 con 0x7f7ff014fb40 2024-03-30T01:43:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.532+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff00747d0 con 0x7f7ff014fb40 2024-03-30T01:43:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.532+0000 7f7fecff9700 1 -- 172.21.15.31:0/1084355197 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7fe400cbb0 con 0x7f7ff014fb40 2024-03-30T01:43:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.532+0000 7f7fecff9700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fdc06ac90 0x7f7fdc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.532+0000 7f7fecff9700 1 -- 172.21.15.31:0/1084355197 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f7fe40890f0 con 0x7f7ff014fb40 2024-03-30T01:43:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.532+0000 7f7feffff700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fdc06ac90 0x7f7fdc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.533+0000 7f7feffff700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fdc06ac90 0x7f7fdc06d150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0000f60 tx=0x7f7fe0007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:03.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.537+0000 7f7fecff9700 1 -- 172.21.15.31:0/1084355197 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7fe40544c0 con 0x7f7ff014fb40 2024-03-30T01:43:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.747+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ff009a330 con 0x7f7ff014fb40 2024-03-30T01:43:03.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.748+0000 7f7fecff9700 1 -- 172.21.15.31:0/1084355197 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fe400ce60 con 0x7f7ff014fb40 2024-03-30T01:43:03.754 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:43:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fdc06ac90 msgr2=0x7f7fdc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fdc06ac90 0x7f7fdc06d150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0000f60 tx=0x7f7fe0007d10 comp rx=0 tx=0).stop 2024-03-30T01:43:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014fb40 msgr2=0x7f7ff0099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014fb40 0x7f7ff0099a00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e870 tx=0x7f7fe400ec30 comp rx=0 tx=0).stop 2024-03-30T01:43:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 shutdown_connections 2024-03-30T01:43:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f7fdc06ac90 0x7f7fdc06d150 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ff014ff20 0x7f7ff01e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ff00a7600 0x7f7ff00c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 --2- 172.21.15.31:0/1084355197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ff014fb40 0x7f7ff0099a00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.750+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 >> 172.21.15.31:0/1084355197 conn(0x7f7ff00c7d60 msgr2=0x7f7ff00684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.751+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 shutdown_connections 2024-03-30T01:43:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.751+0000 7f7ff647e700 1 -- 172.21.15.31:0/1084355197 wait complete. 2024-03-30T01:43:03.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent' 2024-03-30T01:43:03.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.901+0000 7f8d97b05700 1 Processor -- start 2024-03-30T01:43:03.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.901+0000 7f8d97b05700 1 -- start start 2024-03-30T01:43:03.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d97b05700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8d90152100 0x7f8d900c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d97b05700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d900c5dc0 0x7f8d900c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d97b05700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d90070d60 con 0x7f8d90151d20 2024-03-30T01:43:03.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d97b05700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d90157e90 con 0x7f8d90152100 2024-03-30T01:43:03.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d97b05700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d90158170 con 0x7f8d900c5dc0 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d958a1700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8d90151d20 0x7f8d900705a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33396/0 (socket says 172.21.15.31:33396) 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d958a1700 1 -- 172.21.15.31:0/1451785362 learned_addr learned my addr 172.21.15.31:0/1451785362 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d950a0700 1 --2- 172.21.15.31:0/1451785362 >> v2:172.21.15.59:3300/0 conn(0x7f8d90152100 0x7f8d900c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d960a2700 1 --2- 172.21.15.31:0/1451785362 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d900c5dc0 0x7f8d900c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d9489f700 1 -- 172.21.15.31:0/1451785362 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323101783 0 0) 0x7f8d90070d60 con 0x7f8d90151d20 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d9489f700 1 -- 172.21.15.31:0/1451785362 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d7c0035b0 con 0x7f8d90151d20 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d950a0700 1 -- 172.21.15.31:0/1451785362 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d900c5dc0 msgr2=0x7f8d900c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.902+0000 7f8d950a0700 1 --2- 172.21.15.31:0/1451785362 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d900c5dc0 0x7f8d900c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.903+0000 7f8d950a0700 1 -- 172.21.15.31:0/1451785362 >> v1:172.21.15.31:6789/0 conn(0x7f8d90151d20 legacy=0x7f8d900705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.903+0000 7f8d950a0700 1 -- 172.21.15.31:0/1451785362 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90158450 con 0x7f8d90152100 2024-03-30T01:43:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.903+0000 7f8d950a0700 1 --2- 172.21.15.31:0/1451785362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 0x7f8d900c4850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009870 tx=0x7f8d80009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2382442decc1c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.903+0000 7f8d9489f700 1 -- 172.21.15.31:0/1451785362 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80011830 con 0x7f8d90152100 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.903+0000 7f8d9489f700 1 -- 172.21.15.31:0/1451785362 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d80011990 con 0x7f8d90152100 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.903+0000 7f8d9489f700 1 -- 172.21.15.31:0/1451785362 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80011d20 con 0x7f8d90152100 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.903+0000 7f8d97b05700 1 -- 172.21.15.31:0/1451785362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 msgr2=0x7f8d900c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.904+0000 7f8d97b05700 1 --2- 172.21.15.31:0/1451785362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 0x7f8d900c4850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009870 tx=0x7f8d80009b80 comp rx=0 tx=0).stop 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.904+0000 7f8d97b05700 1 -- 172.21.15.31:0/1451785362 shutdown_connections 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.904+0000 7f8d97b05700 1 --2- 172.21.15.31:0/1451785362 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d900c5dc0 0x7f8d900c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.904+0000 7f8d97b05700 1 --2- 172.21.15.31:0/1451785362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 0x7f8d900c4850 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.904+0000 7f8d97b05700 1 -- 172.21.15.31:0/1451785362 >> 172.21.15.31:0/1451785362 conn(0x7f8d9006ad30 msgr2=0x7f8d900a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.904+0000 7f8d97b05700 1 -- 172.21.15.31:0/1451785362 shutdown_connections 2024-03-30T01:43:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.904+0000 7f8d97b05700 1 -- 172.21.15.31:0/1451785362 wait complete. 2024-03-30T01:43:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.905+0000 7f8d97b05700 1 Processor -- start 2024-03-30T01:43:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.905+0000 7f8d97b05700 1 -- start start 2024-03-30T01:43:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d97b05700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d900c5dc0 0x7f8d9006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d97b05700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d90151d20 0x7f8d90150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d97b05700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 0x7f8d90156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d97b05700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d901eab80 con 0x7f8d900c5dc0 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d97b05700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9006f4b0 con 0x7f8d90152100 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d97b05700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d901ea600 con 0x7f8d90151d20 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d958a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d900c5dc0 0x7f8d9006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d958a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d900c5dc0 0x7f8d9006fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51586/0 (socket says 172.21.15.31:51586) 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d958a1700 1 -- 172.21.15.31:0/4081044438 learned_addr learned my addr 172.21.15.31:0/4081044438 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d960a2700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 0x7f8d90156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d950a0700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d90151d20 0x7f8d90150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d958a1700 1 -- 172.21.15.31:0/4081044438 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d90151d20 msgr2=0x7f8d90150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d958a1700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d90151d20 0x7f8d90150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d958a1700 1 -- 172.21.15.31:0/4081044438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 msgr2=0x7f8d90156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d958a1700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 0x7f8d90156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d958a1700 1 -- 172.21.15.31:0/4081044438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d800094a0 con 0x7f8d900c5dc0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d960a2700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 0x7f8d90156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d958a1700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d900c5dc0 0x7f8d9006fe30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c730 tx=0x7f8d8c00caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d867fc700 1 -- 172.21.15.31:0/4081044438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c005d70 con 0x7f8d900c5dc0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d867fc700 1 -- 172.21.15.31:0/4081044438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c0070d0 con 0x7f8d900c5dc0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.906+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d901ea8a0 con 0x7f8d900c5dc0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.907+0000 7f8d867fc700 1 -- 172.21.15.31:0/4081044438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c007440 con 0x7f8d900c5dc0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.907+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d90150dd0 con 0x7f8d900c5dc0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.908+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9004ea80 con 0x7f8d900c5dc0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.908+0000 7f8d867fc700 1 -- 172.21.15.31:0/4081044438 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d8c0075a0 con 0x7f8d900c5dc0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.909+0000 7f8d867fc700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d7406ac90 0x7f8d7406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.909+0000 7f8d867fc700 1 -- 172.21.15.31:0/4081044438 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f8d8c0885e0 con 0x7f8d900c5dc0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.909+0000 7f8d950a0700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d7406ac90 0x7f8d7406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.909+0000 7f8d950a0700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d7406ac90 0x7f8d7406d150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d900aa630 tx=0x7f8d80008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:03.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:03.913+0000 7f8d867fc700 1 -- 172.21.15.31:0/4081044438 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d8c053a60 con 0x7f8d900c5dc0 2024-03-30T01:43:04.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.221+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]} v 0) v1 -- 0x7f8d900c31c0 con 0x7f8d900c5dc0 2024-03-30T01:43:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.989+0000 7f8d867fc700 1 -- 172.21.15.31:0/4081044438 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]}]=0 v140) v1 ==== 187+0+0 (secure 0 0 0) 0x7f8d8c057080 con 0x7f8d900c5dc0 2024-03-30T01:43:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.991+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d7406ac90 msgr2=0x7f8d7406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.991+0000 7f8d97b05700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d7406ac90 0x7f8d7406d150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d900aa630 tx=0x7f8d80008030 comp rx=0 tx=0).stop 2024-03-30T01:43:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.991+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d900c5dc0 msgr2=0x7f8d9006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.992+0000 7f8d97b05700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d900c5dc0 0x7f8d9006fe30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c730 tx=0x7f8d8c00caf0 comp rx=0 tx=0).stop 2024-03-30T01:43:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.992+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 shutdown_connections 2024-03-30T01:43:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.992+0000 7f8d97b05700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d900c5dc0 0x7f8d9006fe30 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.992+0000 7f8d97b05700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d7406ac90 0x7f8d7406d150 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.992+0000 7f8d97b05700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d90152100 0x7f8d90156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.992+0000 7f8d97b05700 1 --2- 172.21.15.31:0/4081044438 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d90151d20 0x7f8d90150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.992+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 >> 172.21.15.31:0/4081044438 conn(0x7f8d9006ad30 msgr2=0x7f8d900c6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.992+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 shutdown_connections 2024-03-30T01:43:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:04.992+0000 7f8d97b05700 1 -- 172.21.15.31:0/4081044438 wait complete. 2024-03-30T01:43:05.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T01:43:05.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.150+0000 7f21c99de700 1 Processor -- start 2024-03-30T01:43:05.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.151+0000 7f21c99de700 1 -- start start 2024-03-30T01:43:05.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.151+0000 7f21c99de700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21c40bc1e0 0x7f21c406c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.151+0000 7f21c99de700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21c4072a30 0x7f21c40c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.151+0000 7f21c99de700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c40c4080 con 0x7f21c406c550 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.151+0000 7f21c99de700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c40c3d20 con 0x7f21c4072a30 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.151+0000 7f21c99de700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c40bc5c0 con 0x7f21c40bc1e0 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c27fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21c4072a30 0x7f21c40c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c27fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21c4072a30 0x7f21c40c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51602/0 (socket says 172.21.15.31:51602) 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c27fc700 1 -- 172.21.15.31:0/3913375073 learned_addr learned my addr 172.21.15.31:0/3913375073 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c2ffd700 1 --2- 172.21.15.31:0/3913375073 >> v2:172.21.15.59:3300/0 conn(0x7f21c40bc1e0 0x7f21c406c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c27fc700 1 -- 172.21.15.31:0/3913375073 >> v2:172.21.15.59:3300/0 conn(0x7f21c40bc1e0 msgr2=0x7f21c406c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c27fc700 1 --2- 172.21.15.31:0/3913375073 >> v2:172.21.15.59:3300/0 conn(0x7f21c40bc1e0 0x7f21c406c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c27fc700 1 -- 172.21.15.31:0/3913375073 >> v1:172.21.15.59:6789/0 conn(0x7f21c406c550 legacy=0x7f21c40c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c27fc700 1 -- 172.21.15.31:0/3913375073 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c4154460 con 0x7f21c4072a30 2024-03-30T01:43:05.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c27fc700 1 --2- 172.21.15.31:0/3913375073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c4072a30 0x7f21c40c4ad0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f21c406e3f0 tx=0x7f21b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642eb08d59d8dec3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c1ffb700 1 -- 172.21.15.31:0/3913375073 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8004230 con 0x7f21c4072a30 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c1ffb700 1 -- 172.21.15.31:0/3913375073 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8004390 con 0x7f21c4072a30 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.152+0000 7f21c1ffb700 1 -- 172.21.15.31:0/3913375073 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80036a0 con 0x7f21c4072a30 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.153+0000 7f21c99de700 1 -- 172.21.15.31:0/3913375073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c4072a30 msgr2=0x7f21c40c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.153+0000 7f21c99de700 1 --2- 172.21.15.31:0/3913375073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c4072a30 0x7f21c40c4ad0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f21c406e3f0 tx=0x7f21b8009bd0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.153+0000 7f21c99de700 1 -- 172.21.15.31:0/3913375073 shutdown_connections 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.153+0000 7f21c99de700 1 --2- 172.21.15.31:0/3913375073 >> v2:172.21.15.59:3300/0 conn(0x7f21c40bc1e0 0x7f21c406c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.153+0000 7f21c99de700 1 --2- 172.21.15.31:0/3913375073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c4072a30 0x7f21c40c4ad0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.153+0000 7f21c99de700 1 -- 172.21.15.31:0/3913375073 >> 172.21.15.31:0/3913375073 conn(0x7f21c406b110 msgr2=0x7f21c40a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:05.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.153+0000 7f21c99de700 1 -- 172.21.15.31:0/3913375073 shutdown_connections 2024-03-30T01:43:05.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.153+0000 7f21c99de700 1 -- 172.21.15.31:0/3913375073 wait complete. 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.154+0000 7f21c99de700 1 Processor -- start 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.154+0000 7f21c99de700 1 -- start start 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c99de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c406c550 0x7f21c40712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c99de700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21c4072a30 0x7f21c4142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c99de700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21c40bc1e0 0x7f21c4143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c99de700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4154230 con 0x7f21c406c550 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c99de700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41e9ec0 con 0x7f21c40bc1e0 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c99de700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c41539b0 con 0x7f21c4072a30 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c406c550 0x7f21c40712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c406c550 0x7f21c40712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51618/0 (socket says 172.21.15.31:51618) 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c2ffd700 1 -- 172.21.15.31:0/1749107230 learned_addr learned my addr 172.21.15.31:0/1749107230 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c37fe700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21c40bc1e0 0x7f21c4143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.155+0000 7f21c27fc700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21c4072a30 0x7f21c4142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21c2ffd700 1 -- 172.21.15.31:0/1749107230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21c4072a30 msgr2=0x7f21c4142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21c2ffd700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21c4072a30 0x7f21c4142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21c2ffd700 1 -- 172.21.15.31:0/1749107230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21c40bc1e0 msgr2=0x7f21c4143260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21c2ffd700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21c40bc1e0 0x7f21c4143260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21c2ffd700 1 -- 172.21.15.31:0/1749107230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b8009550 con 0x7f21c406c550 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21c2ffd700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c406c550 0x7f21c40712c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f21b400c820 tx=0x7f21b400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21abfff700 1 -- 172.21.15.31:0/1749107230 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4004220 con 0x7f21c406c550 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21abfff700 1 -- 172.21.15.31:0/1749107230 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b4004380 con 0x7f21c406c550 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c40bf130 con 0x7f21c406c550 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21abfff700 1 -- 172.21.15.31:0/1749107230 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b40036a0 con 0x7f21c406c550 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.156+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c41ea9c0 con 0x7f21c406c550 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.157+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c41eaca0 con 0x7f21c406c550 2024-03-30T01:43:05.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.158+0000 7f21abfff700 1 -- 172.21.15.31:0/1749107230 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f21b4003800 con 0x7f21c406c550 2024-03-30T01:43:05.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.158+0000 7f21abfff700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f21b006ac90 0x7f21b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.158+0000 7f21abfff700 1 -- 172.21.15.31:0/1749107230 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f21b4088d60 con 0x7f21c406c550 2024-03-30T01:43:05.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.158+0000 7f21c27fc700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f21b006ac90 0x7f21b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.159+0000 7f21c27fc700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f21b006ac90 0x7f21b006d150 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009130 tx=0x7f21b8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:05.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.163+0000 7f21abfff700 1 -- 172.21.15.31:0/1749107230 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f21b4054010 con 0x7f21c406c550 2024-03-30T01:43:05.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.379+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f21c40747d0 con 0x7f21c406c550 2024-03-30T01:43:05.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.385+0000 7f21abfff700 1 -- 172.21.15.31:0/1749107230 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f21b401e070 con 0x7f21c406c550 2024-03-30T01:43:05.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f21b006ac90 msgr2=0x7f21b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f21b006ac90 0x7f21b006d150 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009130 tx=0x7f21b8008030 comp rx=0 tx=0).stop 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c406c550 msgr2=0x7f21c40712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c406c550 0x7f21c40712c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f21b400c820 tx=0x7f21b400cb30 comp rx=0 tx=0).stop 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 shutdown_connections 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21c406c550 0x7f21c40712c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f21b006ac90 0x7f21b006d150 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21c40bc1e0 0x7f21c4143260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 --2- 172.21.15.31:0/1749107230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21c4072a30 0x7f21c4142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 >> 172.21.15.31:0/1749107230 conn(0x7f21c406b110 msgr2=0x7f21c40a4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 shutdown_connections 2024-03-30T01:43:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.388+0000 7f21c99de700 1 -- 172.21.15.31:0/1749107230 wait complete. 2024-03-30T01:43:05.408 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T01:43:05.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T01:43:05.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T01:43:05.500 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T01:43:05.501 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:43:05.501 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:43:05.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247268k,nr_inodes=4061817,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19833 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253136k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-30T01:43:05.522 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:43:05.522 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:43:05.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247272k,nr_inodes=4061818,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1655 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-30T01:43:05.545 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-30T01:43:05.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:43:05.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.699+0000 7f0feac31700 1 Processor -- start 2024-03-30T01:43:05.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.699+0000 7f0feac31700 1 -- start start 2024-03-30T01:43:05.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0feac31700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0fe4071240 0x7f0fe4154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0feac31700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 0x7f0fe4154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0feac31700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe4072160 con 0x7f0fe414e5f0 2024-03-30T01:43:05.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0feac31700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40c2bd0 con 0x7f0fe4071240 2024-03-30T01:43:05.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0feac31700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40c2e90 con 0x7f0fe4071750 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe91ce700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 0x7f0fe4154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe3fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0fe4071240 0x7f0fe4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe89cd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0fe414e5f0 0x7f0fe40c3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:51102/0 (socket says 172.21.15.31:51102) 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe91ce700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 0x7f0fe4154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:60994/0 (socket says 172.21.15.31:60994) 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe89cd700 1 -- 172.21.15.31:0/1424810754 learned_addr learned my addr 172.21.15.31:0/1424810754 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe3fff700 1 -- 172.21.15.31:0/1424810754 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 msgr2=0x7f0fe4154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe3fff700 1 --2- 172.21.15.31:0/1424810754 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 0x7f0fe4154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe3fff700 1 -- 172.21.15.31:0/1424810754 >> v1:172.21.15.59:6789/0 conn(0x7f0fe414e5f0 legacy=0x7f0fe40c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe3fff700 1 -- 172.21.15.31:0/1424810754 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40723b0 con 0x7f0fe4071240 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.700+0000 7f0fe37fe700 1 -- 172.21.15.31:0/1424810754 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047359413 0 0) 0x7f0fe4072160 con 0x7f0fe414e5f0 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0fe3fff700 1 --2- 172.21.15.31:0/1424810754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fe4071240 0x7f0fe4154890 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009840 tx=0x7f0fd0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=137c2d902d363143 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0fe37fe700 1 -- 172.21.15.31:0/1424810754 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0011830 con 0x7f0fe4071240 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0fe37fe700 1 -- 172.21.15.31:0/1424810754 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0011990 con 0x7f0fe4071240 2024-03-30T01:43:05.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0fe37fe700 1 -- 172.21.15.31:0/1424810754 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0011d20 con 0x7f0fe4071240 2024-03-30T01:43:05.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0feac31700 1 -- 172.21.15.31:0/1424810754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fe4071240 msgr2=0x7f0fe4154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0feac31700 1 --2- 172.21.15.31:0/1424810754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fe4071240 0x7f0fe4154890 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009840 tx=0x7f0fd0009b50 comp rx=0 tx=0).stop 2024-03-30T01:43:05.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0feac31700 1 -- 172.21.15.31:0/1424810754 shutdown_connections 2024-03-30T01:43:05.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0feac31700 1 --2- 172.21.15.31:0/1424810754 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 0x7f0fe4154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0feac31700 1 --2- 172.21.15.31:0/1424810754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fe4071240 0x7f0fe4154890 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.701+0000 7f0feac31700 1 -- 172.21.15.31:0/1424810754 >> 172.21.15.31:0/1424810754 conn(0x7f0fe406ba20 msgr2=0x7f0fe406be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:05.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.702+0000 7f0feac31700 1 -- 172.21.15.31:0/1424810754 shutdown_connections 2024-03-30T01:43:05.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.702+0000 7f0feac31700 1 -- 172.21.15.31:0/1424810754 wait complete. 2024-03-30T01:43:05.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.703+0000 7f0feac31700 1 Processor -- start 2024-03-30T01:43:05.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.703+0000 7f0feac31700 1 -- start start 2024-03-30T01:43:05.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0feac31700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe4071240 0x7f0fe414af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0feac31700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 0x7f0fe41ed100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0feac31700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fe414e5f0 0x7f0fe41ed640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0feac31700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe41edb80 con 0x7f0fe4071240 2024-03-30T01:43:05.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0feac31700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe4148890 con 0x7f0fe414e5f0 2024-03-30T01:43:05.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0feac31700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe4158150 con 0x7f0fe4071750 2024-03-30T01:43:05.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe89cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe4071240 0x7f0fe414af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe89cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe4071240 0x7f0fe414af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33554/0 (socket says 172.21.15.31:33554) 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe89cd700 1 -- 172.21.15.31:0/1141300980 learned_addr learned my addr 172.21.15.31:0/1141300980 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe3fff700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 0x7f0fe41ed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe91ce700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fe414e5f0 0x7f0fe41ed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe89cd700 1 -- 172.21.15.31:0/1141300980 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 msgr2=0x7f0fe41ed100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe89cd700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 0x7f0fe41ed100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe89cd700 1 -- 172.21.15.31:0/1141300980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fe414e5f0 msgr2=0x7f0fe41ed640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe89cd700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fe414e5f0 0x7f0fe41ed640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe89cd700 1 -- 172.21.15.31:0/1141300980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00094a0 con 0x7f0fe4071240 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.704+0000 7f0fe89cd700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe4071240 0x7f0fe414af00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80011e0 tx=0x7f0fd800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.705+0000 7f0fe17fa700 1 -- 172.21.15.31:0/1141300980 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8004c80 con 0x7f0fe4071240 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.705+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe414b440 con 0x7f0fe4071240 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.705+0000 7f0fe17fa700 1 -- 172.21.15.31:0/1141300980 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd8004e00 con 0x7f0fe4071240 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.705+0000 7f0fe17fa700 1 -- 172.21.15.31:0/1141300980 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8007310 con 0x7f0fe4071240 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.705+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe4148ae0 con 0x7f0fe4071240 2024-03-30T01:43:05.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.706+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe404ea80 con 0x7f0fe4071240 2024-03-30T01:43:05.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.706+0000 7f0fe17fa700 1 -- 172.21.15.31:0/1141300980 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0fd8007470 con 0x7f0fe4071240 2024-03-30T01:43:05.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.707+0000 7f0fe17fa700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0fc806ac90 0x7f0fc806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:05.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.707+0000 7f0fe17fa700 1 -- 172.21.15.31:0/1141300980 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f0fd8088ec0 con 0x7f0fe4071240 2024-03-30T01:43:05.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.707+0000 7f0fe3fff700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0fc806ac90 0x7f0fc806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:05.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.707+0000 7f0fe3fff700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0fc806ac90 0x7f0fc806d150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000c00 tx=0x7f0fd001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:05.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.711+0000 7f0fe17fa700 1 -- 172.21.15.31:0/1141300980 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0fd8054170 con 0x7f0fe4071240 2024-03-30T01:43:05.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.974+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fe406dd00 con 0x7f0fe4071240 2024-03-30T01:43:05.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.974+0000 7f0fe17fa700 1 -- 172.21.15.31:0/1141300980 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f0fd8057790 con 0x7f0fe4071240 2024-03-30T01:43:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:05.981 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2984255103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2984255103},{"type":"v1","addr":"172.21.15.59:6834","nonce":2984255103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4865,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1099866614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1099866614},{"type":"v1","addr":"172.21.15.59:6835","nonce":1099866614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/4017049280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":4017049280},{"type":"v1","addr":"172.21.15.31:6837","nonce":4017049280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5913,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/2517312735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2517312735},{"type":"v1","addr":"172.21.15.31:6836","nonce":2517312735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0fc806ac90 msgr2=0x7f0fc806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0fc806ac90 0x7f0fc806d150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000c00 tx=0x7f0fd001c040 comp rx=0 tx=0).stop 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe4071240 msgr2=0x7f0fe414af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe4071240 0x7f0fe414af00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80011e0 tx=0x7f0fd800ca30 comp rx=0 tx=0).stop 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 shutdown_connections 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fe4071240 0x7f0fe414af00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0fc806ac90 0x7f0fc806d150 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fe414e5f0 0x7f0fe41ed640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 --2- 172.21.15.31:0/1141300980 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fe4071750 0x7f0fe41ed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 >> 172.21.15.31:0/1141300980 conn(0x7f0fe406ba20 msgr2=0x7f0fe40c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:05.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 shutdown_connections 2024-03-30T01:43:05.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:05.977+0000 7f0feac31700 1 -- 172.21.15.31:0/1141300980 wait complete. 2024-03-30T01:43:05.985 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-03-30T01:43:05.997 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T01:43:05.998 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T01:43:05.998 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T01:43:05.998 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T01:43:05.998 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T01:43:05.998 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T01:43:05.998 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T01:43:05.998 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T01:43:05.999 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T01:43:05.999 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T01:43:05.999 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T01:43:05.999 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T01:43:05.999 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T01:43:05.999 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T01:43:05.999 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T01:43:06.000 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T01:43:06.008 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T01:43:06.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T01:43:06.009 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T01:43:06.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T01:43:06.011 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T01:43:06.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T01:43:06.012 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T01:43:06.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T01:43:06.052 INFO:tasks.ceph.mds.a:Started 2024-03-30T01:43:06.052 DEBUG:teuthology.parallel:result is None 2024-03-30T01:43:06.053 INFO:tasks.ceph.mds.c:Started 2024-03-30T01:43:06.053 DEBUG:teuthology.parallel:result is None 2024-03-30T01:43:06.053 INFO:tasks.ceph.mds.b:Started 2024-03-30T01:43:06.053 DEBUG:teuthology.parallel:result is None 2024-03-30T01:43:06.054 INFO:tasks.ceph.mds.d:Started 2024-03-30T01:43:06.054 DEBUG:teuthology.parallel:result is None 2024-03-30T01:43:06.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T01:43:06.143 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2024-03-30T01:43:06.144 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2024-03-30T01:43:06.150 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2024-03-30T01:43:06.151 INFO:tasks.ceph.mds.b.smithi059.stdout:starting mds.b at 2024-03-30T01:43:06.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.196+0000 7f6f6374b700 1 Processor -- start 2024-03-30T01:43:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.196+0000 7f6f6374b700 1 -- start start 2024-03-30T01:43:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f6374b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f5c0ad010 0x7f6f5c0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f6374b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f5c0a74e0 0x7f6f5c0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f6374b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c153590 con 0x7f6f5c0ac520 2024-03-30T01:43:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f6374b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5c0ab680 con 0x7f6f5c0ad010 2024-03-30T01:43:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f6374b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5c1500c0 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f60ce6700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f5c0ad010 0x7f6f5c0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f60ce6700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f5c0ad010 0x7f6f5c0c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33586/0 (socket says 172.21.15.31:33586) 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f614e7700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6f5c0ac520 0x7f6f5c0ac900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50414/0 (socket says 172.21.15.31:50414) 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f60ce6700 1 -- 172.21.15.31:0/3490305088 learned_addr learned my addr 172.21.15.31:0/3490305088 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f61ce8700 1 --2- 172.21.15.31:0/3490305088 >> v2:172.21.15.59:3300/0 conn(0x7f6f5c0a74e0 0x7f6f5c0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f60ce6700 1 -- 172.21.15.31:0/3490305088 >> v2:172.21.15.59:3300/0 conn(0x7f6f5c0a74e0 msgr2=0x7f6f5c0c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f60ce6700 1 --2- 172.21.15.31:0/3490305088 >> v2:172.21.15.59:3300/0 conn(0x7f6f5c0a74e0 0x7f6f5c0c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f60ce6700 1 -- 172.21.15.31:0/3490305088 >> v1:172.21.15.31:6789/0 conn(0x7f6f5c0ac520 legacy=0x7f6f5c0ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f60ce6700 1 -- 172.21.15.31:0/3490305088 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c0c7990 con 0x7f6f5c0ad010 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f61ce8700 1 --2- 172.21.15.31:0/3490305088 >> v2:172.21.15.59:3300/0 conn(0x7f6f5c0a74e0 0x7f6f5c0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.197+0000 7f6f4ffff700 1 -- 172.21.15.31:0/3490305088 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007161200 0 0) 0x7f6f5c153590 con 0x7f6f5c0ac520 2024-03-30T01:43:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f60ce6700 1 --2- 172.21.15.31:0/3490305088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0ad010 0x7f6f5c0c6000 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0ac4b0 tx=0x7f6f50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13958ad12f7f294 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f4ffff700 1 -- 172.21.15.31:0/3490305088 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5000b870 con 0x7f6f5c0ad010 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f4ffff700 1 -- 172.21.15.31:0/3490305088 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5000b9d0 con 0x7f6f5c0ad010 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f4ffff700 1 -- 172.21.15.31:0/3490305088 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5000bd60 con 0x7f6f5c0ad010 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f6374b700 1 -- 172.21.15.31:0/3490305088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0ad010 msgr2=0x7f6f5c0c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f6374b700 1 --2- 172.21.15.31:0/3490305088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0ad010 0x7f6f5c0c6000 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0ac4b0 tx=0x7f6f50009bd0 comp rx=0 tx=0).stop 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f6374b700 1 -- 172.21.15.31:0/3490305088 shutdown_connections 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f6374b700 1 --2- 172.21.15.31:0/3490305088 >> v2:172.21.15.59:3300/0 conn(0x7f6f5c0a74e0 0x7f6f5c0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f6374b700 1 --2- 172.21.15.31:0/3490305088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0ad010 0x7f6f5c0c6000 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.198+0000 7f6f6374b700 1 -- 172.21.15.31:0/3490305088 >> 172.21.15.31:0/3490305088 conn(0x7f6f5c06ad30 msgr2=0x7f6f5c0a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.199+0000 7f6f6374b700 1 -- 172.21.15.31:0/3490305088 shutdown_connections 2024-03-30T01:43:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.199+0000 7f6f6374b700 1 -- 172.21.15.31:0/3490305088 wait complete. 2024-03-30T01:43:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.200+0000 7f6f6374b700 1 Processor -- start 2024-03-30T01:43:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.200+0000 7f6f6374b700 1 -- start start 2024-03-30T01:43:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f6374b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0a74e0 0x7f6f5c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f6374b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f5c0ac520 0x7f6f5c0c1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f6374b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f5c0ad010 0x7f6f5c14c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f6374b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c0c1ea0 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f6374b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c153950 con 0x7f6f5c0ac520 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f6374b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c153ed0 con 0x7f6f5c0ad010 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f614e7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0a74e0 0x7f6f5c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f614e7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0a74e0 0x7f6f5c151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33596/0 (socket says 172.21.15.31:33596) 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f614e7700 1 -- 172.21.15.31:0/1849190601 learned_addr learned my addr 172.21.15.31:0/1849190601 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f61ce8700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f5c0ad010 0x7f6f5c14c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f60ce6700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f5c0ac520 0x7f6f5c0c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f614e7700 1 -- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f5c0ad010 msgr2=0x7f6f5c14c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f614e7700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f5c0ad010 0x7f6f5c14c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f614e7700 1 -- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f5c0ac520 msgr2=0x7f6f5c0c1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f614e7700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f5c0ac520 0x7f6f5c0c1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f614e7700 1 -- 172.21.15.31:0/1849190601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50009550 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f61ce8700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f5c0ad010 0x7f6f5c14c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f60ce6700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f5c0ac520 0x7f6f5c0c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f614e7700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0a74e0 0x7f6f5c151eb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6f580011e0 tx=0x7f6f5800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f4dffb700 1 -- 172.21.15.31:0/1849190601 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58004c80 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f4dffb700 1 -- 172.21.15.31:0/1849190601 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f58004e00 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c1523f0 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.201+0000 7f6f4dffb700 1 -- 172.21.15.31:0/1849190601 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58007310 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.202+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c06bb90 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.203+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5c06bdb0 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.203+0000 7f6f4dffb700 1 -- 172.21.15.31:0/1849190601 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6f58007470 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.204+0000 7f6f4dffb700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4006ac90 0x7f6f4006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.204+0000 7f6f4dffb700 1 -- 172.21.15.31:0/1849190601 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f6f58088fc0 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.204+0000 7f6f60ce6700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4006ac90 0x7f6f4006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.204+0000 7f6f60ce6700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4006ac90 0x7f6f4006d150 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c070970 tx=0x7f6f50008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.208+0000 7f6f4dffb700 1 -- 172.21.15.31:0/1849190601 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6f580bc580 con 0x7f6f5c0a74e0 2024-03-30T01:43:06.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:06.436+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6f5c1e9a40 con 0x7f6f5c0a74e0 2024-03-30T01:43:07.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.388+0000 7f6f4dffb700 1 -- 172.21.15.31:0/1849190601 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6f58054300 con 0x7f6f5c0a74e0 2024-03-30T01:43:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4006ac90 msgr2=0x7f6f4006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4006ac90 0x7f6f4006d150 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c070970 tx=0x7f6f50008030 comp rx=0 tx=0).stop 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0a74e0 msgr2=0x7f6f5c151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0a74e0 0x7f6f5c151eb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6f580011e0 tx=0x7f6f5800cae0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 shutdown_connections 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f5c0a74e0 0x7f6f5c151eb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f6f4006ac90 0x7f6f4006d150 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f5c0ad010 0x7f6f5c14c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 --2- 172.21.15.31:0/1849190601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f5c0ac520 0x7f6f5c0c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 >> 172.21.15.31:0/1849190601 conn(0x7f6f5c06ad30 msgr2=0x7f6f5c0a9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 shutdown_connections 2024-03-30T01:43:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.392+0000 7f6f6374b700 1 -- 172.21.15.31:0/1849190601 wait complete. 2024-03-30T01:43:07.400 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2024-03-30T01:43:07.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T01:43:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.548+0000 7f5e08835700 1 Processor -- start 2024-03-30T01:43:07.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.548+0000 7f5e08835700 1 -- start start 2024-03-30T01:43:07.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e08835700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 0x7f5e04155300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:07.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e08835700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e0406e0d0 0x7f5e040c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:07.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e08835700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e041522e0 con 0x7f5e0414e910 2024-03-30T01:43:07.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e08835700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e04152000 con 0x7f5e0406e0d0 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e08835700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04154b00 con 0x7f5e0406d5c0 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e02d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e0406e0d0 0x7f5e040c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e0259c700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5e0414e910 0x7f5e040c1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50430/0 (socket says 172.21.15.31:50430) 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e02d9d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e0406e0d0 0x7f5e040c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33598/0 (socket says 172.21.15.31:33598) 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e0259c700 1 -- 172.21.15.31:0/1495076600 learned_addr learned my addr 172.21.15.31:0/1495076600 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.549+0000 7f5e01d9b700 1 --2- 172.21.15.31:0/1495076600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 0x7f5e04155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e02d9d700 1 -- 172.21.15.31:0/1495076600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 msgr2=0x7f5e04155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e02d9d700 1 --2- 172.21.15.31:0/1495076600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 0x7f5e04155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e02d9d700 1 -- 172.21.15.31:0/1495076600 >> v1:172.21.15.31:6789/0 conn(0x7f5e0414e910 legacy=0x7f5e040c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e02d9d700 1 -- 172.21.15.31:0/1495076600 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04154de0 con 0x7f5e0406e0d0 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e0159a700 1 -- 172.21.15.31:0/1495076600 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332012409 0 0) 0x7f5e041522e0 con 0x7f5e0414e910 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e01d9b700 1 --2- 172.21.15.31:0/1495076600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 0x7f5e04155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e02d9d700 1 --2- 172.21.15.31:0/1495076600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0406e0d0 0x7f5e040c3c90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009d90 tx=0x7f5dfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31ed973f034b0fd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e0159a700 1 -- 172.21.15.31:0/1495076600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00e070 con 0x7f5e0406e0d0 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e0159a700 1 -- 172.21.15.31:0/1495076600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc00bbd0 con 0x7f5e0406e0d0 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.550+0000 7f5e0159a700 1 -- 172.21.15.31:0/1495076600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc012430 con 0x7f5e0406e0d0 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.551+0000 7f5e08835700 1 -- 172.21.15.31:0/1495076600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0406e0d0 msgr2=0x7f5e040c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.551+0000 7f5e08835700 1 --2- 172.21.15.31:0/1495076600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0406e0d0 0x7f5e040c3c90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009d90 tx=0x7f5dfc00b330 comp rx=0 tx=0).stop 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.551+0000 7f5e08835700 1 -- 172.21.15.31:0/1495076600 shutdown_connections 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.551+0000 7f5e08835700 1 --2- 172.21.15.31:0/1495076600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0406e0d0 0x7f5e040c3c90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.551+0000 7f5e08835700 1 --2- 172.21.15.31:0/1495076600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 0x7f5e04155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.551+0000 7f5e08835700 1 -- 172.21.15.31:0/1495076600 >> 172.21.15.31:0/1495076600 conn(0x7f5e0406ba20 msgr2=0x7f5e0406be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.551+0000 7f5e08835700 1 -- 172.21.15.31:0/1495076600 shutdown_connections 2024-03-30T01:43:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.551+0000 7f5e08835700 1 -- 172.21.15.31:0/1495076600 wait complete. 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.552+0000 7f5e08835700 1 Processor -- start 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.552+0000 7f5e08835700 1 -- start start 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e08835700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 0x7f5e04148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e08835700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0406e0d0 0x7f5e041477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e08835700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0414e910 0x7f5e041e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e08835700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0406b4e0 con 0x7f5e0414e910 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e08835700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0406b200 con 0x7f5e0406e0d0 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e08835700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c0b70 con 0x7f5e0406d5c0 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e02d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0414e910 0x7f5e041e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e02d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0414e910 0x7f5e041e96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33600/0 (socket says 172.21.15.31:33600) 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e02d9d700 1 -- 172.21.15.31:0/731793735 learned_addr learned my addr 172.21.15.31:0/731793735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e0259c700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 0x7f5e04148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.553+0000 7f5e01d9b700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0406e0d0 0x7f5e041477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5e02d9d700 1 -- 172.21.15.31:0/731793735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 msgr2=0x7f5e04148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5e02d9d700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 0x7f5e04148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5e02d9d700 1 -- 172.21.15.31:0/731793735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0406e0d0 msgr2=0x7f5e041477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5e02d9d700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0406e0d0 0x7f5e041477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5e02d9d700 1 -- 172.21.15.31:0/731793735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc009a70 con 0x7f5e0414e910 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5e02d9d700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0414e910 0x7f5e041e96d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5e0406f540 tx=0x7f5dfc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5dfaffd700 1 -- 172.21.15.31:0/731793735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc012430 con 0x7f5e0414e910 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5dfaffd700 1 -- 172.21.15.31:0/731793735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc020070 con 0x7f5e0414e910 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e040c0e50 con 0x7f5e0414e910 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5dfaffd700 1 -- 172.21.15.31:0/731793735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc004280 con 0x7f5e0414e910 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.554+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e04147cf0 con 0x7f5e0414e910 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.555+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e040c09d0 con 0x7f5e0414e910 2024-03-30T01:43:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.555+0000 7f5dfaffd700 1 -- 172.21.15.31:0/731793735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5dfc0043e0 con 0x7f5e0414e910 2024-03-30T01:43:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.556+0000 7f5dfaffd700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5de406ac90 0x7f5de406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.556+0000 7f5dfaffd700 1 -- 172.21.15.31:0/731793735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f5dfc022070 con 0x7f5e0414e910 2024-03-30T01:43:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.556+0000 7f5e0259c700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5de406ac90 0x7f5de406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:07.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.556+0000 7f5e0259c700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5de406ac90 0x7f5de406d150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5dec007fd0 tx=0x7f5dec007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.561+0000 7f5dfaffd700 1 -- 172.21.15.31:0/731793735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5e040c09d0 con 0x7f5e0414e910 2024-03-30T01:43:07.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.870+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5e040c09d0 con 0x7f5e0414e910 2024-03-30T01:43:07.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.870+0000 7f5dfaffd700 1 -- 172.21.15.31:0/731793735 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5021 (secure 0 0 0) 0x7f5e040c09d0 con 0x7f5e0414e910 2024-03-30T01:43:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:07.877 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-03-30T01:43:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2024-03-30T01:43:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2024-03-30T01:43:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbQdmKS4dOBAAoFJ9d9VJCzRRA6dvzqlvdA==", 2024-03-30T01:43:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.877 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdmttL5ChAABt7Prk29JiHASSSErpnSgg==", 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdmy2PsBBAAmQgxB6hsqmzkMOlsiyA2DQ==", 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdm5o/FGRAAuUz1WBZZuhrB8lrBXkgrzg==", 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.879 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB0bQdmv2kRMBAA+Z/39yEKUrYr3hMcRw6UtQ==", 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2024-03-30T01:43:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB3bQdmgRSYLxAA9DUas+VdQgEmd2nF77YNmw==", 2024-03-30T01:43:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:43:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:43:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB6bQdmjqwOMBAASDQNMVbnD3xzqGdxNYtxTg==", 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQB9bQdmM61yLxAAAeBjdPHisSW+6+y6yUMYfw==", 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:43:07.882 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:43:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2024-03-30T01:43:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCDbQdmAeb7NhAAo/xdDmOPniSMNAWGqXprzg==", 2024-03-30T01:43:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:43:07.883 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCGbQdmsibcNhAAYJXjCRMNskc5suFV3ylOqw==", 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2024-03-30T01:43:07.884 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCJbQdmCi5wNhAAoFVc2HDSZQULOOhYt7lBDA==", 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQCMbQdm2DocNxAAUPu6LwRv6gIMMr801Jsjhw==", 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2024-03-30T01:43:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdmickSHxAAowVfYqM7JYAqRJqYG0fQBw==", 2024-03-30T01:43:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:43:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2024-03-30T01:43:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-30T01:43:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBxbQdmijhSIxAAbia5PslTYSh6u9Xvg/PF2g==", 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw pool=cephfs2_data" 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbQdmrbpUFhAAJIkJomJO7Vhgwhow68CKsg==", 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.887 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.testuser", 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQAObgdm9FgmMBAAWTQz7GlIQ8SAlu2U9ss+0g==", 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow rw fsname=cephfs", 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r fsname=cephfs", 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.888 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbQdmVEftMxAA1lDaA5SAWgLRPojMSwq70g==", 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQBwbQdmSyDMLhAAIk3a6lqgYJDDIrak1TE3Og==", 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2024-03-30T01:43:07.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:07.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-03-30T01:43:07.890 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5de406ac90 msgr2=0x7f5de406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5de406ac90 0x7f5de406d150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5dec007fd0 tx=0x7f5dec007de0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0414e910 msgr2=0x7f5e041e96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0414e910 0x7f5e041e96d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5e0406f540 tx=0x7f5dfc004ad0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 shutdown_connections 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e0414e910 0x7f5e041e96d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5de406ac90 0x7f5de406d150 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e0406d5c0 0x7f5e04148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 --2- 172.21.15.31:0/731793735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0406e0d0 0x7f5e041477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 >> 172.21.15.31:0/731793735 conn(0x7f5e0406ba20 msgr2=0x7f5e040c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 shutdown_connections 2024-03-30T01:43:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:07.873+0000 7f5e08835700 1 -- 172.21.15.31:0/731793735 wait complete. 2024-03-30T01:43:07.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.testuser 2024-03-30T01:43:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.025+0000 7fd08079b700 1 Processor -- start 2024-03-30T01:43:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.025+0000 7fd08079b700 1 -- start start 2024-03-30T01:43:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd08079b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd07813da70 0x7fd0780c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd08079b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 0x7fd0780c3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd08079b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd0780722b0 0x7fd0780c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd08079b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0781554d0 con 0x7fd07813da70 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd08079b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd078154b30 con 0x7fd0780722b0 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd08079b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd078154e10 con 0x7fd07806b110 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd07e537700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd07813da70 0x7fd0780c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd07e537700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd07813da70 0x7fd0780c2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33602/0 (socket says 172.21.15.31:33602) 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd07e537700 1 -- 172.21.15.31:0/1332805690 learned_addr learned my addr 172.21.15.31:0/1332805690 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd07dd36700 1 --2- 172.21.15.31:0/1332805690 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 0x7fd0780c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.026+0000 7fd07ed38700 1 --2- 172.21.15.31:0/1332805690 >> v2:172.21.15.59:3300/0 conn(0x7fd0780722b0 0x7fd0780c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.027+0000 7fd07e537700 1 -- 172.21.15.31:0/1332805690 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 msgr2=0x7fd0780c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.027+0000 7fd07e537700 1 --2- 172.21.15.31:0/1332805690 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 0x7fd0780c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.027+0000 7fd07e537700 1 -- 172.21.15.31:0/1332805690 >> v2:172.21.15.59:3300/0 conn(0x7fd0780722b0 msgr2=0x7fd0780c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.027+0000 7fd07e537700 1 --2- 172.21.15.31:0/1332805690 >> v2:172.21.15.59:3300/0 conn(0x7fd0780722b0 0x7fd0780c0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.027+0000 7fd07e537700 1 -- 172.21.15.31:0/1332805690 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0781550f0 con 0x7fd07813da70 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.027+0000 7fd07e537700 1 --2- 172.21.15.31:0/1332805690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd07813da70 0x7fd0780c2dd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd078072aa0 tx=0x7fd074009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd23879e5ab759e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.027+0000 7fd07d535700 1 -- 172.21.15.31:0/1332805690 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074011830 con 0x7fd07813da70 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.027+0000 7fd07d535700 1 -- 172.21.15.31:0/1332805690 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd074011990 con 0x7fd07813da70 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.027+0000 7fd07d535700 1 -- 172.21.15.31:0/1332805690 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074011d20 con 0x7fd07813da70 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.028+0000 7fd08079b700 1 -- 172.21.15.31:0/1332805690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd07813da70 msgr2=0x7fd0780c2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.028+0000 7fd08079b700 1 --2- 172.21.15.31:0/1332805690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd07813da70 0x7fd0780c2dd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd078072aa0 tx=0x7fd074009bd0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.028+0000 7fd08079b700 1 -- 172.21.15.31:0/1332805690 shutdown_connections 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.028+0000 7fd08079b700 1 --2- 172.21.15.31:0/1332805690 >> v2:172.21.15.59:3300/0 conn(0x7fd0780722b0 0x7fd0780c0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.028+0000 7fd08079b700 1 --2- 172.21.15.31:0/1332805690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd07813da70 0x7fd0780c2dd0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.028+0000 7fd08079b700 1 --2- 172.21.15.31:0/1332805690 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 0x7fd0780c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.028+0000 7fd08079b700 1 -- 172.21.15.31:0/1332805690 >> 172.21.15.31:0/1332805690 conn(0x7fd0780c3a60 msgr2=0x7fd0780c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.028+0000 7fd08079b700 1 -- 172.21.15.31:0/1332805690 shutdown_connections 2024-03-30T01:43:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.028+0000 7fd08079b700 1 -- 172.21.15.31:0/1332805690 wait complete. 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.029+0000 7fd08079b700 1 Processor -- start 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.029+0000 7fd08079b700 1 -- start start 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.030+0000 7fd08079b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 0x7fd0781e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.030+0000 7fd08079b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780722b0 0x7fd078138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.030+0000 7fd08079b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd07813da70 0x7fd078138640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.030+0000 7fd08079b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078158d30 con 0x7fd0780722b0 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.030+0000 7fd08079b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078156390 con 0x7fd07813da70 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.030+0000 7fd08079b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd078156670 con 0x7fd07806b110 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07dd36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780722b0 0x7fd078138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07dd36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780722b0 0x7fd078138100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33618/0 (socket says 172.21.15.31:33618) 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07dd36700 1 -- 172.21.15.31:0/3326468518 learned_addr learned my addr 172.21.15.31:0/3326468518 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07e537700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 0x7fd0781e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07ed38700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd07813da70 0x7fd078138640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07dd36700 1 -- 172.21.15.31:0/3326468518 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 msgr2=0x7fd0781e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07dd36700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 0x7fd0781e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07dd36700 1 -- 172.21.15.31:0/3326468518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd07813da70 msgr2=0x7fd078138640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07dd36700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd07813da70 0x7fd078138640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07dd36700 1 -- 172.21.15.31:0/3326468518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd074009550 con 0x7fd0780722b0 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07e537700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 0x7fd0781e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd07dd36700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780722b0 0x7fd078138100 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd06800e900 tx=0x7fd06800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd06effd700 1 -- 172.21.15.31:0/3326468518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068009690 con 0x7fd0780722b0 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd06effd700 1 -- 172.21.15.31:0/3326468518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06800c850 con 0x7fd0780722b0 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd078156950 con 0x7fd0780722b0 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd06effd700 1 -- 172.21.15.31:0/3326468518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800cc00 con 0x7fd0780722b0 2024-03-30T01:43:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.031+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0781ee3c0 con 0x7fd0780722b0 2024-03-30T01:43:08.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.032+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07804ea80 con 0x7fd0780722b0 2024-03-30T01:43:08.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.033+0000 7fd06effd700 1 -- 172.21.15.31:0/3326468518 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd068013470 con 0x7fd0780722b0 2024-03-30T01:43:08.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.033+0000 7fd06effd700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd05c06ac90 0x7fd05c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.033+0000 7fd06effd700 1 -- 172.21.15.31:0/3326468518 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd068089500 con 0x7fd0780722b0 2024-03-30T01:43:08.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.034+0000 7fd07e537700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd05c06ac90 0x7fd05c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.034+0000 7fd07e537700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd05c06ac90 0x7fd05c06d150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd074014040 tx=0x7fd074008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:08.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.038+0000 7fd06effd700 1 -- 172.21.15.31:0/3326468518 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd0680548e0 con 0x7fd0780722b0 2024-03-30T01:43:08.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.341+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.testuser"} v 0) v1 -- 0x7fd0780ac860 con 0x7fd0780722b0 2024-03-30T01:43:08.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.345+0000 7fd06effd700 1 -- 172.21.15.31:0/3326468518 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.testuser"}]=0 v16) v1 ==== 85+0+0 (secure 0 0 0) 0x7fd068057f00 con 0x7fd0780722b0 2024-03-30T01:43:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd05c06ac90 msgr2=0x7fd05c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd05c06ac90 0x7fd05c06d150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd074014040 tx=0x7fd074008030 comp rx=0 tx=0).stop 2024-03-30T01:43:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780722b0 msgr2=0x7fd078138100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780722b0 0x7fd078138100 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd06800e900 tx=0x7fd06800ecc0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 shutdown_connections 2024-03-30T01:43:08.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fd05c06ac90 0x7fd05c06d150 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd07813da70 0x7fd078138640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd07806b110 0x7fd0781e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 --2- 172.21.15.31:0/3326468518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0780722b0 0x7fd078138100 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 >> 172.21.15.31:0/3326468518 conn(0x7fd0780c3a60 msgr2=0x7fd078153d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:08.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.347+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 shutdown_connections 2024-03-30T01:43:08.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.348+0000 7fd08079b700 1 -- 172.21.15.31:0/3326468518 wait complete. 2024-03-30T01:43:08.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:08.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.498+0000 7f2900c38700 1 Processor -- start 2024-03-30T01:43:08.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.498+0000 7f2900c38700 1 -- start start 2024-03-30T01:43:08.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.499+0000 7f2900c38700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc0bd7f0 0x7f28fc155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.499+0000 7f2900c38700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28fc154760 0x7f28fc06b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.499+0000 7f2900c38700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc0c8d10 con 0x7f28fc06b170 2024-03-30T01:43:08.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.499+0000 7f2900c38700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc06bef0 con 0x7f28fc154760 2024-03-30T01:43:08.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.499+0000 7f2900c38700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0c98b0 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.499+0000 7f28f9d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc0bd7f0 0x7f28fc155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.499+0000 7f28fad9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28fc154760 0x7f28fc06b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.499+0000 7f28f9d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc0bd7f0 0x7f28fc155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:32850/0 (socket says 172.21.15.31:32850) 2024-03-30T01:43:08.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.499+0000 7f28f9d9b700 1 -- 172.21.15.31:0/1735060215 learned_addr learned my addr 172.21.15.31:0/1735060215 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:08.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.500+0000 7f28fad9d700 1 -- 172.21.15.31:0/1735060215 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc0bd7f0 msgr2=0x7f28fc155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.500+0000 7f28fad9d700 1 --2- 172.21.15.31:0/1735060215 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc0bd7f0 0x7f28fc155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.500+0000 7f28fad9d700 1 -- 172.21.15.31:0/1735060215 >> v1:172.21.15.59:6789/0 conn(0x7f28fc06b170 legacy=0x7f28fc0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.500+0000 7f28fad9d700 1 -- 172.21.15.31:0/1735060215 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc0c8100 con 0x7f28fc154760 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.500+0000 7f28f9d9b700 1 --2- 172.21.15.31:0/1735060215 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc0bd7f0 0x7f28fc155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.500+0000 7f28fad9d700 1 --2- 172.21.15.31:0/1735060215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc154760 0x7f28fc06b550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009d90 tx=0x7f28f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cda9399c0b68de8a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.500+0000 7f28f959a700 1 -- 172.21.15.31:0/1735060215 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f400e070 con 0x7f28fc154760 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.500+0000 7f28f959a700 1 -- 172.21.15.31:0/1735060215 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f400bb20 con 0x7f28fc154760 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.500+0000 7f28f959a700 1 -- 172.21.15.31:0/1735060215 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4012430 con 0x7f28fc154760 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.501+0000 7f2900c38700 1 -- 172.21.15.31:0/1735060215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc154760 msgr2=0x7f28fc06b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.501+0000 7f2900c38700 1 --2- 172.21.15.31:0/1735060215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc154760 0x7f28fc06b550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009d90 tx=0x7f28f400b330 comp rx=0 tx=0).stop 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.501+0000 7f2900c38700 1 -- 172.21.15.31:0/1735060215 shutdown_connections 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.501+0000 7f2900c38700 1 --2- 172.21.15.31:0/1735060215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc154760 0x7f28fc06b550 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.501+0000 7f2900c38700 1 --2- 172.21.15.31:0/1735060215 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc0bd7f0 0x7f28fc155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.501+0000 7f2900c38700 1 -- 172.21.15.31:0/1735060215 >> 172.21.15.31:0/1735060215 conn(0x7f28fc0ca0d0 msgr2=0x7f28fc0ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.501+0000 7f2900c38700 1 -- 172.21.15.31:0/1735060215 shutdown_connections 2024-03-30T01:43:08.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.501+0000 7f2900c38700 1 -- 172.21.15.31:0/1735060215 wait complete. 2024-03-30T01:43:08.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.502+0000 7f2900c38700 1 Processor -- start 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.502+0000 7f2900c38700 1 -- start start 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f2900c38700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc06b170 0x7f28fc156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f2900c38700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc0bd7f0 0x7f28fc0bbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f2900c38700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc154760 0x7f28fc0c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f2900c38700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0c2d80 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f2900c38700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0c05a0 con 0x7f28fc06b170 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f2900c38700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0c0810 con 0x7f28fc154760 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28f9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc0bd7f0 0x7f28fc0bbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28f9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc0bd7f0 0x7f28fc0bbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33620/0 (socket says 172.21.15.31:33620) 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28f9d9b700 1 -- 172.21.15.31:0/65655126 learned_addr learned my addr 172.21.15.31:0/65655126 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28fad9d700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc154760 0x7f28fc0c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28fa59c700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc06b170 0x7f28fc156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28f9d9b700 1 -- 172.21.15.31:0/65655126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc154760 msgr2=0x7f28fc0c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28f9d9b700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc154760 0x7f28fc0c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28f9d9b700 1 -- 172.21.15.31:0/65655126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc06b170 msgr2=0x7f28fc156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28f9d9b700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc06b170 0x7f28fc156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28f9d9b700 1 -- 172.21.15.31:0/65655126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f0009550 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.503+0000 7f28fa59c700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc06b170 0x7f28fc156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.504+0000 7f28f9d9b700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc0bd7f0 0x7f28fc0bbf40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f28f000ebe0 tx=0x7f28f000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.504+0000 7f28eaffd700 1 -- 172.21.15.31:0/65655126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000c850 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.504+0000 7f28eaffd700 1 -- 172.21.15.31:0/65655126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f000c9b0 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.504+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f4009a70 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.504+0000 7f28eaffd700 1 -- 172.21.15.31:0/65655126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000cd60 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.504+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc0c0dd0 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.505+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc0bcb00 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.506+0000 7f28eaffd700 1 -- 172.21.15.31:0/65655126 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f28f0013470 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.506+0000 7f28eaffd700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f28dc06ac90 0x7f28dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.506+0000 7f28eaffd700 1 -- 172.21.15.31:0/65655126 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f28f00895f0 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.506+0000 7f28fa59c700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f28dc06ac90 0x7f28dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.506+0000 7f28fa59c700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f28dc06ac90 0x7f28dc06d150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000f60 tx=0x7f28ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:08.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.511+0000 7f28eaffd700 1 -- 172.21.15.31:0/65655126 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f28f0054960 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.726+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28fc04ea80 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.727+0000 7f28eaffd700 1 -- 172.21.15.31:0/65655126 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+12546 (secure 0 0 0) 0x7f28f0057f80 con 0x7f28fc0bd7f0 2024-03-30T01:43:08.734 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:08.734 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":56,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:07.385989+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:08.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f28dc06ac90 msgr2=0x7f28dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f28dc06ac90 0x7f28dc06d150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f28ec000f60 tx=0x7f28ec009040 comp rx=0 tx=0).stop 2024-03-30T01:43:08.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc0bd7f0 msgr2=0x7f28fc0bbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc0bd7f0 0x7f28fc0bbf40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f28f000ebe0 tx=0x7f28f000eef0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 shutdown_connections 2024-03-30T01:43:08.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f28dc06ac90 0x7f28dc06d150 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28fc154760 0x7f28fc0c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc06b170 0x7f28fc156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 --2- 172.21.15.31:0/65655126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28fc0bd7f0 0x7f28fc0bbf40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 >> 172.21.15.31:0/65655126 conn(0x7f28fc0ca0d0 msgr2=0x7f28fc0ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:08.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 shutdown_connections 2024-03-30T01:43:08.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.730+0000 7f2900c38700 1 -- 172.21.15.31:0/65655126 wait complete. 2024-03-30T01:43:08.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:43:08.752 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:43:08.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.885+0000 7f77be0ee700 1 Processor -- start 2024-03-30T01:43:08.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.885+0000 7f77be0ee700 1 -- start start 2024-03-30T01:43:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77be0ee700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f77b8155490 0x7f77b806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77be0ee700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b8155000 con 0x7f77b80721f0 2024-03-30T01:43:08.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77be0ee700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b806e650 con 0x7f77b8151ea0 2024-03-30T01:43:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77be0ee700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b806e930 con 0x7f77b8155490 2024-03-30T01:43:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b6ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f77b80721f0 0x7f77b806a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50438/0 (socket says 172.21.15.31:50438) 2024-03-30T01:43:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b6ffd700 1 -- 172.21.15.31:0/1689517666 learned_addr learned my addr 172.21.15.31:0/1689517666 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b7fff700 1 --2- 172.21.15.31:0/1689517666 >> v2:172.21.15.59:3300/0 conn(0x7f77b8155490 0x7f77b806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b67fc700 1 -- 172.21.15.31:0/1689517666 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800098294 0 0) 0x7f77b8155000 con 0x7f77b80721f0 2024-03-30T01:43:08.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b67fc700 1 -- 172.21.15.31:0/1689517666 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a40035b0 con 0x7f77b80721f0 2024-03-30T01:43:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b67fc700 1 -- 172.21.15.31:0/1689517666 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242374485 0 0) 0x7f77b806e650 con 0x7f77b8151ea0 2024-03-30T01:43:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b67fc700 1 -- 172.21.15.31:0/1689517666 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b8155000 con 0x7f77b8151ea0 2024-03-30T01:43:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b7fff700 1 -- 172.21.15.31:0/1689517666 >> v1:172.21.15.59:6789/0 conn(0x7f77b8151ea0 legacy=0x7f77b80c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b7fff700 1 -- 172.21.15.31:0/1689517666 >> v1:172.21.15.31:6789/0 conn(0x7f77b80721f0 legacy=0x7f77b806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.886+0000 7f77b7fff700 1 -- 172.21.15.31:0/1689517666 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b81549b0 con 0x7f77b8155490 2024-03-30T01:43:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.887+0000 7f77b7fff700 1 --2- 172.21.15.31:0/1689517666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b8155490 0x7f77b806ae10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009d90 tx=0x7f77b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b162c6ded2545c9b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:08.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.887+0000 7f77b67fc700 1 -- 172.21.15.31:0/1689517666 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b000e070 con 0x7f77b8155490 2024-03-30T01:43:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.887+0000 7f77b67fc700 1 -- 172.21.15.31:0/1689517666 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b000bb20 con 0x7f77b8155490 2024-03-30T01:43:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.887+0000 7f77b67fc700 1 -- 172.21.15.31:0/1689517666 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0012430 con 0x7f77b8155490 2024-03-30T01:43:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.887+0000 7f77be0ee700 1 -- 172.21.15.31:0/1689517666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b8155490 msgr2=0x7f77b806ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.887+0000 7f77be0ee700 1 --2- 172.21.15.31:0/1689517666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b8155490 0x7f77b806ae10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009d90 tx=0x7f77b000b330 comp rx=0 tx=0).stop 2024-03-30T01:43:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.888+0000 7f77be0ee700 1 -- 172.21.15.31:0/1689517666 shutdown_connections 2024-03-30T01:43:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.888+0000 7f77be0ee700 1 --2- 172.21.15.31:0/1689517666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b8155490 0x7f77b806ae10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.888+0000 7f77be0ee700 1 -- 172.21.15.31:0/1689517666 >> 172.21.15.31:0/1689517666 conn(0x7f77b80c3980 msgr2=0x7f77b80c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.888+0000 7f77be0ee700 1 -- 172.21.15.31:0/1689517666 shutdown_connections 2024-03-30T01:43:08.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.888+0000 7f77be0ee700 1 -- 172.21.15.31:0/1689517666 wait complete. 2024-03-30T01:43:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.889+0000 7f77be0ee700 1 Processor -- start 2024-03-30T01:43:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.889+0000 7f77be0ee700 1 -- start start 2024-03-30T01:43:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77be0ee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80721f0 0x7f77b8148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77be0ee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b8151ea0 0x7f77b8149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77be0ee700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b8155490 0x7f77b81ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77be0ee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b81496a0 con 0x7f77b80721f0 2024-03-30T01:43:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77be0ee700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b80c2410 con 0x7f77b8155490 2024-03-30T01:43:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77be0ee700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b814b1b0 con 0x7f77b8151ea0 2024-03-30T01:43:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80721f0 0x7f77b8148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80721f0 0x7f77b8148c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33622/0 (socket says 172.21.15.31:33622) 2024-03-30T01:43:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b77fe700 1 -- 172.21.15.31:0/2277346269 learned_addr learned my addr 172.21.15.31:0/2277346269 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b7fff700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b8155490 0x7f77b81ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b6ffd700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b8151ea0 0x7f77b8149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b77fe700 1 -- 172.21.15.31:0/2277346269 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b8151ea0 msgr2=0x7f77b8149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b77fe700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b8151ea0 0x7f77b8149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b77fe700 1 -- 172.21.15.31:0/2277346269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b8155490 msgr2=0x7f77b81ed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b77fe700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b8155490 0x7f77b81ed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.890+0000 7f77b77fe700 1 -- 172.21.15.31:0/2277346269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b0009a70 con 0x7f77b80721f0 2024-03-30T01:43:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.891+0000 7f77b77fe700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80721f0 0x7f77b8148c20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c520 tx=0x7f77a800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.891+0000 7f77affff700 1 -- 172.21.15.31:0/2277346269 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a80052d0 con 0x7f77b80721f0 2024-03-30T01:43:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.891+0000 7f77affff700 1 -- 172.21.15.31:0/2277346269 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a8005430 con 0x7f77b80721f0 2024-03-30T01:43:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.891+0000 7f77affff700 1 -- 172.21.15.31:0/2277346269 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a80038c0 con 0x7f77b80721f0 2024-03-30T01:43:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.891+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b814b490 con 0x7f77b80721f0 2024-03-30T01:43:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.891+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b8155b70 con 0x7f77b80721f0 2024-03-30T01:43:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.892+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b80747d0 con 0x7f77b80721f0 2024-03-30T01:43:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.892+0000 7f77affff700 1 -- 172.21.15.31:0/2277346269 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f77a8003a20 con 0x7f77b80721f0 2024-03-30T01:43:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.893+0000 7f77affff700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779806ac90 0x7f779806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.893+0000 7f77affff700 1 -- 172.21.15.31:0/2277346269 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f77a80884b0 con 0x7f77b80721f0 2024-03-30T01:43:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.893+0000 7f77b6ffd700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779806ac90 0x7f779806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.893+0000 7f77b6ffd700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779806ac90 0x7f779806d150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f77b8149a10 tx=0x7f77a0007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:08.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:08.897+0000 7f77affff700 1 -- 172.21.15.31:0/2277346269 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f77a8053930 con 0x7f77b80721f0 2024-03-30T01:43:09.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.110+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77b80c1290 con 0x7f77b80721f0 2024-03-30T01:43:09.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.111+0000 7f77affff700 1 -- 172.21.15.31:0/2277346269 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77a8017020 con 0x7f77b80721f0 2024-03-30T01:43:09.117 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:43:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779806ac90 msgr2=0x7f779806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779806ac90 0x7f779806d150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f77b8149a10 tx=0x7f77a0007dc0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80721f0 msgr2=0x7f77b8148c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80721f0 0x7f77b8148c20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c520 tx=0x7f77a800c830 comp rx=0 tx=0).stop 2024-03-30T01:43:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 shutdown_connections 2024-03-30T01:43:09.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b80721f0 0x7f77b8148c20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f779806ac90 0x7f779806d150 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b8155490 0x7f77b81ed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 --2- 172.21.15.31:0/2277346269 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b8151ea0 0x7f77b8149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 >> 172.21.15.31:0/2277346269 conn(0x7f77b80c3980 msgr2=0x7f77b80c02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 shutdown_connections 2024-03-30T01:43:09.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.113+0000 7f77be0ee700 1 -- 172.21.15.31:0/2277346269 wait complete. 2024-03-30T01:43:09.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T01:43:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.264+0000 7f01b03d1700 1 Processor -- start 2024-03-30T01:43:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.264+0000 7f01b03d1700 1 -- start start 2024-03-30T01:43:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.265+0000 7f01b03d1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a80c5f50 0x7f01a80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.265+0000 7f01b03d1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01a8157d90 0x7f01a8158200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.265+0000 7f01b03d1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a8155220 con 0x7f01a8151f90 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.265+0000 7f01b03d1700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80aa4c0 con 0x7f01a8157d90 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.265+0000 7f01b03d1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80c3380 con 0x7f01a80c5f50 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.265+0000 7f01ae96e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01a8157d90 0x7f01a8158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.265+0000 7f01ae96e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f01a8157d90 0x7f01a8158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33626/0 (socket says 172.21.15.31:33626) 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.265+0000 7f01ae96e700 1 -- 172.21.15.31:0/3160737240 learned_addr learned my addr 172.21.15.31:0/3160737240 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.265+0000 7f01ad96c700 1 --2- 172.21.15.31:0/3160737240 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a80c5f50 0x7f01a80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.266+0000 7f01ae96e700 1 -- 172.21.15.31:0/3160737240 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a80c5f50 msgr2=0x7f01a80705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.266+0000 7f01ae96e700 1 --2- 172.21.15.31:0/3160737240 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a80c5f50 0x7f01a80705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.266+0000 7f01ae96e700 1 -- 172.21.15.31:0/3160737240 >> v1:172.21.15.59:6789/0 conn(0x7f01a8151f90 legacy=0x7f01a80c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.266+0000 7f01ad16b700 1 -- 172.21.15.31:0/3160737240 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212743297 0 0) 0x7f01a8155220 con 0x7f01a8151f90 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.266+0000 7f01ae96e700 1 -- 172.21.15.31:0/3160737240 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a81556b0 con 0x7f01a8157d90 2024-03-30T01:43:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.266+0000 7f01ae96e700 1 --2- 172.21.15.31:0/3160737240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a8157d90 0x7f01a8158200 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009e50 tx=0x7f01a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6805cf670b9cb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.266+0000 7f01ad16b700 1 -- 172.21.15.31:0/3160737240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000e070 con 0x7f01a8157d90 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.266+0000 7f01ad16b700 1 -- 172.21.15.31:0/3160737240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a000b910 con 0x7f01a8157d90 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.266+0000 7f01ad16b700 1 -- 172.21.15.31:0/3160737240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000bcc0 con 0x7f01a8157d90 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.267+0000 7f01b03d1700 1 -- 172.21.15.31:0/3160737240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a8157d90 msgr2=0x7f01a8158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.267+0000 7f01b03d1700 1 --2- 172.21.15.31:0/3160737240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a8157d90 0x7f01a8158200 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009e50 tx=0x7f01a000b330 comp rx=0 tx=0).stop 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.267+0000 7f01b03d1700 1 -- 172.21.15.31:0/3160737240 shutdown_connections 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.267+0000 7f01b03d1700 1 --2- 172.21.15.31:0/3160737240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a8157d90 0x7f01a8158200 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.267+0000 7f01b03d1700 1 --2- 172.21.15.31:0/3160737240 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a80c5f50 0x7f01a80705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.267+0000 7f01b03d1700 1 -- 172.21.15.31:0/3160737240 >> 172.21.15.31:0/3160737240 conn(0x7f01a806ad30 msgr2=0x7f01a80a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.267+0000 7f01b03d1700 1 -- 172.21.15.31:0/3160737240 shutdown_connections 2024-03-30T01:43:09.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.267+0000 7f01b03d1700 1 -- 172.21.15.31:0/3160737240 wait complete. 2024-03-30T01:43:09.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.268+0000 7f01b03d1700 1 Processor -- start 2024-03-30T01:43:09.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.268+0000 7f01b03d1700 1 -- start start 2024-03-30T01:43:09.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01b03d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a80c5f50 0x7f01a8152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01b03d1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01a8151f90 0x7f01a814e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01b03d1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a8157d90 0x7f01a81ed840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01b03d1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81ed420 con 0x7f01a80c5f50 2024-03-30T01:43:09.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01b03d1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81ed0d0 con 0x7f01a8151f90 2024-03-30T01:43:09.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01b03d1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80a8e20 con 0x7f01a8157d90 2024-03-30T01:43:09.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ae16d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a80c5f50 0x7f01a8152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ae16d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a80c5f50 0x7f01a8152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33636/0 (socket says 172.21.15.31:33636) 2024-03-30T01:43:09.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ae16d700 1 -- 172.21.15.31:0/3348351116 learned_addr learned my addr 172.21.15.31:0/3348351116 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:09.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ae96e700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a8157d90 0x7f01a81ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ad96c700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01a8151f90 0x7f01a814e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ae16d700 1 -- 172.21.15.31:0/3348351116 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a8157d90 msgr2=0x7f01a81ed840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ae16d700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a8157d90 0x7f01a81ed840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ae16d700 1 -- 172.21.15.31:0/3348351116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01a8151f90 msgr2=0x7f01a814e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ae16d700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01a8151f90 0x7f01a814e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.269+0000 7f01ae16d700 1 -- 172.21.15.31:0/3348351116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0009b30 con 0x7f01a80c5f50 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.270+0000 7f01ae16d700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a80c5f50 0x7f01a8152e40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c550 tx=0x7f01a400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.270+0000 7f019affd700 1 -- 172.21.15.31:0/3348351116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a40054e0 con 0x7f01a80c5f50 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.270+0000 7f019affd700 1 -- 172.21.15.31:0/3348351116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4003680 con 0x7f01a80c5f50 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.270+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a806fcd0 con 0x7f01a80c5f50 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.270+0000 7f019affd700 1 -- 172.21.15.31:0/3348351116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4003a30 con 0x7f01a80c5f50 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.270+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a814e900 con 0x7f01a80c5f50 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.271+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a81ed560 con 0x7f01a80c5f50 2024-03-30T01:43:09.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.271+0000 7f019affd700 1 -- 172.21.15.31:0/3348351116 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f01a4003b90 con 0x7f01a80c5f50 2024-03-30T01:43:09.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.272+0000 7f019affd700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f018c06ac90 0x7f018c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.272+0000 7f019affd700 1 -- 172.21.15.31:0/3348351116 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f01a40883d0 con 0x7f01a80c5f50 2024-03-30T01:43:09.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.272+0000 7f01ad96c700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f018c06ac90 0x7f018c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.272+0000 7f01ad96c700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f018c06ac90 0x7f018c06d150 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f01a806f9e0 tx=0x7f019c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:09.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.276+0000 7f019affd700 1 -- 172.21.15.31:0/3348351116 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f01a81ed560 con 0x7f01a80c5f50 2024-03-30T01:43:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.493+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f01a81ed560 con 0x7f01a80c5f50 2024-03-30T01:43:09.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.493+0000 7f019affd700 1 -- 172.21.15.31:0/3348351116 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+65 (secure 0 0 0) 0x7f01a81ed560 con 0x7f01a80c5f50 2024-03-30T01:43:09.500 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:09.500 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-03-30T01:43:09.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:09.500 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-03-30T01:43:09.500 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-03-30T01:43:09.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:09.500 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-03-30T01:43:09.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.495+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f018c06ac90 msgr2=0x7f018c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.495+0000 7f01b03d1700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f018c06ac90 0x7f018c06d150 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f01a806f9e0 tx=0x7f019c009210 comp rx=0 tx=0).stop 2024-03-30T01:43:09.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.495+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a80c5f50 msgr2=0x7f01a8152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.495+0000 7f01b03d1700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a80c5f50 0x7f01a8152e40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c550 tx=0x7f01a400c860 comp rx=0 tx=0).stop 2024-03-30T01:43:09.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.495+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 shutdown_connections 2024-03-30T01:43:09.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.496+0000 7f01b03d1700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01a80c5f50 0x7f01a8152e40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.496+0000 7f01b03d1700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f018c06ac90 0x7f018c06d150 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.496+0000 7f01b03d1700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01a8157d90 0x7f01a81ed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.496+0000 7f01b03d1700 1 --2- 172.21.15.31:0/3348351116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01a8151f90 0x7f01a814e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.496+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 >> 172.21.15.31:0/3348351116 conn(0x7f01a806ad30 msgr2=0x7f01a80c5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:09.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.496+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 shutdown_connections 2024-03-30T01:43:09.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.496+0000 7f01b03d1700 1 -- 172.21.15.31:0/3348351116 wait complete. 2024-03-30T01:43:09.516 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T01:43:09.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T01:43:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.649+0000 7fc3084e5700 1 Processor -- start 2024-03-30T01:43:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.649+0000 7fc3084e5700 1 -- start start 2024-03-30T01:43:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.650+0000 7fc3084e5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3000c4960 0x7fc3000abe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.650+0000 7fc3084e5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3000ac3b0 0x7fc30015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.650+0000 7fc3084e5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3000c7100 con 0x7fc300151ce0 2024-03-30T01:43:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.650+0000 7fc3084e5700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3000c5750 con 0x7fc3000c4960 2024-03-30T01:43:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.650+0000 7fc3084e5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3000c4f00 con 0x7fc3000ac3b0 2024-03-30T01:43:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc305a80700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3000c4960 0x7fc3000abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc305a80700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3000c4960 0x7fc3000abe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33648/0 (socket says 172.21.15.31:33648) 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc305a80700 1 -- 172.21.15.31:0/3396039714 learned_addr learned my addr 172.21.15.31:0/3396039714 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc306a82700 1 --2- 172.21.15.31:0/3396039714 >> v2:172.21.15.59:3300/0 conn(0x7fc3000ac3b0 0x7fc30015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc305a80700 1 -- 172.21.15.31:0/3396039714 >> v2:172.21.15.59:3300/0 conn(0x7fc3000ac3b0 msgr2=0x7fc30015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc305a80700 1 --2- 172.21.15.31:0/3396039714 >> v2:172.21.15.59:3300/0 conn(0x7fc3000ac3b0 0x7fc30015f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc305a80700 1 -- 172.21.15.31:0/3396039714 >> v1:172.21.15.59:6789/0 conn(0x7fc300151ce0 legacy=0x7fc3000c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc305a80700 1 -- 172.21.15.31:0/3396039714 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300157db0 con 0x7fc3000c4960 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc306a82700 1 --2- 172.21.15.31:0/3396039714 >> v2:172.21.15.59:3300/0 conn(0x7fc3000ac3b0 0x7fc30015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc305a80700 1 --2- 172.21.15.31:0/3396039714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000c4960 0x7fc3000abe70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc3000c6840 tx=0x7fc2f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6208b79cccbcf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc30527f700 1 -- 172.21.15.31:0/3396039714 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000b870 con 0x7fc3000c4960 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc30527f700 1 -- 172.21.15.31:0/3396039714 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f000b9d0 con 0x7fc3000c4960 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.651+0000 7fc30527f700 1 -- 172.21.15.31:0/3396039714 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000bd60 con 0x7fc3000c4960 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.652+0000 7fc3084e5700 1 -- 172.21.15.31:0/3396039714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000c4960 msgr2=0x7fc3000abe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.652+0000 7fc3084e5700 1 --2- 172.21.15.31:0/3396039714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000c4960 0x7fc3000abe70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc3000c6840 tx=0x7fc2f0009bd0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.652+0000 7fc3084e5700 1 -- 172.21.15.31:0/3396039714 shutdown_connections 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.652+0000 7fc3084e5700 1 --2- 172.21.15.31:0/3396039714 >> v2:172.21.15.59:3300/0 conn(0x7fc3000ac3b0 0x7fc30015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.652+0000 7fc3084e5700 1 --2- 172.21.15.31:0/3396039714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000c4960 0x7fc3000abe70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.652+0000 7fc3084e5700 1 -- 172.21.15.31:0/3396039714 >> 172.21.15.31:0/3396039714 conn(0x7fc300072010 msgr2=0x7fc300072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.652+0000 7fc3084e5700 1 -- 172.21.15.31:0/3396039714 shutdown_connections 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.652+0000 7fc3084e5700 1 -- 172.21.15.31:0/3396039714 wait complete. 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.653+0000 7fc3084e5700 1 Processor -- start 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.653+0000 7fc3084e5700 1 -- start start 2024-03-30T01:43:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc3084e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000ac3b0 0x7fc3001edad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc3084e5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3000c4960 0x7fc3001476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc3084e5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc300151ce0 0x7fc3000c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc3084e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000c1680 con 0x7fc3000ac3b0 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc3084e5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000c1ac0 con 0x7fc300151ce0 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc3084e5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc300147c30 con 0x7fc3000c4960 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc306281700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000ac3b0 0x7fc3001edad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc306281700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000ac3b0 0x7fc3001edad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33664/0 (socket says 172.21.15.31:33664) 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc306281700 1 -- 172.21.15.31:0/2040149547 learned_addr learned my addr 172.21.15.31:0/2040149547 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc305a80700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3000c4960 0x7fc3001476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc306a82700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc300151ce0 0x7fc3000c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.654+0000 7fc306281700 1 -- 172.21.15.31:0/2040149547 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3000c4960 msgr2=0x7fc3001476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc306281700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3000c4960 0x7fc3001476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc306281700 1 -- 172.21.15.31:0/2040149547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc300151ce0 msgr2=0x7fc3000c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc306281700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc300151ce0 0x7fc3000c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc306281700 1 -- 172.21.15.31:0/2040149547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0009550 con 0x7fc3000ac3b0 2024-03-30T01:43:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc306281700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000ac3b0 0x7fc3001edad0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00c450 tx=0x7fc2fc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc2f6ffd700 1 -- 172.21.15.31:0/2040149547 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0052d0 con 0x7fc3000ac3b0 2024-03-30T01:43:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc2f6ffd700 1 -- 172.21.15.31:0/2040149547 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc005430 con 0x7fc3000ac3b0 2024-03-30T01:43:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3001ecff0 con 0x7fc3000ac3b0 2024-03-30T01:43:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc2f6ffd700 1 -- 172.21.15.31:0/2040149547 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0038c0 con 0x7fc3000ac3b0 2024-03-30T01:43:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.655+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3001ed5b0 con 0x7fc3000ac3b0 2024-03-30T01:43:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.656+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc300073300 con 0x7fc3000ac3b0 2024-03-30T01:43:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.657+0000 7fc2f6ffd700 1 -- 172.21.15.31:0/2040149547 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc2fc003a20 con 0x7fc3000ac3b0 2024-03-30T01:43:09.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.657+0000 7fc2f6ffd700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc2e406ac90 0x7fc2e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:09.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.657+0000 7fc2f6ffd700 1 -- 172.21.15.31:0/2040149547 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc2fc0883a0 con 0x7fc3000ac3b0 2024-03-30T01:43:09.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.657+0000 7fc305a80700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc2e406ac90 0x7fc2e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:09.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.658+0000 7fc305a80700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc2e406ac90 0x7fc2e406d150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:09.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.662+0000 7fc2f6ffd700 1 -- 172.21.15.31:0/2040149547 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc2fc053820 con 0x7fc3000ac3b0 2024-03-30T01:43:09.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:09.878+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc3000c18d0 con 0x7fc3000ac3b0 2024-03-30T01:43:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.782+0000 7fc2f6ffd700 1 -- 172.21.15.31:0/2040149547 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc2fc00ce60 con 0x7fc3000ac3b0 2024-03-30T01:43:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.788+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc2e406ac90 msgr2=0x7fc2e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.788+0000 7fc3084e5700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc2e406ac90 0x7fc2e406d150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0008030 comp rx=0 tx=0).stop 2024-03-30T01:43:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.788+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000ac3b0 msgr2=0x7fc3001edad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.788+0000 7fc3084e5700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000ac3b0 0x7fc3001edad0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00c450 tx=0x7fc2fc00c760 comp rx=0 tx=0).stop 2024-03-30T01:43:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.789+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 shutdown_connections 2024-03-30T01:43:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.789+0000 7fc3084e5700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3000ac3b0 0x7fc3001edad0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.789+0000 7fc3084e5700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fc2e406ac90 0x7fc2e406d150 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.789+0000 7fc3084e5700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc300151ce0 0x7fc3000c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.789+0000 7fc3084e5700 1 --2- 172.21.15.31:0/2040149547 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3000c4960 0x7fc3001476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.789+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 >> 172.21.15.31:0/2040149547 conn(0x7fc300072010 msgr2=0x7fc300149840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.789+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 shutdown_connections 2024-03-30T01:43:10.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.789+0000 7fc3084e5700 1 -- 172.21.15.31:0/2040149547 wait complete. 2024-03-30T01:43:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-03-30T01:43:10.812 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T01:43:10.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.953+0000 7f52b3a25700 1 Processor -- start 2024-03-30T01:43:10.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.953+0000 7f52b3a25700 1 -- start start 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.954+0000 7f52b3a25700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52ac0c46c0 0x7f52ac06ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.954+0000 7f52b3a25700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52ac0c4aa0 0x7f52ac06bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.954+0000 7f52b3a25700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ac070610 con 0x7f52ac0c5d10 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.954+0000 7f52b3a25700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac0708f0 con 0x7f52ac0c46c0 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.954+0000 7f52b3a25700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac06c1c0 con 0x7f52ac0c4aa0 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.954+0000 7f52b17c1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52ac0c46c0 0x7f52ac06ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.954+0000 7f52b0fc0700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52ac0c5d10 0x7f52ac0c26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50442/0 (socket says 172.21.15.31:50442) 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52b17c1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52ac0c46c0 0x7f52ac06ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33674/0 (socket says 172.21.15.31:33674) 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52b0fc0700 1 -- 172.21.15.31:0/2093677021 learned_addr learned my addr 172.21.15.31:0/2093677021 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52b1fc2700 1 --2- 172.21.15.31:0/2093677021 >> v2:172.21.15.59:3300/0 conn(0x7f52ac0c4aa0 0x7f52ac06bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52b17c1700 1 -- 172.21.15.31:0/2093677021 >> v2:172.21.15.59:3300/0 conn(0x7f52ac0c4aa0 msgr2=0x7f52ac06bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:10.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52b17c1700 1 --2- 172.21.15.31:0/2093677021 >> v2:172.21.15.59:3300/0 conn(0x7f52ac0c4aa0 0x7f52ac06bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:10.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52b17c1700 1 -- 172.21.15.31:0/2093677021 >> v1:172.21.15.31:6789/0 conn(0x7f52ac0c5d10 legacy=0x7f52ac0c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:10.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52b17c1700 1 -- 172.21.15.31:0/2093677021 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0c3210 con 0x7f52ac0c46c0 2024-03-30T01:43:10.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52b17c1700 1 --2- 172.21.15.31:0/2093677021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c46c0 0x7f52ac06ce80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0c2500 tx=0x7f52a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe6bf1d8c81641d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:10.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52a3fff700 1 -- 172.21.15.31:0/2093677021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8011830 con 0x7f52ac0c46c0 2024-03-30T01:43:10.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52a3fff700 1 -- 172.21.15.31:0/2093677021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a8011990 con 0x7f52ac0c46c0 2024-03-30T01:43:10.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.955+0000 7f52a3fff700 1 -- 172.21.15.31:0/2093677021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8011d20 con 0x7f52ac0c46c0 2024-03-30T01:43:10.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.956+0000 7f52b3a25700 1 -- 172.21.15.31:0/2093677021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c46c0 msgr2=0x7f52ac06ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:10.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.956+0000 7f52b3a25700 1 --2- 172.21.15.31:0/2093677021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c46c0 0x7f52ac06ce80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0c2500 tx=0x7f52a8009b20 comp rx=0 tx=0).stop 2024-03-30T01:43:10.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.956+0000 7f52b3a25700 1 -- 172.21.15.31:0/2093677021 shutdown_connections 2024-03-30T01:43:10.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.956+0000 7f52b3a25700 1 --2- 172.21.15.31:0/2093677021 >> v2:172.21.15.59:3300/0 conn(0x7f52ac0c4aa0 0x7f52ac06bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:10.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.956+0000 7f52b3a25700 1 --2- 172.21.15.31:0/2093677021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c46c0 0x7f52ac06ce80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:10.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.956+0000 7f52b3a25700 1 -- 172.21.15.31:0/2093677021 >> 172.21.15.31:0/2093677021 conn(0x7f52ac0c7cf0 msgr2=0x7f52ac0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:10.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.956+0000 7f52b3a25700 1 -- 172.21.15.31:0/2093677021 shutdown_connections 2024-03-30T01:43:10.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.956+0000 7f52b3a25700 1 -- 172.21.15.31:0/2093677021 wait complete. 2024-03-30T01:43:10.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.957+0000 7f52b3a25700 1 Processor -- start 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.957+0000 7f52b3a25700 1 -- start start 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b3a25700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac0c46c0 0x7f52ac15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b3a25700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac0c4aa0 0x7f52ac15aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b3a25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c5d10 0x7f52ac1edab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b3a25700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0a9450 con 0x7f52ac0c5d10 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b3a25700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac15b030 con 0x7f52ac0c4aa0 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b3a25700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac152e10 con 0x7f52ac0c46c0 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b1fc2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c5d10 0x7f52ac1edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b1fc2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c5d10 0x7f52ac1edab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33686/0 (socket says 172.21.15.31:33686) 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b1fc2700 1 -- 172.21.15.31:0/3154630128 learned_addr learned my addr 172.21.15.31:0/3154630128 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b17c1700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac0c46c0 0x7f52ac15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b0fc0700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac0c4aa0 0x7f52ac15aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b1fc2700 1 -- 172.21.15.31:0/3154630128 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac0c46c0 msgr2=0x7f52ac15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b1fc2700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac0c46c0 0x7f52ac15a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:10.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b1fc2700 1 -- 172.21.15.31:0/3154630128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac0c4aa0 msgr2=0x7f52ac15aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b1fc2700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac0c4aa0 0x7f52ac15aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b1fc2700 1 -- 172.21.15.31:0/3154630128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a80094a0 con 0x7f52ac0c5d10 2024-03-30T01:43:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.958+0000 7f52b0fc0700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac0c4aa0 0x7f52ac15aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.959+0000 7f52b1fc2700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c5d10 0x7f52ac1edab0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f52a4007e40 tx=0x7f52a400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.959+0000 7f52a1ffb700 1 -- 172.21.15.31:0/3154630128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400eea0 con 0x7f52ac0c5d10 2024-03-30T01:43:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.959+0000 7f52a1ffb700 1 -- 172.21.15.31:0/3154630128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4014420 con 0x7f52ac0c5d10 2024-03-30T01:43:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.959+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac14d990 con 0x7f52ac0c5d10 2024-03-30T01:43:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.959+0000 7f52a1ffb700 1 -- 172.21.15.31:0/3154630128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40147b0 con 0x7f52ac0c5d10 2024-03-30T01:43:10.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.959+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac14df50 con 0x7f52ac0c5d10 2024-03-30T01:43:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.960+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac0747d0 con 0x7f52ac0c5d10 2024-03-30T01:43:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.961+0000 7f52a1ffb700 1 -- 172.21.15.31:0/3154630128 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f52a4014910 con 0x7f52ac0c5d10 2024-03-30T01:43:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.961+0000 7f52a1ffb700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.961+0000 7f52a1ffb700 1 -- 172.21.15.31:0/3154630128 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f52a40899a0 con 0x7f52ac0c5d10 2024-03-30T01:43:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.961+0000 7f52b17c1700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:10.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.962+0000 7f52b17c1700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f52a80090b0 tx=0x7f52a8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:10.966+0000 7f52a1ffb700 1 -- 172.21.15.31:0/3154630128 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f52a4054cb0 con 0x7f52ac0c5d10 2024-03-30T01:43:11.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.183+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f52ac0c83b0 con 0x7f52ac0c5d10 2024-03-30T01:43:11.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.786+0000 7f52a1ffb700 1 -- 172.21.15.31:0/3154630128 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7f52a40582d0 con 0x7f52ac0c5d10 2024-03-30T01:43:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.795+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 msgr2=0x7f529006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.795+0000 7f52b3a25700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f52a80090b0 tx=0x7f52a8008030 comp rx=0 tx=0).stop 2024-03-30T01:43:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.795+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c5d10 msgr2=0x7f52ac1edab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.795+0000 7f52b3a25700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c5d10 0x7f52ac1edab0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f52a4007e40 tx=0x7f52a400e3c0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.798+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 shutdown_connections 2024-03-30T01:43:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.798+0000 7f52b3a25700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52ac0c5d10 0x7f52ac1edab0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.798+0000 7f52b3a25700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f529006ac90 0x7f529006d150 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.798+0000 7f52b3a25700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52ac0c46c0 0x7f52ac15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.798+0000 7f52b3a25700 1 --2- 172.21.15.31:0/3154630128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac0c4aa0 0x7f52ac15aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.798+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 >> 172.21.15.31:0/3154630128 conn(0x7f52ac0c7cf0 msgr2=0x7f52ac0c6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.799+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 shutdown_connections 2024-03-30T01:43:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.800+0000 7f52b3a25700 1 -- 172.21.15.31:0/3154630128 wait complete. 2024-03-30T01:43:11.812 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-03-30T01:43:11.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T01:43:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.964+0000 7fef068ed700 1 Processor -- start 2024-03-30T01:43:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.964+0000 7fef068ed700 1 -- start start 2024-03-30T01:43:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef068ed700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 0x7fef000c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef068ed700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fef001525e0 0x7fef00155a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef068ed700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fef000a91c0 0x7fef0006f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef068ed700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef000aa330 con 0x7fef000a91c0 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef068ed700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef00156930 con 0x7fef001525e0 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef068ed700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef00156370 con 0x7fef0014fce0 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef04e8a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fef000a91c0 0x7fef0006f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef04e8a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fef000a91c0 0x7fef0006f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33692/0 (socket says 172.21.15.31:33692) 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef04e8a700 1 -- 172.21.15.31:0/2322785077 learned_addr learned my addr 172.21.15.31:0/2322785077 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7feefffff700 1 --2- 172.21.15.31:0/2322785077 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 0x7fef000c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7feeff7fe700 1 --2- 172.21.15.31:0/2322785077 >> v2:172.21.15.59:3300/0 conn(0x7fef001525e0 0x7fef00155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef04e8a700 1 -- 172.21.15.31:0/2322785077 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 msgr2=0x7fef000c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef04e8a700 1 --2- 172.21.15.31:0/2322785077 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 0x7fef000c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef04e8a700 1 -- 172.21.15.31:0/2322785077 >> v2:172.21.15.59:3300/0 conn(0x7fef001525e0 msgr2=0x7fef00155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef04e8a700 1 --2- 172.21.15.31:0/2322785077 >> v2:172.21.15.59:3300/0 conn(0x7fef001525e0 0x7fef00155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7fef04e8a700 1 -- 172.21.15.31:0/2322785077 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00156650 con 0x7fef000a91c0 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7feeff7fe700 1 --2- 172.21.15.31:0/2322785077 >> v2:172.21.15.59:3300/0 conn(0x7fef001525e0 0x7fef00155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.965+0000 7feefffff700 1 --2- 172.21.15.31:0/2322785077 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 0x7fef000c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef04e8a700 1 --2- 172.21.15.31:0/2322785077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 0x7fef0006f330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fef000c3870 tx=0x7feef800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5c7de9b12e4522 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7feefeffd700 1 -- 172.21.15.31:0/2322785077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef800e070 con 0x7fef000a91c0 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7feefeffd700 1 -- 172.21.15.31:0/2322785077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef800b9c0 con 0x7fef000a91c0 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7feefeffd700 1 -- 172.21.15.31:0/2322785077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef800bd70 con 0x7fef000a91c0 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef068ed700 1 -- 172.21.15.31:0/2322785077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 msgr2=0x7fef0006f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef068ed700 1 --2- 172.21.15.31:0/2322785077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 0x7fef0006f330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fef000c3870 tx=0x7feef800b330 comp rx=0 tx=0).stop 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef068ed700 1 -- 172.21.15.31:0/2322785077 shutdown_connections 2024-03-30T01:43:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef068ed700 1 --2- 172.21.15.31:0/2322785077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 0x7fef0006f330 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef068ed700 1 --2- 172.21.15.31:0/2322785077 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 0x7fef000c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef068ed700 1 --2- 172.21.15.31:0/2322785077 >> v2:172.21.15.59:3300/0 conn(0x7fef001525e0 0x7fef00155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef068ed700 1 -- 172.21.15.31:0/2322785077 >> 172.21.15.31:0/2322785077 conn(0x7fef000c3910 msgr2=0x7fef000c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef068ed700 1 -- 172.21.15.31:0/2322785077 shutdown_connections 2024-03-30T01:43:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.966+0000 7fef068ed700 1 -- 172.21.15.31:0/2322785077 wait complete. 2024-03-30T01:43:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.968+0000 7fef068ed700 1 Processor -- start 2024-03-30T01:43:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.968+0000 7fef068ed700 1 -- start start 2024-03-30T01:43:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.968+0000 7fef068ed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 0x7fef001ed600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.968+0000 7fef068ed700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 0x7fef0014e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.968+0000 7fef068ed700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef001525e0 0x7fef001423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.968+0000 7fef068ed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef001edb40 con 0x7fef000a91c0 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.968+0000 7fef068ed700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef000c6760 con 0x7fef001525e0 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7fef068ed700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef00155920 con 0x7fef0014fce0 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 0x7fef001ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 0x7fef001ed600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33704/0 (socket says 172.21.15.31:33704) 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefffff700 1 -- 172.21.15.31:0/4263161822 learned_addr learned my addr 172.21.15.31:0/4263161822 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feeff7fe700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 0x7fef0014e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7fef04e8a700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef001525e0 0x7fef001423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefffff700 1 -- 172.21.15.31:0/4263161822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 msgr2=0x7fef0014e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefffff700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 0x7fef0014e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefffff700 1 -- 172.21.15.31:0/4263161822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef001525e0 msgr2=0x7fef001423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefffff700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef001525e0 0x7fef001423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefffff700 1 -- 172.21.15.31:0/4263161822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef8009b30 con 0x7fef000a91c0 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefffff700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 0x7fef001ed600 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7feef000c740 tx=0x7feef000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefcff9700 1 -- 172.21.15.31:0/4263161822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0011800 con 0x7fef000a91c0 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefcff9700 1 -- 172.21.15.31:0/4263161822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0011960 con 0x7fef000a91c0 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef00155640 con 0x7fef000a91c0 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7feefcff9700 1 -- 172.21.15.31:0/4263161822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0011cd0 con 0x7fef000a91c0 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.969+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef00154b00 con 0x7fef000a91c0 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.970+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef00066f90 con 0x7fef000a91c0 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.971+0000 7feefcff9700 1 -- 172.21.15.31:0/4263161822 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feef0023480 con 0x7fef000a91c0 2024-03-30T01:43:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.971+0000 7feefcff9700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7feee006ac90 0x7feee006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.972+0000 7feefcff9700 1 -- 172.21.15.31:0/4263161822 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5929+0+0 (secure 0 0 0) 0x7feef00897d0 con 0x7fef000a91c0 2024-03-30T01:43:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.972+0000 7feeff7fe700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7feee006ac90 0x7feee006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.972+0000 7feeff7fe700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7feee006ac90 0x7feee006d150 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feee8006fd0 tx=0x7feee8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:11.976+0000 7feefcff9700 1 -- 172.21.15.31:0/4263161822 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feef00548e0 con 0x7fef000a91c0 2024-03-30T01:43:12.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.243+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fef000727b0 con 0x7fef000a91c0 2024-03-30T01:43:12.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.249+0000 7feefcff9700 1 -- 172.21.15.31:0/4263161822 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7feef0057f00 con 0x7fef000a91c0 2024-03-30T01:43:12.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.253+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7feee006ac90 msgr2=0x7feee006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.253+0000 7fef068ed700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7feee006ac90 0x7feee006d150 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feee8006fd0 tx=0x7feee8008040 comp rx=0 tx=0).stop 2024-03-30T01:43:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.253+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 msgr2=0x7fef001ed600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.253+0000 7fef068ed700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 0x7fef001ed600 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7feef000c740 tx=0x7feef000cb00 comp rx=0 tx=0).stop 2024-03-30T01:43:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.255+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 shutdown_connections 2024-03-30T01:43:12.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.255+0000 7fef068ed700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef000a91c0 0x7fef001ed600 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.255+0000 7fef068ed700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7feee006ac90 0x7feee006d150 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feee8006fd0 tx=0x7feee8008040 comp rx=0 tx=0).stop 2024-03-30T01:43:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.255+0000 7fef068ed700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef001525e0 0x7fef001423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.255+0000 7fef068ed700 1 --2- 172.21.15.31:0/4263161822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef0014fce0 0x7fef0014e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.255+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 >> 172.21.15.31:0/4263161822 conn(0x7fef000c3910 msgr2=0x7fef00071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:12.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.263+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 shutdown_connections 2024-03-30T01:43:12.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.263+0000 7fef068ed700 1 -- 172.21.15.31:0/4263161822 wait complete. 2024-03-30T01:43:12.274 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 9 and data pool 10 2024-03-30T01:43:12.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:12.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.421+0000 7fde3a044700 1 Processor -- start 2024-03-30T01:43:12.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.421+0000 7fde3a044700 1 -- start start 2024-03-30T01:43:12.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde3a044700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340ac520 0x7fde340ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde3a044700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde340a74e0 0x7fde340ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde3a044700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde340c6000 0x7fde340c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde3a044700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde340ab680 con 0x7fde340c6000 2024-03-30T01:43:12.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde3a044700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde341500c0 con 0x7fde340a74e0 2024-03-30T01:43:12.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde3a044700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde34153590 con 0x7fde340ac520 2024-03-30T01:43:12.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde33fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde340c6000 0x7fde340c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde33fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde340c6000 0x7fde340c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33716/0 (socket says 172.21.15.31:33716) 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde33fff700 1 -- 172.21.15.31:0/22663626 learned_addr learned my addr 172.21.15.31:0/22663626 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde337fe700 1 --2- 172.21.15.31:0/22663626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340ac520 0x7fde340ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde32ffd700 1 --2- 172.21.15.31:0/22663626 >> v2:172.21.15.59:3300/0 conn(0x7fde340a74e0 0x7fde340ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde33fff700 1 -- 172.21.15.31:0/22663626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340ac520 msgr2=0x7fde340ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde33fff700 1 --2- 172.21.15.31:0/22663626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340ac520 0x7fde340ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde33fff700 1 -- 172.21.15.31:0/22663626 >> v2:172.21.15.59:3300/0 conn(0x7fde340a74e0 msgr2=0x7fde340ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde33fff700 1 --2- 172.21.15.31:0/22663626 >> v2:172.21.15.59:3300/0 conn(0x7fde340a74e0 0x7fde340ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde33fff700 1 -- 172.21.15.31:0/22663626 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde340c7990 con 0x7fde340c6000 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde337fe700 1 --2- 172.21.15.31:0/22663626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340ac520 0x7fde340ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde32ffd700 1 --2- 172.21.15.31:0/22663626 >> v2:172.21.15.59:3300/0 conn(0x7fde340a74e0 0x7fde340ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde33fff700 1 --2- 172.21.15.31:0/22663626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340c6000 0x7fde340c6490 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fde3406b680 tx=0x7fde2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17bd7ad2d2da8d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.422+0000 7fde327fc700 1 -- 172.21.15.31:0/22663626 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00bbc0 con 0x7fde340c6000 2024-03-30T01:43:12.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde327fc700 1 -- 172.21.15.31:0/22663626 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c00bd20 con 0x7fde340c6000 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde327fc700 1 -- 172.21.15.31:0/22663626 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c014b10 con 0x7fde340c6000 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde3a044700 1 -- 172.21.15.31:0/22663626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340c6000 msgr2=0x7fde340c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde3a044700 1 --2- 172.21.15.31:0/22663626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340c6000 0x7fde340c6490 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fde3406b680 tx=0x7fde2c00b330 comp rx=0 tx=0).stop 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde3a044700 1 -- 172.21.15.31:0/22663626 shutdown_connections 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde3a044700 1 --2- 172.21.15.31:0/22663626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340c6000 0x7fde340c6490 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde3a044700 1 --2- 172.21.15.31:0/22663626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340ac520 0x7fde340ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde3a044700 1 --2- 172.21.15.31:0/22663626 >> v2:172.21.15.59:3300/0 conn(0x7fde340a74e0 0x7fde340ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde3a044700 1 -- 172.21.15.31:0/22663626 >> 172.21.15.31:0/22663626 conn(0x7fde3406ad30 msgr2=0x7fde340a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde3a044700 1 -- 172.21.15.31:0/22663626 shutdown_connections 2024-03-30T01:43:12.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.423+0000 7fde3a044700 1 -- 172.21.15.31:0/22663626 wait complete. 2024-03-30T01:43:12.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.424+0000 7fde3a044700 1 Processor -- start 2024-03-30T01:43:12.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde3a044700 1 -- start start 2024-03-30T01:43:12.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde3a044700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340a74e0 0x7fde341e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde3a044700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde340ac520 0x7fde341516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde3a044700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340c6000 0x7fde3414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde3a044700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde341e9ad0 con 0x7fde340a74e0 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde3a044700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde340a8830 con 0x7fde340ac520 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde3a044700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde340a8a80 con 0x7fde340c6000 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340a74e0 0x7fde341e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340a74e0 0x7fde341e5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33718/0 (socket says 172.21.15.31:33718) 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.425+0000 7fde337fe700 1 -- 172.21.15.31:0/1867848015 learned_addr learned my addr 172.21.15.31:0/1867848015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde33fff700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340c6000 0x7fde3414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde32ffd700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde340ac520 0x7fde341516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde337fe700 1 -- 172.21.15.31:0/1867848015 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340c6000 msgr2=0x7fde3414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde337fe700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340c6000 0x7fde3414d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde337fe700 1 -- 172.21.15.31:0/1867848015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde340ac520 msgr2=0x7fde341516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde337fe700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde340ac520 0x7fde341516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde337fe700 1 -- 172.21.15.31:0/1867848015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c009b30 con 0x7fde340a74e0 2024-03-30T01:43:12.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde337fe700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340a74e0 0x7fde341e5fa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fde2400e840 tx=0x7fde2400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde1bfff700 1 -- 172.21.15.31:0/1867848015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24004230 con 0x7fde340a74e0 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde1bfff700 1 -- 172.21.15.31:0/1867848015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde24004390 con 0x7fde340a74e0 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde340a8d60 con 0x7fde340a74e0 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde1bfff700 1 -- 172.21.15.31:0/1867848015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde240036a0 con 0x7fde340a74e0 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.426+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde340a7c30 con 0x7fde340a74e0 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.428+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde341e8c90 con 0x7fde340a74e0 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.428+0000 7fde1bfff700 1 -- 172.21.15.31:0/1867848015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fde24003800 con 0x7fde340a74e0 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.429+0000 7fde1bfff700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fde2006ac90 0x7fde2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.429+0000 7fde1bfff700 1 -- 172.21.15.31:0/1867848015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fde24088d80 con 0x7fde340a74e0 2024-03-30T01:43:12.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.429+0000 7fde32ffd700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fde2006ac90 0x7fde2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.429+0000 7fde32ffd700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fde2006ac90 0x7fde2006d150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fde2800db20 tx=0x7fde2800f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:12.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.433+0000 7fde1bfff700 1 -- 172.21.15.31:0/1867848015 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fde24053eb0 con 0x7fde340a74e0 2024-03-30T01:43:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.649+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde3404ea80 con 0x7fde340a74e0 2024-03-30T01:43:12.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.650+0000 7fde1bfff700 1 -- 172.21.15.31:0/1867848015 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16213 (secure 0 0 0) 0x7fde24020090 con 0x7fde340a74e0 2024-03-30T01:43:12.657 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:12.657 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":59,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:12.245026+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:43:09.911033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T01:43:11.216248+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fde2006ac90 msgr2=0x7fde2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fde2006ac90 0x7fde2006d150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fde2800db20 tx=0x7fde2800f040 comp rx=0 tx=0).stop 2024-03-30T01:43:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340a74e0 msgr2=0x7fde341e5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340a74e0 0x7fde341e5fa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fde2400e840 tx=0x7fde2400eb50 comp rx=0 tx=0).stop 2024-03-30T01:43:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 shutdown_connections 2024-03-30T01:43:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde340a74e0 0x7fde341e5fa0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fde2006ac90 0x7fde2006d150 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde340c6000 0x7fde3414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 --2- 172.21.15.31:0/1867848015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde340ac520 0x7fde341516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 >> 172.21.15.31:0/1867848015 conn(0x7fde3406ad30 msgr2=0x7fde3406faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 shutdown_connections 2024-03-30T01:43:12.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.652+0000 7fde3a044700 1 -- 172.21.15.31:0/1867848015 wait complete. 2024-03-30T01:43:12.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.802+0000 7f9e0fb70700 1 Processor -- start 2024-03-30T01:43:12.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.802+0000 7f9e0fb70700 1 -- start start 2024-03-30T01:43:12.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.803+0000 7f9e0fb70700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9e080ac520 0x7f9e080ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.803+0000 7f9e0fb70700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9e080a74e0 0x7f9e080ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.803+0000 7f9e0fb70700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080c6000 0x7f9e080c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.803+0000 7f9e0fb70700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e080ab680 con 0x7f9e080a74e0 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.803+0000 7f9e0fb70700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e081500c0 con 0x7f9e080ac520 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.803+0000 7f9e0fb70700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08153590 con 0x7f9e080c6000 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.803+0000 7f9e0d10b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9e080a74e0 0x7f9e080ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.803+0000 7f9e0d10b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9e080a74e0 0x7f9e080ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33720/0 (socket says 172.21.15.31:33720) 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.803+0000 7f9e0d10b700 1 -- 172.21.15.31:0/3070866857 learned_addr learned my addr 172.21.15.31:0/3070866857 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0e10d700 1 --2- 172.21.15.31:0/3070866857 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080c6000 0x7f9e080c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0d10b700 1 -- 172.21.15.31:0/3070866857 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080c6000 msgr2=0x7f9e080c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0d90c700 1 --2- 172.21.15.31:0/3070866857 >> v2:172.21.15.59:3300/0 conn(0x7f9e080ac520 0x7f9e080ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0d10b700 1 --2- 172.21.15.31:0/3070866857 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080c6000 0x7f9e080c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0d10b700 1 -- 172.21.15.31:0/3070866857 >> v2:172.21.15.59:3300/0 conn(0x7f9e080ac520 msgr2=0x7f9e080ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0d10b700 1 --2- 172.21.15.31:0/3070866857 >> v2:172.21.15.59:3300/0 conn(0x7f9e080ac520 0x7f9e080ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0d10b700 1 -- 172.21.15.31:0/3070866857 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080c7990 con 0x7f9e080a74e0 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0d90c700 1 --2- 172.21.15.31:0/3070866857 >> v2:172.21.15.59:3300/0 conn(0x7f9e080ac520 0x7f9e080ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0d10b700 1 --2- 172.21.15.31:0/3070866857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 0x7f9e080ace40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9e0806b750 tx=0x7f9df8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b407e3bc166865 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0c90a700 1 -- 172.21.15.31:0/3070866857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800b870 con 0x7f9e080a74e0 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0c90a700 1 -- 172.21.15.31:0/3070866857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df800b9d0 con 0x7f9e080a74e0 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.804+0000 7f9e0c90a700 1 -- 172.21.15.31:0/3070866857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800bd60 con 0x7f9e080a74e0 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.805+0000 7f9e0fb70700 1 -- 172.21.15.31:0/3070866857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 msgr2=0x7f9e080ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.805+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/3070866857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 0x7f9e080ace40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9e0806b750 tx=0x7f9df8009bd0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.805+0000 7f9e0fb70700 1 -- 172.21.15.31:0/3070866857 shutdown_connections 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.805+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/3070866857 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080c6000 0x7f9e080c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.805+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/3070866857 >> v2:172.21.15.59:3300/0 conn(0x7f9e080ac520 0x7f9e080ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.805+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/3070866857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 0x7f9e080ace40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.805+0000 7f9e0fb70700 1 -- 172.21.15.31:0/3070866857 >> 172.21.15.31:0/3070866857 conn(0x7f9e0806ad30 msgr2=0x7f9e080a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.805+0000 7f9e0fb70700 1 -- 172.21.15.31:0/3070866857 shutdown_connections 2024-03-30T01:43:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.805+0000 7f9e0fb70700 1 -- 172.21.15.31:0/3070866857 wait complete. 2024-03-30T01:43:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.806+0000 7f9e0fb70700 1 Processor -- start 2024-03-30T01:43:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.807+0000 7f9e0fb70700 1 -- start start 2024-03-30T01:43:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.807+0000 7f9e0fb70700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 0x7f9e08155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.807+0000 7f9e0fb70700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080ac520 0x7f9e081514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.807+0000 7f9e0fb70700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080c6000 0x7f9e0814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0fb70700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e081e8cf0 con 0x7f9e080a74e0 2024-03-30T01:43:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0fb70700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08156a60 con 0x7f9e080c6000 2024-03-30T01:43:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0fb70700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080bdd70 con 0x7f9e080ac520 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d90c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 0x7f9e08155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d90c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 0x7f9e08155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33724/0 (socket says 172.21.15.31:33724) 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d90c700 1 -- 172.21.15.31:0/1266747965 learned_addr learned my addr 172.21.15.31:0/1266747965 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d10b700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080ac520 0x7f9e081514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0e10d700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080c6000 0x7f9e0814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d90c700 1 -- 172.21.15.31:0/1266747965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080ac520 msgr2=0x7f9e081514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d90c700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080ac520 0x7f9e081514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d90c700 1 -- 172.21.15.31:0/1266747965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080c6000 msgr2=0x7f9e0814c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d90c700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080c6000 0x7f9e0814c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d90c700 1 -- 172.21.15.31:0/1266747965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8009550 con 0x7f9e080a74e0 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0e10d700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080c6000 0x7f9e0814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0d90c700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 0x7f9e08155710 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400c770 tx=0x7f9e0400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9dfe7fc700 1 -- 172.21.15.31:0/1266747965 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04011800 con 0x7f9e080a74e0 2024-03-30T01:43:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9dfe7fc700 1 -- 172.21.15.31:0/1266747965 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e04011960 con 0x7f9e080a74e0 2024-03-30T01:43:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e08155280 con 0x7f9e080a74e0 2024-03-30T01:43:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9dfe7fc700 1 -- 172.21.15.31:0/1266747965 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04011cd0 con 0x7f9e080a74e0 2024-03-30T01:43:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.808+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e08156780 con 0x7f9e080a74e0 2024-03-30T01:43:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.809+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e08155050 con 0x7f9e080a74e0 2024-03-30T01:43:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.810+0000 7f9dfe7fc700 1 -- 172.21.15.31:0/1266747965 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9e04021350 con 0x7f9e080a74e0 2024-03-30T01:43:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.810+0000 7f9dfe7fc700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9dec06ac90 0x7f9dec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.811+0000 7f9dfe7fc700 1 -- 172.21.15.31:0/1266747965 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9e04089280 con 0x7f9e080a74e0 2024-03-30T01:43:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.811+0000 7f9e0d10b700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9dec06ac90 0x7f9dec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:12.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.811+0000 7f9e0d10b700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9dec06ac90 0x7f9dec06d150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000c00 tx=0x7f9df8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:12.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:12.815+0000 7f9dfe7fc700 1 -- 172.21.15.31:0/1266747965 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9e08155050 con 0x7f9e080a74e0 2024-03-30T01:43:13.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.036+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e08155560 con 0x7f9e080a74e0 2024-03-30T01:43:13.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.037+0000 7f9dfe7fc700 1 -- 172.21.15.31:0/1266747965 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16213 (secure 0 0 0) 0x7f9e04057e40 con 0x7f9e080a74e0 2024-03-30T01:43:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:13.044 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":59,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:12.245026+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:43:09.911033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T01:43:11.216248+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:13.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9dec06ac90 msgr2=0x7f9dec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9dec06ac90 0x7f9dec06d150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000c00 tx=0x7f9df8008030 comp rx=0 tx=0).stop 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 msgr2=0x7f9e08155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 0x7f9e08155710 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400c770 tx=0x7f9e0400ca80 comp rx=0 tx=0).stop 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 shutdown_connections 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e080a74e0 0x7f9e08155710 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9dec06ac90 0x7f9dec06d150 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e080c6000 0x7f9e0814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 --2- 172.21.15.31:0/1266747965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e080ac520 0x7f9e081514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 >> 172.21.15.31:0/1266747965 conn(0x7f9e0806ad30 msgr2=0x7f9e08070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 shutdown_connections 2024-03-30T01:43:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.040+0000 7f9e0fb70700 1 -- 172.21.15.31:0/1266747965 wait complete. 2024-03-30T01:43:13.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T01:43:13.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.193+0000 7f15d98f7700 1 Processor -- start 2024-03-30T01:43:13.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.194+0000 7f15d98f7700 1 -- start start 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.194+0000 7f15d98f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 0x7f15d41546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.194+0000 7f15d98f7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15d406c3f0 0x7f15d406c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.194+0000 7f15d98f7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d406bc80 con 0x7f15d40c4c90 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.194+0000 7f15d98f7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d406cdc0 con 0x7f15d406c3f0 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.194+0000 7f15d98f7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40c5110 con 0x7f15d40c48b0 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15d406c3f0 0x7f15d406c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d37fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15d406c3f0 0x7f15d406c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33730/0 (socket says 172.21.15.31:33730) 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d37fe700 1 -- 172.21.15.31:0/3927906319 learned_addr learned my addr 172.21.15.31:0/3927906319 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d2ffd700 1 --2- 172.21.15.31:0/3927906319 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 0x7f15d41546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:13.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d37fe700 1 -- 172.21.15.31:0/3927906319 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 msgr2=0x7f15d41546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d37fe700 1 --2- 172.21.15.31:0/3927906319 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 0x7f15d41546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d37fe700 1 -- 172.21.15.31:0/3927906319 >> v1:172.21.15.59:6789/0 conn(0x7f15d40c4c90 legacy=0x7f15d40c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d37fe700 1 -- 172.21.15.31:0/3927906319 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40c7120 con 0x7f15d406c3f0 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d37fe700 1 --2- 172.21.15.31:0/3927906319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d406c3f0 0x7f15d406c880 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009e50 tx=0x7f15cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9479876f1b7156ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d1ffb700 1 -- 172.21.15.31:0/3927906319 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00e070 con 0x7f15d406c3f0 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d1ffb700 1 -- 172.21.15.31:0/3927906319 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc00b9c0 con 0x7f15d406c3f0 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.195+0000 7f15d1ffb700 1 -- 172.21.15.31:0/3927906319 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00bd70 con 0x7f15d406c3f0 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.196+0000 7f15d98f7700 1 -- 172.21.15.31:0/3927906319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d406c3f0 msgr2=0x7f15d406c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.196+0000 7f15d98f7700 1 --2- 172.21.15.31:0/3927906319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d406c3f0 0x7f15d406c880 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009e50 tx=0x7f15cc00b330 comp rx=0 tx=0).stop 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.196+0000 7f15d98f7700 1 -- 172.21.15.31:0/3927906319 shutdown_connections 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.196+0000 7f15d98f7700 1 --2- 172.21.15.31:0/3927906319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d406c3f0 0x7f15d406c880 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.196+0000 7f15d98f7700 1 --2- 172.21.15.31:0/3927906319 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 0x7f15d41546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.196+0000 7f15d98f7700 1 -- 172.21.15.31:0/3927906319 >> 172.21.15.31:0/3927906319 conn(0x7f15d40abac0 msgr2=0x7f15d40abed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:13.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.196+0000 7f15d98f7700 1 -- 172.21.15.31:0/3927906319 shutdown_connections 2024-03-30T01:43:13.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.196+0000 7f15d98f7700 1 -- 172.21.15.31:0/3927906319 wait complete. 2024-03-30T01:43:13.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.197+0000 7f15d98f7700 1 Processor -- start 2024-03-30T01:43:13.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.197+0000 7f15d98f7700 1 -- start start 2024-03-30T01:43:13.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d98f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d406c3f0 0x7f15d40adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:13.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d98f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 0x7f15d40adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:13.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d98f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d40c4c90 0x7f15d41e7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:13.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d98f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d41ed640 con 0x7f15d40c4c90 2024-03-30T01:43:13.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d98f7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d41ee2b0 con 0x7f15d406c3f0 2024-03-30T01:43:13.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d98f7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40acce0 con 0x7f15d40c48b0 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d40c4c90 0x7f15d41e7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d40c4c90 0x7f15d41e7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33736/0 (socket says 172.21.15.31:33736) 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d37fe700 1 -- 172.21.15.31:0/351862494 learned_addr learned my addr 172.21.15.31:0/351862494 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d2ffd700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d406c3f0 0x7f15d40adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.198+0000 7f15d27fc700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 0x7f15d40adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15d37fe700 1 -- 172.21.15.31:0/351862494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 msgr2=0x7f15d40adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15d37fe700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 0x7f15d40adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15d37fe700 1 -- 172.21.15.31:0/351862494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d406c3f0 msgr2=0x7f15d40adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15d37fe700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d406c3f0 0x7f15d40adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15d37fe700 1 -- 172.21.15.31:0/351862494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc009b30 con 0x7f15d40c4c90 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15d37fe700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d40c4c90 0x7f15d41e7000 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f15d406d9a0 tx=0x7f15cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15cbfff700 1 -- 172.21.15.31:0/351862494 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc004840 con 0x7f15d40c4c90 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15cbfff700 1 -- 172.21.15.31:0/351862494 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc0099b0 con 0x7f15d40c4c90 2024-03-30T01:43:13.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15cbfff700 1 -- 172.21.15.31:0/351862494 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc01c680 con 0x7f15d40c4c90 2024-03-30T01:43:13.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d40acfc0 con 0x7f15d40c4c90 2024-03-30T01:43:13.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.199+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d40ad580 con 0x7f15d40c4c90 2024-03-30T01:43:13.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.200+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d40747d0 con 0x7f15d40c4c90 2024-03-30T01:43:13.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.201+0000 7f15cbfff700 1 -- 172.21.15.31:0/351862494 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f15cc01c7e0 con 0x7f15d40c4c90 2024-03-30T01:43:13.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.202+0000 7f15cbfff700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15b406ac90 0x7f15b406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:13.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.202+0000 7f15cbfff700 1 -- 172.21.15.31:0/351862494 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f15cc08b550 con 0x7f15d40c4c90 2024-03-30T01:43:13.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.202+0000 7f15d2ffd700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15b406ac90 0x7f15b406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:13.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.202+0000 7f15d2ffd700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15b406ac90 0x7f15b406d150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006b20 tx=0x7f15c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:13.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.206+0000 7f15cbfff700 1 -- 172.21.15.31:0/351862494 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f15cc056660 con 0x7f15d40c4c90 2024-03-30T01:43:13.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:13.471+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f15d40c7e30 con 0x7f15d40c4c90 2024-03-30T01:43:14.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.262+0000 7f15cbfff700 1 -- 172.21.15.31:0/351862494 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f15cc059c80 con 0x7f15d40c4c90 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15b406ac90 msgr2=0x7f15b406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15b406ac90 0x7f15b406d150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006b20 tx=0x7f15c4008040 comp rx=0 tx=0).stop 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d40c4c90 msgr2=0x7f15d41e7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d40c4c90 0x7f15d41e7000 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f15d406d9a0 tx=0x7f15cc004ad0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 shutdown_connections 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15d40c4c90 0x7f15d41e7000 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f15b406ac90 0x7f15b406d150 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d406c3f0 0x7f15d40adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 --2- 172.21.15.31:0/351862494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d40c48b0 0x7f15d40adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 >> 172.21.15.31:0/351862494 conn(0x7f15d40abac0 msgr2=0x7f15d4149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 shutdown_connections 2024-03-30T01:43:14.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.265+0000 7f15d98f7700 1 -- 172.21.15.31:0/351862494 wait complete. 2024-03-30T01:43:14.287 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T01:43:14.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T01:43:14.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.423+0000 7f70c4ceb700 1 Processor -- start 2024-03-30T01:43:14.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.424+0000 7f70c4ceb700 1 -- start start 2024-03-30T01:43:14.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70c4ceb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70c0151ea0 0x7f70c00c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:14.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70c4ceb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c00c07f0 0x7f70c006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:14.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70c4ceb700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c0155000 con 0x7f70c00721f0 2024-03-30T01:43:14.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70c4ceb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c006e930 con 0x7f70c0151ea0 2024-03-30T01:43:14.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70c4ceb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c006df40 con 0x7f70c00c07f0 2024-03-30T01:43:14.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bdd9b700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f70c00721f0 0x7f70c006a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50450/0 (socket says 172.21.15.31:50450) 2024-03-30T01:43:14.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bdd9b700 1 -- 172.21.15.31:0/2792954765 learned_addr learned my addr 172.21.15.31:0/2792954765 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:14.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bed9d700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c00c07f0 0x7f70c006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70be59c700 1 --2- 172.21.15.31:0/2792954765 >> v2:172.21.15.59:3300/0 conn(0x7f70c0151ea0 0x7f70c00c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549948239 0 0) 0x7f70c0155000 con 0x7f70c00721f0 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a80035b0 con 0x7f70c00721f0 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1819044012 0 0) 0x7f70a80035b0 con 0x7f70c00721f0 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c00c07f0 msgr2=0x7f70c006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c00c07f0 0x7f70c006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 >> v2:172.21.15.59:3300/0 conn(0x7f70c0151ea0 msgr2=0x7f70c00c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 --2- 172.21.15.31:0/2792954765 >> v2:172.21.15.59:3300/0 conn(0x7f70c0151ea0 0x7f70c00c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c006e650 con 0x7f70c00721f0 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f70b4001670 con 0x7f70c00721f0 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 >> v1:172.21.15.31:6789/0 conn(0x7f70c00721f0 legacy=0x7f70c006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c00c07f0 0x7f70a8003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.425+0000 7f70bd59a700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bd59a700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70a8004d90 0x7f70a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c006e930 con 0x7f70a8004060 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a800bac0 con 0x7f70a8004d90 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a800bc80 con 0x7f70c00c07f0 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bdd9b700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70be59c700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c00c07f0 0x7f70a8003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bed9d700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70a8004d90 0x7f70a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bdd9b700 1 -- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c00c07f0 msgr2=0x7f70a8003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bdd9b700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c00c07f0 0x7f70a8003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bdd9b700 1 -- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70a8004d90 msgr2=0x7f70a8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bdd9b700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70a8004d90 0x7f70a8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bdd9b700 1 -- 172.21.15.31:0/2792954765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a80035b0 con 0x7f70a8004060 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bdd9b700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70a8004850 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c370 tx=0x7f70a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4c7e67d2cc20be server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4007c60 con 0x7f70a8004060 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b4005960 con 0x7f70a8004060 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.426+0000 7f70bd59a700 1 -- 172.21.15.31:0/2792954765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4005d10 con 0x7f70a8004060 2024-03-30T01:43:14.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.427+0000 7f70c4ceb700 1 -- 172.21.15.31:0/2792954765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 msgr2=0x7f70a8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.427+0000 7f70c4ceb700 1 --2- 172.21.15.31:0/2792954765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70a8004850 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c370 tx=0x7f70a8005880 comp rx=0 tx=0).stop 2024-03-30T01:43:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.427+0000 7f70bed9d700 1 -- 172.21.15.31:0/2792954765 reap_dead start 2024-03-30T01:43:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.427+0000 7f70c4ceb700 1 -- 172.21.15.31:0/2792954765 shutdown_connections 2024-03-30T01:43:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.427+0000 7f70c4ceb700 1 -- 172.21.15.31:0/2792954765 >> 172.21.15.31:0/2792954765 conn(0x7f70c00c3980 msgr2=0x7f70c00c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.427+0000 7f70c4ceb700 1 -- 172.21.15.31:0/2792954765 shutdown_connections 2024-03-30T01:43:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.427+0000 7f70c4ceb700 1 -- 172.21.15.31:0/2792954765 wait complete. 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.429+0000 7f70c4ceb700 1 Processor -- start 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.429+0000 7f70c4ceb700 1 -- start start 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.429+0000 7f70c4ceb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70c0153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.429+0000 7f70c4ceb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70c01544f0 0x7f70c0144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.429+0000 7f70c4ceb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c01537b0 0x7f70c01eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.429+0000 7f70c4ceb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c01ea400 con 0x7f70a8004060 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.429+0000 7f70c4ceb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c01526b0 con 0x7f70c01544f0 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.429+0000 7f70c4ceb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c0152990 con 0x7f70c01537b0 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.429+0000 7f70be59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70c0153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70be59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70c0153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33748/0 (socket says 172.21.15.31:33748) 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70be59c700 1 -- 172.21.15.31:0/1230009212 learned_addr learned my addr 172.21.15.31:0/1230009212 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70bed9d700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c01537b0 0x7f70c01eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:14.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70bdd9b700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70c01544f0 0x7f70c0144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70be59c700 1 -- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c01537b0 msgr2=0x7f70c01eacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70be59c700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c01537b0 0x7f70c01eacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70be59c700 1 -- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70c01544f0 msgr2=0x7f70c0144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70be59c700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70c01544f0 0x7f70c0144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70be59c700 1 -- 172.21.15.31:0/1230009212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c0152c70 con 0x7f70a8004060 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70bdd9b700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70c01544f0 0x7f70c0144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70bed9d700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c01537b0 0x7f70c01eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70be59c700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70c0153190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f70b0011aa0 tx=0x7f70b0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70aeffd700 1 -- 172.21.15.31:0/1230009212 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000f980 con 0x7f70a8004060 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70aeffd700 1 -- 172.21.15.31:0/1230009212 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b000fb00 con 0x7f70a8004060 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c0145140 con 0x7f70a8004060 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70aeffd700 1 -- 172.21.15.31:0/1230009212 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000feb0 con 0x7f70a8004060 2024-03-30T01:43:14.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.430+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c01ea6e0 con 0x7f70a8004060 2024-03-30T01:43:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.431+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c0152fe0 con 0x7f70a8004060 2024-03-30T01:43:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.432+0000 7f70aeffd700 1 -- 172.21.15.31:0/1230009212 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f70b0014440 con 0x7f70a8004060 2024-03-30T01:43:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.432+0000 7f70aeffd700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70a006ac90 0x7f70a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.432+0000 7f70aeffd700 1 -- 172.21.15.31:0/1230009212 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f70b0089c20 con 0x7f70a8004060 2024-03-30T01:43:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.432+0000 7f70bdd9b700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70a006ac90 0x7f70a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:14.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.433+0000 7f70bdd9b700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70a006ac90 0x7f70a006d150 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70c0154980 tx=0x7f70b400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:14.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.437+0000 7f70aeffd700 1 -- 172.21.15.31:0/1230009212 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f70b0054d70 con 0x7f70a8004060 2024-03-30T01:43:14.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:14.700+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f70c0152e00 con 0x7f70a8004060 2024-03-30T01:43:15.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.263+0000 7f70aeffd700 1 -- 172.21.15.31:0/1230009212 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f70b0058390 con 0x7f70a8004060 2024-03-30T01:43:15.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70a006ac90 msgr2=0x7f70a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70a006ac90 0x7f70a006d150 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70c0154980 tx=0x7f70b400a000 comp rx=0 tx=0).stop 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 msgr2=0x7f70c0153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70c0153190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f70b0011aa0 tx=0x7f70b0011e60 comp rx=0 tx=0).stop 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 shutdown_connections 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70a8004060 0x7f70c0153190 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f70a006ac90 0x7f70a006d150 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70c01537b0 0x7f70c01eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 --2- 172.21.15.31:0/1230009212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70c01544f0 0x7f70c0144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 >> 172.21.15.31:0/1230009212 conn(0x7f70c00c3980 msgr2=0x7f70c0150070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 shutdown_connections 2024-03-30T01:43:15.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.267+0000 7f70c4ceb700 1 -- 172.21.15.31:0/1230009212 wait complete. 2024-03-30T01:43:15.289 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T01:43:15.289 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:43:15.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.422+0000 7fcc4c4b4700 1 Processor -- start 2024-03-30T01:43:15.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.422+0000 7fcc4c4b4700 1 -- start start 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4c4b4700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcc4414e5f0 0x7fcc440c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4c4b4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 0x7fcc44154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4c4b4700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc44072160 con 0x7fcc440c4100 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4c4b4700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc440c2bd0 con 0x7fcc4414e5f0 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4c4b4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc440c2e90 con 0x7fcc44071240 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4a250700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcc4414e5f0 0x7fcc440c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4a250700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcc4414e5f0 0x7fcc440c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54356/0 (socket says 172.21.15.31:54356) 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4aa51700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcc440c4100 0x7fcc44154dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58812/0 (socket says 172.21.15.31:58812) 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4a250700 1 -- 172.21.15.31:0/164860295 learned_addr learned my addr 172.21.15.31:0/164860295 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc49a4f700 1 --2- 172.21.15.31:0/164860295 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 0x7fcc44154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4a250700 1 -- 172.21.15.31:0/164860295 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 msgr2=0x7fcc44154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4a250700 1 --2- 172.21.15.31:0/164860295 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 0x7fcc44154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4a250700 1 -- 172.21.15.31:0/164860295 >> v1:172.21.15.31:6789/0 conn(0x7fcc440c4100 legacy=0x7fcc44154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4a250700 1 -- 172.21.15.31:0/164860295 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc440723b0 con 0x7fcc4414e5f0 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4924e700 1 -- 172.21.15.31:0/164860295 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048595406 0 0) 0x7fcc44072160 con 0x7fcc440c4100 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.423+0000 7fcc4a250700 1 --2- 172.21.15.31:0/164860295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 0x7fcc440c3bc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcc400098f0 tx=0x7fcc40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56f18d08bd34930 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4924e700 1 -- 172.21.15.31:0/164860295 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40011830 con 0x7fcc4414e5f0 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4924e700 1 -- 172.21.15.31:0/164860295 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc40011990 con 0x7fcc4414e5f0 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4924e700 1 -- 172.21.15.31:0/164860295 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40011d20 con 0x7fcc4414e5f0 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/164860295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 msgr2=0x7fcc440c3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4c4b4700 1 --2- 172.21.15.31:0/164860295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 0x7fcc440c3bc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcc400098f0 tx=0x7fcc40009c00 comp rx=0 tx=0).stop 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/164860295 shutdown_connections 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4c4b4700 1 --2- 172.21.15.31:0/164860295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 0x7fcc440c3bc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4c4b4700 1 --2- 172.21.15.31:0/164860295 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 0x7fcc44154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/164860295 >> 172.21.15.31:0/164860295 conn(0x7fcc4406ba20 msgr2=0x7fcc4406be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/164860295 shutdown_connections 2024-03-30T01:43:15.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.424+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/164860295 wait complete. 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.426+0000 7fcc4c4b4700 1 Processor -- start 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.426+0000 7fcc4c4b4700 1 -- start start 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.426+0000 7fcc4c4b4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 0x7fcc44152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.426+0000 7fcc4c4b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc440c4100 0x7fcc44151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.426+0000 7fcc4c4b4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 0x7fcc441499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.426+0000 7fcc4c4b4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc441533d0 con 0x7fcc4414e5f0 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.426+0000 7fcc4c4b4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc440c4f10 con 0x7fcc440c4100 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.426+0000 7fcc4c4b4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc441e99a0 con 0x7fcc44071240 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4aa51700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 0x7fcc441499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4aa51700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 0x7fcc441499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54368/0 (socket says 172.21.15.31:54368) 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4aa51700 1 -- 172.21.15.31:0/3789428184 learned_addr learned my addr 172.21.15.31:0/3789428184 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4a250700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 0x7fcc44152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc49a4f700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc440c4100 0x7fcc44151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4aa51700 1 -- 172.21.15.31:0/3789428184 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 msgr2=0x7fcc44152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4aa51700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 0x7fcc44152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4aa51700 1 -- 172.21.15.31:0/3789428184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc440c4100 msgr2=0x7fcc44151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4aa51700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc440c4100 0x7fcc44151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4aa51700 1 -- 172.21.15.31:0/3789428184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc40009550 con 0x7fcc4414e5f0 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4aa51700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 0x7fcc441499e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c002230 tx=0x7fcc3c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc36ffd700 1 -- 172.21.15.31:0/3789428184 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00aac0 con 0x7fcc4414e5f0 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc36ffd700 1 -- 172.21.15.31:0/3789428184 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c00ac40 con 0x7fcc4414e5f0 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc441539e0 con 0x7fcc4414e5f0 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc36ffd700 1 -- 172.21.15.31:0/3789428184 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0070d0 con 0x7fcc4414e5f0 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.427+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc44153fa0 con 0x7fcc4414e5f0 2024-03-30T01:43:15.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.428+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc441e9bc0 con 0x7fcc4414e5f0 2024-03-30T01:43:15.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.429+0000 7fcc36ffd700 1 -- 172.21.15.31:0/3789428184 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcc3c007250 con 0x7fcc4414e5f0 2024-03-30T01:43:15.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.429+0000 7fcc36ffd700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcc2806ac90 0x7fcc2806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.429+0000 7fcc36ffd700 1 -- 172.21.15.31:0/3789428184 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcc3c088e70 con 0x7fcc4414e5f0 2024-03-30T01:43:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.430+0000 7fcc4a250700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcc2806ac90 0x7fcc2806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.430+0000 7fcc4a250700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcc2806ac90 0x7fcc2806d150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcc40014040 tx=0x7fcc40008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:15.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.434+0000 7fcc36ffd700 1 -- 172.21.15.31:0/3789428184 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcc441e9bc0 con 0x7fcc4414e5f0 2024-03-30T01:43:15.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.700+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc441e9b60 con 0x7fcc4414e5f0 2024-03-30T01:43:15.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.700+0000 7fcc36ffd700 1 -- 172.21.15.31:0/3789428184 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fcc3c018020 con 0x7fcc4414e5f0 2024-03-30T01:43:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:15.707 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3969533250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3969533250},{"type":"v1","addr":"172.21.15.31:6836","nonce":3969533250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3308862528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3308862528},{"type":"v1","addr":"172.21.15.59:6835","nonce":3308862528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3281184299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3281184299},{"type":"v1","addr":"172.21.15.31:6837","nonce":3281184299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:43:12.245458+0000","modified":"2024-03-30T01:43:15.261882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6000":{"gid":6000,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3498371628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3498371628},{"type":"v1","addr":"172.21.15.59:6834","nonce":3498371628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T01:43:15.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcc2806ac90 msgr2=0x7fcc2806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcc2806ac90 0x7fcc2806d150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcc40014040 tx=0x7fcc40008030 comp rx=0 tx=0).stop 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 msgr2=0x7fcc441499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 0x7fcc441499e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c002230 tx=0x7fcc3c00cf50 comp rx=0 tx=0).stop 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 shutdown_connections 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc4414e5f0 0x7fcc441499e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcc2806ac90 0x7fcc2806d150 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc44071240 0x7fcc44152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 --2- 172.21.15.31:0/3789428184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc440c4100 0x7fcc44151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 >> 172.21.15.31:0/3789428184 conn(0x7fcc4406ba20 msgr2=0x7fcc44148040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 shutdown_connections 2024-03-30T01:43:15.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.703+0000 7fcc4c4b4700 1 -- 172.21.15.31:0/3789428184 wait complete. 2024-03-30T01:43:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 31 2024-03-30T01:43:15.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:15.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.852+0000 7f9c1781c700 1 Processor -- start 2024-03-30T01:43:15.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.852+0000 7f9c1781c700 1 -- start start 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.853+0000 7f9c1781c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9c10150110 0x7f9c10156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.853+0000 7f9c1781c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c100c4d60 con 0x7f9c1014fd30 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.853+0000 7f9c1781c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c100c5040 con 0x7f9c100c47c0 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.853+0000 7f9c1781c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c100c5320 con 0x7f9c10150110 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.853+0000 7f9c155b8700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9c1014fd30 0x7f9c100c8380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58822/0 (socket says 172.21.15.31:58822) 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.853+0000 7f9c155b8700 1 -- 172.21.15.31:0/2385035151 learned_addr learned my addr 172.21.15.31:0/2385035151 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.853+0000 7f9c14db7700 1 --2- 172.21.15.31:0/2385035151 >> v2:172.21.15.59:3300/0 conn(0x7f9c10150110 0x7f9c10156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c07fff700 1 -- 172.21.15.31:0/2385035151 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363958909 0 0) 0x7f9c100c4d60 con 0x7f9c1014fd30 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c07fff700 1 -- 172.21.15.31:0/2385035151 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bfc0035b0 con 0x7f9c1014fd30 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c07fff700 1 -- 172.21.15.31:0/2385035151 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035107422 0 0) 0x7f9c100c5040 con 0x7f9c100c47c0 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c14db7700 1 -- 172.21.15.31:0/2385035151 >> v1:172.21.15.59:6789/0 conn(0x7f9c100c47c0 legacy=0x7f9c100ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c14db7700 1 -- 172.21.15.31:0/2385035151 >> v1:172.21.15.31:6789/0 conn(0x7f9c1014fd30 legacy=0x7f9c100c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c14db7700 1 -- 172.21.15.31:0/2385035151 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c100c5600 con 0x7f9c10150110 2024-03-30T01:43:15.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c14db7700 1 --2- 172.21.15.31:0/2385035151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c10150110 0x7f9c10156800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009840 tx=0x7f9c00009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8941cea0d9c4343a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:15.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c07fff700 1 -- 172.21.15.31:0/2385035151 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c001670 con 0x7f9c10150110 2024-03-30T01:43:15.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c07fff700 1 -- 172.21.15.31:0/2385035151 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c00004030 con 0x7f9c10150110 2024-03-30T01:43:15.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.854+0000 7f9c07fff700 1 -- 172.21.15.31:0/2385035151 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c000043c0 con 0x7f9c10150110 2024-03-30T01:43:15.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.855+0000 7f9c1781c700 1 -- 172.21.15.31:0/2385035151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c10150110 msgr2=0x7f9c10156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.855+0000 7f9c1781c700 1 --2- 172.21.15.31:0/2385035151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c10150110 0x7f9c10156800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009840 tx=0x7f9c00009b50 comp rx=0 tx=0).stop 2024-03-30T01:43:15.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.855+0000 7f9c1781c700 1 -- 172.21.15.31:0/2385035151 shutdown_connections 2024-03-30T01:43:15.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.855+0000 7f9c1781c700 1 --2- 172.21.15.31:0/2385035151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c10150110 0x7f9c10156800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.855+0000 7f9c1781c700 1 -- 172.21.15.31:0/2385035151 >> 172.21.15.31:0/2385035151 conn(0x7f9c1006ab20 msgr2=0x7f9c1006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:15.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.855+0000 7f9c1781c700 1 -- 172.21.15.31:0/2385035151 shutdown_connections 2024-03-30T01:43:15.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.855+0000 7f9c1781c700 1 -- 172.21.15.31:0/2385035151 wait complete. 2024-03-30T01:43:15.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.856+0000 7f9c1781c700 1 Processor -- start 2024-03-30T01:43:15.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.856+0000 7f9c1781c700 1 -- start start 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c1781c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c100c47c0 0x7f9c10154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c1781c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c1014fd30 0x7f9c100c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c1781c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c10150110 0x7f9c1014d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c1781c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c101e9510 con 0x7f9c1014fd30 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c1781c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c101e97d0 con 0x7f9c10150110 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c1781c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c101e9ab0 con 0x7f9c100c47c0 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c14db7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c1014fd30 0x7f9c100c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c14db7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c1014fd30 0x7f9c100c1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54374/0 (socket says 172.21.15.31:54374) 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c14db7700 1 -- 172.21.15.31:0/395572487 learned_addr learned my addr 172.21.15.31:0/395572487 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c155b8700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c100c47c0 0x7f9c10154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.857+0000 7f9c15db9700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c10150110 0x7f9c1014d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c14db7700 1 -- 172.21.15.31:0/395572487 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c100c47c0 msgr2=0x7f9c10154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c14db7700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c100c47c0 0x7f9c10154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c14db7700 1 -- 172.21.15.31:0/395572487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c10150110 msgr2=0x7f9c1014d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c14db7700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c10150110 0x7f9c1014d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c14db7700 1 -- 172.21.15.31:0/395572487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c000094a0 con 0x7f9c1014fd30 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c14db7700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c1014fd30 0x7f9c100c1470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9c10071cc0 tx=0x7f9c00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c05ffb700 1 -- 172.21.15.31:0/395572487 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c000045b0 con 0x7f9c1014fd30 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c05ffb700 1 -- 172.21.15.31:0/395572487 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c00010590 con 0x7f9c1014fd30 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c101e9d90 con 0x7f9c1014fd30 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c05ffb700 1 -- 172.21.15.31:0/395572487 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00010920 con 0x7f9c1014fd30 2024-03-30T01:43:15.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.858+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c100aa890 con 0x7f9c1014fd30 2024-03-30T01:43:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.860+0000 7f9c05ffb700 1 -- 172.21.15.31:0/395572487 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9c00010a80 con 0x7f9c1014fd30 2024-03-30T01:43:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.860+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c10073760 con 0x7f9c1014fd30 2024-03-30T01:43:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.860+0000 7f9c05ffb700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9bf406ac90 0x7f9bf406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.861+0000 7f9c155b8700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9bf406ac90 0x7f9bf406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.861+0000 7f9c05ffb700 1 -- 172.21.15.31:0/395572487 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9c0008abb0 con 0x7f9c1014fd30 2024-03-30T01:43:15.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.861+0000 7f9c155b8700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9bf406ac90 0x7f9bf406d150 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c001bb0 tx=0x7f9c0c001b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:15.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:15.865+0000 7f9c05ffb700 1 -- 172.21.15.31:0/395572487 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9c00055c40 con 0x7f9c1014fd30 2024-03-30T01:43:16.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.084+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c1004ea80 con 0x7f9c1014fd30 2024-03-30T01:43:16.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.085+0000 7f9c05ffb700 1 -- 172.21.15.31:0/395572487 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16200 (secure 0 0 0) 0x7f9c00020090 con 0x7f9c1014fd30 2024-03-30T01:43:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:16.091 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":60,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:13.247511+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:43:09.911033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T01:43:11.216248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:16.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9bf406ac90 msgr2=0x7f9bf406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9bf406ac90 0x7f9bf406d150 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c001bb0 tx=0x7f9c0c001b40 comp rx=0 tx=0).stop 2024-03-30T01:43:16.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c1014fd30 msgr2=0x7f9c100c1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c1014fd30 0x7f9c100c1470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9c10071cc0 tx=0x7f9c00004750 comp rx=0 tx=0).stop 2024-03-30T01:43:16.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 shutdown_connections 2024-03-30T01:43:16.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9bf406ac90 0x7f9bf406d150 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c10150110 0x7f9c1014d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c100c47c0 0x7f9c10154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 --2- 172.21.15.31:0/395572487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c1014fd30 0x7f9c100c1470 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 >> 172.21.15.31:0/395572487 conn(0x7f9c1006ab20 msgr2=0x7f9c100acd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:16.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.087+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 shutdown_connections 2024-03-30T01:43:16.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.088+0000 7f9c1781c700 1 -- 172.21.15.31:0/395572487 wait complete. 2024-03-30T01:43:16.108 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T01:43:16.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:43:16.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.238+0000 7f92e525d700 1 Processor -- start 2024-03-30T01:43:16.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.238+0000 7f92e525d700 1 -- start start 2024-03-30T01:43:16.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.238+0000 7f92e525d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e00c1300 0x7f92e006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92e525d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f92e0152650 0x7f92e0152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92e525d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e00c2f60 con 0x7f92e014fc30 2024-03-30T01:43:16.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92e525d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e00c2a40 con 0x7f92e0152650 2024-03-30T01:43:16.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92e525d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e006eca0 con 0x7f92e00c1300 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92df59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f92e0152650 0x7f92e0152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92ded9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f92e014fc30 0x7f92e006a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58836/0 (socket says 172.21.15.31:58836) 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92ded9d700 1 -- 172.21.15.31:0/2042962783 learned_addr learned my addr 172.21.15.31:0/2042962783 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92de59c700 1 --2- 172.21.15.31:0/2042962783 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e00c1300 0x7f92e006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92ddd9b700 1 -- 172.21.15.31:0/2042962783 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957787215 0 0) 0x7f92e00c2f60 con 0x7f92e014fc30 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92ddd9b700 1 -- 172.21.15.31:0/2042962783 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92c80035b0 con 0x7f92e014fc30 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92df59e700 1 -- 172.21.15.31:0/2042962783 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e00c1300 msgr2=0x7f92e006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92df59e700 1 --2- 172.21.15.31:0/2042962783 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e00c1300 0x7f92e006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92df59e700 1 -- 172.21.15.31:0/2042962783 >> v1:172.21.15.31:6789/0 conn(0x7f92e014fc30 legacy=0x7f92e006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92df59e700 1 -- 172.21.15.31:0/2042962783 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e006b350 con 0x7f92e0152650 2024-03-30T01:43:16.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92df59e700 1 --2- 172.21.15.31:0/2042962783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e0152650 0x7f92e0152ae0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009d90 tx=0x7f92d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525c9347ae21843f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.239+0000 7f92ddd9b700 1 -- 172.21.15.31:0/2042962783 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800e070 con 0x7f92e0152650 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92ddd9b700 1 -- 172.21.15.31:0/2042962783 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d800bb20 con 0x7f92e0152650 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92ddd9b700 1 -- 172.21.15.31:0/2042962783 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8012430 con 0x7f92e0152650 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92e525d700 1 -- 172.21.15.31:0/2042962783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e0152650 msgr2=0x7f92e0152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92e525d700 1 --2- 172.21.15.31:0/2042962783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e0152650 0x7f92e0152ae0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009d90 tx=0x7f92d800b330 comp rx=0 tx=0).stop 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92e525d700 1 -- 172.21.15.31:0/2042962783 shutdown_connections 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92e525d700 1 --2- 172.21.15.31:0/2042962783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e0152650 0x7f92e0152ae0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92e525d700 1 --2- 172.21.15.31:0/2042962783 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e00c1300 0x7f92e006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92e525d700 1 -- 172.21.15.31:0/2042962783 >> 172.21.15.31:0/2042962783 conn(0x7f92e00c3a60 msgr2=0x7f92e00c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92e525d700 1 -- 172.21.15.31:0/2042962783 shutdown_connections 2024-03-30T01:43:16.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.240+0000 7f92e525d700 1 -- 172.21.15.31:0/2042962783 wait complete. 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.242+0000 7f92e525d700 1 Processor -- start 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.242+0000 7f92e525d700 1 -- start start 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.242+0000 7f92e525d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e00c1300 0x7f92e00a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.242+0000 7f92e525d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e014fc30 0x7f92e01de5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92e525d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e0152650 0x7f92e0143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92e525d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e01de170 con 0x7f92e014fc30 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92e525d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e014ad40 con 0x7f92e00c1300 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92e525d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e014b020 con 0x7f92e0152650 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92de59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e014fc30 0x7f92e01de5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92de59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e014fc30 0x7f92e01de5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54382/0 (socket says 172.21.15.31:54382) 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92de59c700 1 -- 172.21.15.31:0/92939644 learned_addr learned my addr 172.21.15.31:0/92939644 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92ded9d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e00c1300 0x7f92e00a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92df59e700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e0152650 0x7f92e0143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92de59c700 1 -- 172.21.15.31:0/92939644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e0152650 msgr2=0x7f92e0143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92de59c700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e0152650 0x7f92e0143660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92de59c700 1 -- 172.21.15.31:0/92939644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e00c1300 msgr2=0x7f92e00a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92de59c700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e00c1300 0x7f92e00a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92de59c700 1 -- 172.21.15.31:0/92939644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d8009a70 con 0x7f92e014fc30 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92ded9d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e00c1300 0x7f92e00a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92de59c700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e014fc30 0x7f92e01de5c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9b0 tx=0x7f92d400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92c77fe700 1 -- 172.21.15.31:0/92939644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4009710 con 0x7f92e014fc30 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92c77fe700 1 -- 172.21.15.31:0/92939644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d400c850 con 0x7f92e014fc30 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e014b300 con 0x7f92e014fc30 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92c77fe700 1 -- 172.21.15.31:0/92939644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d400cbc0 con 0x7f92e014fc30 2024-03-30T01:43:16.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.243+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e014b5e0 con 0x7f92e014fc30 2024-03-30T01:43:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.244+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e004ea80 con 0x7f92e014fc30 2024-03-30T01:43:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.245+0000 7f92c77fe700 1 -- 172.21.15.31:0/92939644 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f92d400cd20 con 0x7f92e014fc30 2024-03-30T01:43:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.245+0000 7f92c77fe700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f92cc06ac90 0x7f92cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.245+0000 7f92c77fe700 1 -- 172.21.15.31:0/92939644 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f92d40894e0 con 0x7f92e014fc30 2024-03-30T01:43:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.246+0000 7f92ded9d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f92cc06ac90 0x7f92cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.246+0000 7f92ded9d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f92cc06ac90 0x7f92cc06d150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f92d00011e0 tx=0x7f92d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.250+0000 7f92c77fe700 1 -- 172.21.15.31:0/92939644 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f92d4054610 con 0x7f92e014fc30 2024-03-30T01:43:16.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.461+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92e014b190 con 0x7f92cc06ac90 2024-03-30T01:43:16.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.469+0000 7f92c77fe700 1 -- 172.21.15.31:0/92939644 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289194 (secure 0 0 0) 0x7f92e014b190 con 0x7f92cc06ac90 2024-03-30T01:43:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f92cc06ac90 msgr2=0x7f92cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f92cc06ac90 0x7f92cc06d150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f92d00011e0 tx=0x7f92d0009040 comp rx=0 tx=0).stop 2024-03-30T01:43:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e014fc30 msgr2=0x7f92e01de5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e014fc30 0x7f92e01de5c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9b0 tx=0x7f92d400ed70 comp rx=0 tx=0).stop 2024-03-30T01:43:16.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 shutdown_connections 2024-03-30T01:43:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f92cc06ac90 0x7f92cc06d150 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f92e0152650 0x7f92e0143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e00c1300 0x7f92e00a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 --2- 172.21.15.31:0/92939644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92e014fc30 0x7f92e01de5c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 >> 172.21.15.31:0/92939644 conn(0x7f92e00c3a60 msgr2=0x7f92e0152a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 shutdown_connections 2024-03-30T01:43:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.473+0000 7f92e525d700 1 -- 172.21.15.31:0/92939644 wait complete. 2024-03-30T01:43:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:43:16.493 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-03-30T01:43:15.839398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222852,"kb_used_data":6660,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751868,"statfs":{"total":773094113280,"available":772865912832,"internally_reserved":0,"allocated":6819840,"data_stored":3748597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.057389"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271679+0000","last_change":"2024-03-30T01:43:12.269722+0000","last_active":"2024-03-30T01:43:12.271679+0000","last_peered":"2024-03-30T01:43:12.271679+0000","last_clean":"2024-03-30T01:43:12.271679+0000","last_became_active":"2024-03-30T01:43:12.269445+0000","last_became_peered":"2024-03-30T01:43:12.269445+0000","last_unstale":"2024-03-30T01:43:12.271679+0000","last_undegraded":"2024-03-30T01:43:12.271679+0000","last_fullsized":"2024-03-30T01:43:12.271679+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.269072+0000","last_change":"2024-03-30T01:43:12.261636+0000","last_active":"2024-03-30T01:43:12.269072+0000","last_peered":"2024-03-30T01:43:12.269072+0000","last_clean":"2024-03-30T01:43:12.269072+0000","last_became_active":"2024-03-30T01:43:12.261289+0000","last_became_peered":"2024-03-30T01:43:12.261289+0000","last_unstale":"2024-03-30T01:43:12.269072+0000","last_undegraded":"2024-03-30T01:43:12.269072+0000","last_fullsized":"2024-03-30T01:43:12.269072+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:53.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282260+0000","last_change":"2024-03-30T01:43:12.273399+0000","last_active":"2024-03-30T01:43:12.282260+0000","last_peered":"2024-03-30T01:43:12.282260+0000","last_clean":"2024-03-30T01:43:12.282260+0000","last_became_active":"2024-03-30T01:43:12.273099+0000","last_became_peered":"2024-03-30T01:43:12.273099+0000","last_unstale":"2024-03-30T01:43:12.282260+0000","last_undegraded":"2024-03-30T01:43:12.282260+0000","last_fullsized":"2024-03-30T01:43:12.282260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281552+0000","last_change":"2024-03-30T01:43:12.277385+0000","last_active":"2024-03-30T01:43:12.281552+0000","last_peered":"2024-03-30T01:43:12.281552+0000","last_clean":"2024-03-30T01:43:12.281552+0000","last_became_active":"2024-03-30T01:43:12.277091+0000","last_became_peered":"2024-03-30T01:43:12.277091+0000","last_unstale":"2024-03-30T01:43:12.281552+0000","last_undegraded":"2024-03-30T01:43:12.281552+0000","last_fullsized":"2024-03-30T01:43:12.281552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:09:35.397960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281955+0000","last_change":"2024-03-30T01:43:12.276432+0000","last_active":"2024-03-30T01:43:12.281955+0000","last_peered":"2024-03-30T01:43:12.281955+0000","last_clean":"2024-03-30T01:43:12.281955+0000","last_became_active":"2024-03-30T01:43:12.276157+0000","last_became_peered":"2024-03-30T01:43:12.276157+0000","last_unstale":"2024-03-30T01:43:12.281955+0000","last_undegraded":"2024-03-30T01:43:12.281955+0000","last_fullsized":"2024-03-30T01:43:12.281955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:56.492934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.277103+0000","last_change":"2024-03-30T01:43:12.273311+0000","last_active":"2024-03-30T01:43:12.277103+0000","last_peered":"2024-03-30T01:43:12.277103+0000","last_clean":"2024-03-30T01:43:12.277103+0000","last_became_active":"2024-03-30T01:43:12.272939+0000","last_became_peered":"2024-03-30T01:43:12.272939+0000","last_unstale":"2024-03-30T01:43:12.277103+0000","last_undegraded":"2024-03-30T01:43:12.277103+0000","last_fullsized":"2024-03-30T01:43:12.277103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:38.756747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281349+0000","last_change":"2024-03-30T01:43:12.279192+0000","last_active":"2024-03-30T01:43:12.281349+0000","last_peered":"2024-03-30T01:43:12.281349+0000","last_clean":"2024-03-30T01:43:12.281349+0000","last_became_active":"2024-03-30T01:43:12.278893+0000","last_became_peered":"2024-03-30T01:43:12.278893+0000","last_unstale":"2024-03-30T01:43:12.281349+0000","last_undegraded":"2024-03-30T01:43:12.281349+0000","last_fullsized":"2024-03-30T01:43:12.281349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:21:09.466088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281307+0000","last_change":"2024-03-30T01:43:12.274078+0000","last_active":"2024-03-30T01:43:12.281307+0000","last_peered":"2024-03-30T01:43:12.281307+0000","last_clean":"2024-03-30T01:43:12.281307+0000","last_became_active":"2024-03-30T01:43:12.273703+0000","last_became_peered":"2024-03-30T01:43:12.273703+0000","last_unstale":"2024-03-30T01:43:12.281307+0000","last_undegraded":"2024-03-30T01:43:12.281307+0000","last_fullsized":"2024-03-30T01:43:12.281307+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282502+0000","last_change":"2024-03-30T01:43:12.274182+0000","last_active":"2024-03-30T01:43:12.282502+0000","last_peered":"2024-03-30T01:43:12.282502+0000","last_clean":"2024-03-30T01:43:12.282502+0000","last_became_active":"2024-03-30T01:43:12.273863+0000","last_became_peered":"2024-03-30T01:43:12.273863+0000","last_unstale":"2024-03-30T01:43:12.282502+0000","last_undegraded":"2024-03-30T01:43:12.282502+0000","last_fullsized":"2024-03-30T01:43:12.282502+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282458+0000","last_change":"2024-03-30T01:43:12.277023+0000","last_active":"2024-03-30T01:43:12.282458+0000","last_peered":"2024-03-30T01:43:12.282458+0000","last_clean":"2024-03-30T01:43:12.282458+0000","last_became_active":"2024-03-30T01:43:12.276685+0000","last_became_peered":"2024-03-30T01:43:12.276685+0000","last_unstale":"2024-03-30T01:43:12.282458+0000","last_undegraded":"2024-03-30T01:43:12.282458+0000","last_fullsized":"2024-03-30T01:43:12.282458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:53.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.280490+0000","last_change":"2024-03-30T01:43:12.273450+0000","last_active":"2024-03-30T01:43:12.280490+0000","last_peered":"2024-03-30T01:43:12.280490+0000","last_clean":"2024-03-30T01:43:12.280490+0000","last_became_active":"2024-03-30T01:43:12.273185+0000","last_became_peered":"2024-03-30T01:43:12.273185+0000","last_unstale":"2024-03-30T01:43:12.280490+0000","last_undegraded":"2024-03-30T01:43:12.280490+0000","last_fullsized":"2024-03-30T01:43:12.280490+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.280952+0000","last_change":"2024-03-30T01:43:12.273238+0000","last_active":"2024-03-30T01:43:12.280952+0000","last_peered":"2024-03-30T01:43:12.280952+0000","last_clean":"2024-03-30T01:43:12.280952+0000","last_became_active":"2024-03-30T01:43:12.272970+0000","last_became_peered":"2024-03-30T01:43:12.272970+0000","last_unstale":"2024-03-30T01:43:12.280952+0000","last_undegraded":"2024-03-30T01:43:12.280952+0000","last_fullsized":"2024-03-30T01:43:12.280952+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:49.877712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281107+0000","last_change":"2024-03-30T01:43:12.276660+0000","last_active":"2024-03-30T01:43:12.281107+0000","last_peered":"2024-03-30T01:43:12.281107+0000","last_clean":"2024-03-30T01:43:12.281107+0000","last_became_active":"2024-03-30T01:43:12.276437+0000","last_became_peered":"2024-03-30T01:43:12.276437+0000","last_unstale":"2024-03-30T01:43:12.281107+0000","last_undegraded":"2024-03-30T01:43:12.281107+0000","last_fullsized":"2024-03-30T01:43:12.281107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282427+0000","last_change":"2024-03-30T01:43:12.276823+0000","last_active":"2024-03-30T01:43:12.282427+0000","last_peered":"2024-03-30T01:43:12.282427+0000","last_clean":"2024-03-30T01:43:12.282427+0000","last_became_active":"2024-03-30T01:43:12.272988+0000","last_became_peered":"2024-03-30T01:43:12.272988+0000","last_unstale":"2024-03-30T01:43:12.282427+0000","last_undegraded":"2024-03-30T01:43:12.282427+0000","last_fullsized":"2024-03-30T01:43:12.282427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282708+0000","last_change":"2024-03-30T01:43:12.277931+0000","last_active":"2024-03-30T01:43:12.282708+0000","last_peered":"2024-03-30T01:43:12.282708+0000","last_clean":"2024-03-30T01:43:12.282708+0000","last_became_active":"2024-03-30T01:43:12.277601+0000","last_became_peered":"2024-03-30T01:43:12.277601+0000","last_unstale":"2024-03-30T01:43:12.282708+0000","last_undegraded":"2024-03-30T01:43:12.282708+0000","last_fullsized":"2024-03-30T01:43:12.282708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279537+0000","last_change":"2024-03-30T01:43:12.275724+0000","last_active":"2024-03-30T01:43:12.279537+0000","last_peered":"2024-03-30T01:43:12.279537+0000","last_clean":"2024-03-30T01:43:12.279537+0000","last_became_active":"2024-03-30T01:43:12.275438+0000","last_became_peered":"2024-03-30T01:43:12.275438+0000","last_unstale":"2024-03-30T01:43:12.279537+0000","last_undegraded":"2024-03-30T01:43:12.279537+0000","last_fullsized":"2024-03-30T01:43:12.279537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:18.506294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270111+0000","last_change":"2024-03-30T01:43:12.268431+0000","last_active":"2024-03-30T01:43:12.270111+0000","last_peered":"2024-03-30T01:43:12.270111+0000","last_clean":"2024-03-30T01:43:12.270111+0000","last_became_active":"2024-03-30T01:43:12.268148+0000","last_became_peered":"2024-03-30T01:43:12.268148+0000","last_unstale":"2024-03-30T01:43:12.270111+0000","last_undegraded":"2024-03-30T01:43:12.270111+0000","last_fullsized":"2024-03-30T01:43:12.270111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:49.877712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.280063+0000","last_change":"2024-03-30T01:43:12.275632+0000","last_active":"2024-03-30T01:43:12.280063+0000","last_peered":"2024-03-30T01:43:12.280063+0000","last_clean":"2024-03-30T01:43:12.280063+0000","last_became_active":"2024-03-30T01:43:12.275329+0000","last_became_peered":"2024-03-30T01:43:12.275329+0000","last_unstale":"2024-03-30T01:43:12.280063+0000","last_undegraded":"2024-03-30T01:43:12.280063+0000","last_fullsized":"2024-03-30T01:43:12.280063+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:44.982486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.267611+0000","last_change":"2024-03-30T01:43:12.265240+0000","last_active":"2024-03-30T01:43:12.267611+0000","last_peered":"2024-03-30T01:43:12.267611+0000","last_clean":"2024-03-30T01:43:12.267611+0000","last_became_active":"2024-03-30T01:43:12.264980+0000","last_became_peered":"2024-03-30T01:43:12.264980+0000","last_unstale":"2024-03-30T01:43:12.267611+0000","last_undegraded":"2024-03-30T01:43:12.267611+0000","last_fullsized":"2024-03-30T01:43:12.267611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:15:38.037358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271658+0000","last_change":"2024-03-30T01:43:12.266614+0000","last_active":"2024-03-30T01:43:12.271658+0000","last_peered":"2024-03-30T01:43:12.271658+0000","last_clean":"2024-03-30T01:43:12.271658+0000","last_became_active":"2024-03-30T01:43:12.266325+0000","last_became_peered":"2024-03-30T01:43:12.266325+0000","last_unstale":"2024-03-30T01:43:12.271658+0000","last_undegraded":"2024-03-30T01:43:12.271658+0000","last_fullsized":"2024-03-30T01:43:12.271658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:45:23.286791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.272595+0000","last_change":"2024-03-30T01:43:12.269577+0000","last_active":"2024-03-30T01:43:12.272595+0000","last_peered":"2024-03-30T01:43:12.272595+0000","last_clean":"2024-03-30T01:43:12.272595+0000","last_became_active":"2024-03-30T01:43:12.269263+0000","last_became_peered":"2024-03-30T01:43:12.269263+0000","last_unstale":"2024-03-30T01:43:12.272595+0000","last_undegraded":"2024-03-30T01:43:12.272595+0000","last_fullsized":"2024-03-30T01:43:12.272595+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282429+0000","last_change":"2024-03-30T01:43:12.276626+0000","last_active":"2024-03-30T01:43:12.282429+0000","last_peered":"2024-03-30T01:43:12.282429+0000","last_clean":"2024-03-30T01:43:12.282429+0000","last_became_active":"2024-03-30T01:43:12.276302+0000","last_became_peered":"2024-03-30T01:43:12.276302+0000","last_unstale":"2024-03-30T01:43:12.282429+0000","last_undegraded":"2024-03-30T01:43:12.282429+0000","last_fullsized":"2024-03-30T01:43:12.282429+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:11.027051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282442+0000","last_change":"2024-03-30T01:43:12.276658+0000","last_active":"2024-03-30T01:43:12.282442+0000","last_peered":"2024-03-30T01:43:12.282442+0000","last_clean":"2024-03-30T01:43:12.282442+0000","last_became_active":"2024-03-30T01:43:12.276363+0000","last_became_peered":"2024-03-30T01:43:12.276363+0000","last_unstale":"2024-03-30T01:43:12.282442+0000","last_undegraded":"2024-03-30T01:43:12.282442+0000","last_fullsized":"2024-03-30T01:43:12.282442+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:06.329557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282484+0000","last_change":"2024-03-30T01:43:12.273270+0000","last_active":"2024-03-30T01:43:12.282484+0000","last_peered":"2024-03-30T01:43:12.282484+0000","last_clean":"2024-03-30T01:43:12.282484+0000","last_became_active":"2024-03-30T01:43:12.272973+0000","last_became_peered":"2024-03-30T01:43:12.272973+0000","last_unstale":"2024-03-30T01:43:12.282484+0000","last_undegraded":"2024-03-30T01:43:12.282484+0000","last_fullsized":"2024-03-30T01:43:12.282484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.273511+0000","last_change":"2024-03-30T01:43:12.270679+0000","last_active":"2024-03-30T01:43:12.273511+0000","last_peered":"2024-03-30T01:43:12.273511+0000","last_clean":"2024-03-30T01:43:12.273511+0000","last_became_active":"2024-03-30T01:43:12.270457+0000","last_became_peered":"2024-03-30T01:43:12.270457+0000","last_unstale":"2024-03-30T01:43:12.273511+0000","last_undegraded":"2024-03-30T01:43:12.273511+0000","last_fullsized":"2024-03-30T01:43:12.273511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:45.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.273366+0000","last_change":"2024-03-30T01:43:12.269460+0000","last_active":"2024-03-30T01:43:12.273366+0000","last_peered":"2024-03-30T01:43:12.273366+0000","last_clean":"2024-03-30T01:43:12.273366+0000","last_became_active":"2024-03-30T01:43:12.269159+0000","last_became_peered":"2024-03-30T01:43:12.269159+0000","last_unstale":"2024-03-30T01:43:12.273366+0000","last_undegraded":"2024-03-30T01:43:12.273366+0000","last_fullsized":"2024-03-30T01:43:12.273366+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281445+0000","last_change":"2024-03-30T01:43:12.276051+0000","last_active":"2024-03-30T01:43:12.281445+0000","last_peered":"2024-03-30T01:43:12.281445+0000","last_clean":"2024-03-30T01:43:12.281445+0000","last_became_active":"2024-03-30T01:43:12.275296+0000","last_became_peered":"2024-03-30T01:43:12.275296+0000","last_unstale":"2024-03-30T01:43:12.281445+0000","last_undegraded":"2024-03-30T01:43:12.281445+0000","last_fullsized":"2024-03-30T01:43:12.281445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:17.176008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270043+0000","last_change":"2024-03-30T01:43:12.266882+0000","last_active":"2024-03-30T01:43:12.270043+0000","last_peered":"2024-03-30T01:43:12.270043+0000","last_clean":"2024-03-30T01:43:12.270043+0000","last_became_active":"2024-03-30T01:43:12.266450+0000","last_became_peered":"2024-03-30T01:43:12.266450+0000","last_unstale":"2024-03-30T01:43:12.270043+0000","last_undegraded":"2024-03-30T01:43:12.270043+0000","last_fullsized":"2024-03-30T01:43:12.270043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282693+0000","last_change":"2024-03-30T01:43:12.279605+0000","last_active":"2024-03-30T01:43:12.282693+0000","last_peered":"2024-03-30T01:43:12.282693+0000","last_clean":"2024-03-30T01:43:12.282693+0000","last_became_active":"2024-03-30T01:43:12.279323+0000","last_became_peered":"2024-03-30T01:43:12.279323+0000","last_unstale":"2024-03-30T01:43:12.282693+0000","last_undegraded":"2024-03-30T01:43:12.282693+0000","last_fullsized":"2024-03-30T01:43:12.282693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:32:07.610941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281138+0000","last_change":"2024-03-30T01:43:12.277845+0000","last_active":"2024-03-30T01:43:12.281138+0000","last_peered":"2024-03-30T01:43:12.281138+0000","last_clean":"2024-03-30T01:43:12.281138+0000","last_became_active":"2024-03-30T01:43:12.277482+0000","last_became_peered":"2024-03-30T01:43:12.277482+0000","last_unstale":"2024-03-30T01:43:12.281138+0000","last_undegraded":"2024-03-30T01:43:12.281138+0000","last_fullsized":"2024-03-30T01:43:12.281138+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281528+0000","last_change":"2024-03-30T01:43:12.276064+0000","last_active":"2024-03-30T01:43:12.281528+0000","last_peered":"2024-03-30T01:43:12.281528+0000","last_clean":"2024-03-30T01:43:12.281528+0000","last_became_active":"2024-03-30T01:43:12.275670+0000","last_became_peered":"2024-03-30T01:43:12.275670+0000","last_unstale":"2024-03-30T01:43:12.281528+0000","last_undegraded":"2024-03-30T01:43:12.281528+0000","last_fullsized":"2024-03-30T01:43:12.281528+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:45.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270338+0000","last_change":"2024-03-30T01:43:12.260768+0000","last_active":"2024-03-30T01:43:12.270338+0000","last_peered":"2024-03-30T01:43:12.270338+0000","last_clean":"2024-03-30T01:43:12.270338+0000","last_became_active":"2024-03-30T01:43:12.260512+0000","last_became_peered":"2024-03-30T01:43:12.260512+0000","last_unstale":"2024-03-30T01:43:12.270338+0000","last_undegraded":"2024-03-30T01:43:12.270338+0000","last_fullsized":"2024-03-30T01:43:12.270338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:59.579718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.274705+0000","last_change":"2024-03-30T01:43:12.270602+0000","last_active":"2024-03-30T01:43:12.274705+0000","last_peered":"2024-03-30T01:43:12.274705+0000","last_clean":"2024-03-30T01:43:12.274705+0000","last_became_active":"2024-03-30T01:43:12.270328+0000","last_became_peered":"2024-03-30T01:43:12.270328+0000","last_unstale":"2024-03-30T01:43:12.274705+0000","last_undegraded":"2024-03-30T01:43:12.274705+0000","last_fullsized":"2024-03-30T01:43:12.274705+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:24.040846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282085+0000","last_change":"2024-03-30T01:43:11.800815+0000","last_active":"2024-03-30T01:43:12.282085+0000","last_peered":"2024-03-30T01:43:12.282085+0000","last_clean":"2024-03-30T01:43:12.282085+0000","last_became_active":"2024-03-30T01:43:11.800527+0000","last_became_peered":"2024-03-30T01:43:11.800527+0000","last_unstale":"2024-03-30T01:43:12.282085+0000","last_undegraded":"2024-03-30T01:43:12.282085+0000","last_fullsized":"2024-03-30T01:43:12.282085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:36.821233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279794+0000","last_change":"2024-03-30T01:43:12.268225+0000","last_active":"2024-03-30T01:43:12.279794+0000","last_peered":"2024-03-30T01:43:12.279794+0000","last_clean":"2024-03-30T01:43:12.279794+0000","last_became_active":"2024-03-30T01:43:12.267769+0000","last_became_peered":"2024-03-30T01:43:12.267769+0000","last_unstale":"2024-03-30T01:43:12.279794+0000","last_undegraded":"2024-03-30T01:43:12.279794+0000","last_fullsized":"2024-03-30T01:43:12.279794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282134+0000","last_change":"2024-03-30T01:43:11.806420+0000","last_active":"2024-03-30T01:43:12.282134+0000","last_peered":"2024-03-30T01:43:12.282134+0000","last_clean":"2024-03-30T01:43:12.282134+0000","last_became_active":"2024-03-30T01:43:11.806144+0000","last_became_peered":"2024-03-30T01:43:11.806144+0000","last_unstale":"2024-03-30T01:43:12.282134+0000","last_undegraded":"2024-03-30T01:43:12.282134+0000","last_fullsized":"2024-03-30T01:43:12.282134+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:19.302319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.268410+0000","last_change":"2024-03-30T01:43:12.259615+0000","last_active":"2024-03-30T01:43:12.268410+0000","last_peered":"2024-03-30T01:43:12.268410+0000","last_clean":"2024-03-30T01:43:12.268410+0000","last_became_active":"2024-03-30T01:43:12.259331+0000","last_became_peered":"2024-03-30T01:43:12.259331+0000","last_unstale":"2024-03-30T01:43:12.268410+0000","last_undegraded":"2024-03-30T01:43:12.268410+0000","last_fullsized":"2024-03-30T01:43:12.268410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:10.944027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281404+0000","last_change":"2024-03-30T01:43:11.811391+0000","last_active":"2024-03-30T01:43:12.281404+0000","last_peered":"2024-03-30T01:43:12.281404+0000","last_clean":"2024-03-30T01:43:12.281404+0000","last_became_active":"2024-03-30T01:43:11.810873+0000","last_became_peered":"2024-03-30T01:43:11.810873+0000","last_unstale":"2024-03-30T01:43:12.281404+0000","last_undegraded":"2024-03-30T01:43:12.281404+0000","last_fullsized":"2024-03-30T01:43:12.281404+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:22.774118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282370+0000","last_change":"2024-03-30T01:43:12.276578+0000","last_active":"2024-03-30T01:43:12.282370+0000","last_peered":"2024-03-30T01:43:12.282370+0000","last_clean":"2024-03-30T01:43:12.282370+0000","last_became_active":"2024-03-30T01:43:12.276293+0000","last_became_peered":"2024-03-30T01:43:12.276293+0000","last_unstale":"2024-03-30T01:43:12.282370+0000","last_undegraded":"2024-03-30T01:43:12.282370+0000","last_fullsized":"2024-03-30T01:43:12.282370+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270389+0000","last_change":"2024-03-30T01:43:11.803525+0000","last_active":"2024-03-30T01:43:12.270389+0000","last_peered":"2024-03-30T01:43:12.270389+0000","last_clean":"2024-03-30T01:43:12.270389+0000","last_became_active":"2024-03-30T01:43:11.803115+0000","last_became_peered":"2024-03-30T01:43:11.803115+0000","last_unstale":"2024-03-30T01:43:12.270389+0000","last_undegraded":"2024-03-30T01:43:12.270389+0000","last_fullsized":"2024-03-30T01:43:12.270389+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:10.022632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271495+0000","last_change":"2024-03-30T01:43:12.267451+0000","last_active":"2024-03-30T01:43:12.271495+0000","last_peered":"2024-03-30T01:43:12.271495+0000","last_clean":"2024-03-30T01:43:12.271495+0000","last_became_active":"2024-03-30T01:43:12.267249+0000","last_became_peered":"2024-03-30T01:43:12.267249+0000","last_unstale":"2024-03-30T01:43:12.271495+0000","last_undegraded":"2024-03-30T01:43:12.271495+0000","last_fullsized":"2024-03-30T01:43:12.271495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:49.333290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271580+0000","last_change":"2024-03-30T01:43:11.804981+0000","last_active":"2024-03-30T01:43:12.271580+0000","last_peered":"2024-03-30T01:43:12.271580+0000","last_clean":"2024-03-30T01:43:12.271580+0000","last_became_active":"2024-03-30T01:43:11.804690+0000","last_became_peered":"2024-03-30T01:43:11.804690+0000","last_unstale":"2024-03-30T01:43:12.271580+0000","last_undegraded":"2024-03-30T01:43:12.271580+0000","last_fullsized":"2024-03-30T01:43:12.271580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:25:59.487668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270642+0000","last_change":"2024-03-30T01:43:12.268426+0000","last_active":"2024-03-30T01:43:12.270642+0000","last_peered":"2024-03-30T01:43:12.270642+0000","last_clean":"2024-03-30T01:43:12.270642+0000","last_became_active":"2024-03-30T01:43:12.268106+0000","last_became_peered":"2024-03-30T01:43:12.268106+0000","last_unstale":"2024-03-30T01:43:12.270642+0000","last_undegraded":"2024-03-30T01:43:12.270642+0000","last_fullsized":"2024-03-30T01:43:12.270642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:32:07.610941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271583+0000","last_change":"2024-03-30T01:43:11.798840+0000","last_active":"2024-03-30T01:43:12.271583+0000","last_peered":"2024-03-30T01:43:12.271583+0000","last_clean":"2024-03-30T01:43:12.271583+0000","last_became_active":"2024-03-30T01:43:11.798493+0000","last_became_peered":"2024-03-30T01:43:11.798493+0000","last_unstale":"2024-03-30T01:43:12.271583+0000","last_undegraded":"2024-03-30T01:43:12.271583+0000","last_fullsized":"2024-03-30T01:43:12.271583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:35.032315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.276646+0000","last_change":"2024-03-30T01:43:12.272426+0000","last_active":"2024-03-30T01:43:12.276646+0000","last_peered":"2024-03-30T01:43:12.276646+0000","last_clean":"2024-03-30T01:43:12.276646+0000","last_became_active":"2024-03-30T01:43:12.272160+0000","last_became_peered":"2024-03-30T01:43:12.272160+0000","last_unstale":"2024-03-30T01:43:12.276646+0000","last_undegraded":"2024-03-30T01:43:12.276646+0000","last_fullsized":"2024-03-30T01:43:12.276646+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:37.825651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.249041+0000","last_change":"2024-03-30T01:43:11.805468+0000","last_active":"2024-03-30T01:43:12.249041+0000","last_peered":"2024-03-30T01:43:12.249041+0000","last_clean":"2024-03-30T01:43:12.249041+0000","last_became_active":"2024-03-30T01:43:11.804984+0000","last_became_peered":"2024-03-30T01:43:11.804984+0000","last_unstale":"2024-03-30T01:43:12.249041+0000","last_undegraded":"2024-03-30T01:43:12.249041+0000","last_fullsized":"2024-03-30T01:43:12.249041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:16.171589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271678+0000","last_change":"2024-03-30T01:43:12.265005+0000","last_active":"2024-03-30T01:43:12.271678+0000","last_peered":"2024-03-30T01:43:12.271678+0000","last_clean":"2024-03-30T01:43:12.271678+0000","last_became_active":"2024-03-30T01:43:12.264726+0000","last_became_peered":"2024-03-30T01:43:12.264726+0000","last_unstale":"2024-03-30T01:43:12.271678+0000","last_undegraded":"2024-03-30T01:43:12.271678+0000","last_fullsized":"2024-03-30T01:43:12.271678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:41:19.989099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.256348+0000","last_change":"2024-03-30T01:43:11.799020+0000","last_active":"2024-03-30T01:43:12.256348+0000","last_peered":"2024-03-30T01:43:12.256348+0000","last_clean":"2024-03-30T01:43:12.256348+0000","last_became_active":"2024-03-30T01:43:11.797840+0000","last_became_peered":"2024-03-30T01:43:11.797840+0000","last_unstale":"2024-03-30T01:43:12.256348+0000","last_undegraded":"2024-03-30T01:43:12.256348+0000","last_fullsized":"2024-03-30T01:43:12.256348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:48:17.963383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.269985+0000","last_change":"2024-03-30T01:43:12.265237+0000","last_active":"2024-03-30T01:43:12.269985+0000","last_peered":"2024-03-30T01:43:12.269985+0000","last_clean":"2024-03-30T01:43:12.269985+0000","last_became_active":"2024-03-30T01:43:12.264995+0000","last_became_peered":"2024-03-30T01:43:12.264995+0000","last_unstale":"2024-03-30T01:43:12.269985+0000","last_undegraded":"2024-03-30T01:43:12.269985+0000","last_fullsized":"2024-03-30T01:43:12.269985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.256029+0000","last_change":"2024-03-30T01:43:11.803287+0000","last_active":"2024-03-30T01:43:12.256029+0000","last_peered":"2024-03-30T01:43:12.256029+0000","last_clean":"2024-03-30T01:43:12.256029+0000","last_became_active":"2024-03-30T01:43:11.803067+0000","last_became_peered":"2024-03-30T01:43:11.803067+0000","last_unstale":"2024-03-30T01:43:12.256029+0000","last_undegraded":"2024-03-30T01:43:12.256029+0000","last_fullsized":"2024-03-30T01:43:12.256029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:36.821233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282452+0000","last_change":"2024-03-30T01:43:12.275223+0000","last_active":"2024-03-30T01:43:12.282452+0000","last_peered":"2024-03-30T01:43:12.282452+0000","last_clean":"2024-03-30T01:43:12.282452+0000","last_became_active":"2024-03-30T01:43:12.274949+0000","last_became_peered":"2024-03-30T01:43:12.274949+0000","last_unstale":"2024-03-30T01:43:12.282452+0000","last_undegraded":"2024-03-30T01:43:12.282452+0000","last_fullsized":"2024-03-30T01:43:12.282452+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:34.024584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.266940+0000","last_change":"2024-03-30T01:43:11.802972+0000","last_active":"2024-03-30T01:43:12.266940+0000","last_peered":"2024-03-30T01:43:12.266940+0000","last_clean":"2024-03-30T01:43:12.266940+0000","last_became_active":"2024-03-30T01:43:11.802665+0000","last_became_peered":"2024-03-30T01:43:11.802665+0000","last_unstale":"2024-03-30T01:43:12.266940+0000","last_undegraded":"2024-03-30T01:43:12.266940+0000","last_fullsized":"2024-03-30T01:43:12.266940+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:24.494514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.274408+0000","last_change":"2024-03-30T01:43:12.271580+0000","last_active":"2024-03-30T01:43:12.274408+0000","last_peered":"2024-03-30T01:43:12.274408+0000","last_clean":"2024-03-30T01:43:12.274408+0000","last_became_active":"2024-03-30T01:43:12.271342+0000","last_became_peered":"2024-03-30T01:43:12.271342+0000","last_unstale":"2024-03-30T01:43:12.274408+0000","last_undegraded":"2024-03-30T01:43:12.274408+0000","last_fullsized":"2024-03-30T01:43:12.274408+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:25.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.278679+0000","last_change":"2024-03-30T01:43:11.802252+0000","last_active":"2024-03-30T01:43:12.278679+0000","last_peered":"2024-03-30T01:43:12.278679+0000","last_clean":"2024-03-30T01:43:12.278679+0000","last_became_active":"2024-03-30T01:43:11.801845+0000","last_became_peered":"2024-03-30T01:43:11.801845+0000","last_unstale":"2024-03-30T01:43:12.278679+0000","last_undegraded":"2024-03-30T01:43:12.278679+0000","last_fullsized":"2024-03-30T01:43:12.278679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279509+0000","last_change":"2024-03-30T01:43:12.268100+0000","last_active":"2024-03-30T01:43:12.279509+0000","last_peered":"2024-03-30T01:43:12.279509+0000","last_clean":"2024-03-30T01:43:12.279509+0000","last_became_active":"2024-03-30T01:43:12.267755+0000","last_became_peered":"2024-03-30T01:43:12.267755+0000","last_unstale":"2024-03-30T01:43:12.279509+0000","last_undegraded":"2024-03-30T01:43:12.279509+0000","last_fullsized":"2024-03-30T01:43:12.279509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282865+0000","last_change":"2024-03-30T01:43:11.806909+0000","last_active":"2024-03-30T01:43:12.282865+0000","last_peered":"2024-03-30T01:43:12.282865+0000","last_clean":"2024-03-30T01:43:12.282865+0000","last_became_active":"2024-03-30T01:43:11.806609+0000","last_became_peered":"2024-03-30T01:43:11.806609+0000","last_unstale":"2024-03-30T01:43:12.282865+0000","last_undegraded":"2024-03-30T01:43:12.282865+0000","last_fullsized":"2024-03-30T01:43:12.282865+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282323+0000","last_change":"2024-03-30T01:43:12.280292+0000","last_active":"2024-03-30T01:43:12.282323+0000","last_peered":"2024-03-30T01:43:12.282323+0000","last_clean":"2024-03-30T01:43:12.282323+0000","last_became_active":"2024-03-30T01:43:12.279966+0000","last_became_peered":"2024-03-30T01:43:12.279966+0000","last_unstale":"2024-03-30T01:43:12.282323+0000","last_undegraded":"2024-03-30T01:43:12.282323+0000","last_fullsized":"2024-03-30T01:43:12.282323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.272246+0000","last_change":"2024-03-30T01:43:11.804635+0000","last_active":"2024-03-30T01:43:12.272246+0000","last_peered":"2024-03-30T01:43:12.272246+0000","last_clean":"2024-03-30T01:43:12.272246+0000","last_became_active":"2024-03-30T01:43:11.804355+0000","last_became_peered":"2024-03-30T01:43:11.804355+0000","last_unstale":"2024-03-30T01:43:12.272246+0000","last_undegraded":"2024-03-30T01:43:12.272246+0000","last_fullsized":"2024-03-30T01:43:12.272246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:57.117126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.272575+0000","last_change":"2024-03-30T01:43:12.268978+0000","last_active":"2024-03-30T01:43:12.272575+0000","last_peered":"2024-03-30T01:43:12.272575+0000","last_clean":"2024-03-30T01:43:12.272575+0000","last_became_active":"2024-03-30T01:43:12.268633+0000","last_became_peered":"2024-03-30T01:43:12.268633+0000","last_unstale":"2024-03-30T01:43:12.272575+0000","last_undegraded":"2024-03-30T01:43:12.272575+0000","last_fullsized":"2024-03-30T01:43:12.272575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.255367+0000","last_change":"2024-03-30T01:43:11.811100+0000","last_active":"2024-03-30T01:43:12.255367+0000","last_peered":"2024-03-30T01:43:12.255367+0000","last_clean":"2024-03-30T01:43:12.255367+0000","last_became_active":"2024-03-30T01:43:11.810811+0000","last_became_peered":"2024-03-30T01:43:11.810811+0000","last_unstale":"2024-03-30T01:43:12.255367+0000","last_undegraded":"2024-03-30T01:43:12.255367+0000","last_fullsized":"2024-03-30T01:43:12.255367+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.267805+0000","last_change":"2024-03-30T01:43:12.265729+0000","last_active":"2024-03-30T01:43:12.267805+0000","last_peered":"2024-03-30T01:43:12.267805+0000","last_clean":"2024-03-30T01:43:12.267805+0000","last_became_active":"2024-03-30T01:43:12.265427+0000","last_became_peered":"2024-03-30T01:43:12.265427+0000","last_unstale":"2024-03-30T01:43:12.267805+0000","last_undegraded":"2024-03-30T01:43:12.267805+0000","last_fullsized":"2024-03-30T01:43:12.267805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:40:39.587585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.257366+0000","last_change":"2024-03-30T01:43:11.802626+0000","last_active":"2024-03-30T01:43:12.257366+0000","last_peered":"2024-03-30T01:43:12.257366+0000","last_clean":"2024-03-30T01:43:12.257366+0000","last_became_active":"2024-03-30T01:43:11.802314+0000","last_became_peered":"2024-03-30T01:43:11.802314+0000","last_unstale":"2024-03-30T01:43:12.257366+0000","last_undegraded":"2024-03-30T01:43:12.257366+0000","last_fullsized":"2024-03-30T01:43:12.257366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279701+0000","last_change":"2024-03-30T01:43:12.274115+0000","last_active":"2024-03-30T01:43:12.279701+0000","last_peered":"2024-03-30T01:43:12.279701+0000","last_clean":"2024-03-30T01:43:12.279701+0000","last_became_active":"2024-03-30T01:43:12.273842+0000","last_became_peered":"2024-03-30T01:43:12.273842+0000","last_unstale":"2024-03-30T01:43:12.279701+0000","last_undegraded":"2024-03-30T01:43:12.279701+0000","last_fullsized":"2024-03-30T01:43:12.279701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:44:49.086267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.283586+0000","last_change":"2024-03-30T01:43:11.802309+0000","last_active":"2024-03-30T01:43:12.283586+0000","last_peered":"2024-03-30T01:43:12.283586+0000","last_clean":"2024-03-30T01:43:12.283586+0000","last_became_active":"2024-03-30T01:43:11.802008+0000","last_became_peered":"2024-03-30T01:43:11.802008+0000","last_unstale":"2024-03-30T01:43:12.283586+0000","last_undegraded":"2024-03-30T01:43:12.283586+0000","last_fullsized":"2024-03-30T01:43:12.283586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:11.268414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282670+0000","last_change":"2024-03-30T01:43:12.279554+0000","last_active":"2024-03-30T01:43:12.282670+0000","last_peered":"2024-03-30T01:43:12.282670+0000","last_clean":"2024-03-30T01:43:12.282670+0000","last_became_active":"2024-03-30T01:43:12.279266+0000","last_became_peered":"2024-03-30T01:43:12.279266+0000","last_unstale":"2024-03-30T01:43:12.282670+0000","last_undegraded":"2024-03-30T01:43:12.282670+0000","last_fullsized":"2024-03-30T01:43:12.282670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.273448+0000","last_change":"2024-03-30T01:43:11.809877+0000","last_active":"2024-03-30T01:43:12.273448+0000","last_peered":"2024-03-30T01:43:12.273448+0000","last_clean":"2024-03-30T01:43:12.273448+0000","last_became_active":"2024-03-30T01:43:11.808130+0000","last_became_peered":"2024-03-30T01:43:11.808130+0000","last_unstale":"2024-03-30T01:43:12.273448+0000","last_undegraded":"2024-03-30T01:43:12.273448+0000","last_fullsized":"2024-03-30T01:43:12.273448+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281930+0000","last_change":"2024-03-30T01:43:12.275996+0000","last_active":"2024-03-30T01:43:12.281930+0000","last_peered":"2024-03-30T01:43:12.281930+0000","last_clean":"2024-03-30T01:43:12.281930+0000","last_became_active":"2024-03-30T01:43:12.275705+0000","last_became_peered":"2024-03-30T01:43:12.275705+0000","last_unstale":"2024-03-30T01:43:12.281930+0000","last_undegraded":"2024-03-30T01:43:12.281930+0000","last_fullsized":"2024-03-30T01:43:12.281930+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:47.117365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279812+0000","last_change":"2024-03-30T01:43:11.806234+0000","last_active":"2024-03-30T01:43:12.279812+0000","last_peered":"2024-03-30T01:43:12.279812+0000","last_clean":"2024-03-30T01:43:12.279812+0000","last_became_active":"2024-03-30T01:43:11.806012+0000","last_became_peered":"2024-03-30T01:43:11.806012+0000","last_unstale":"2024-03-30T01:43:12.279812+0000","last_undegraded":"2024-03-30T01:43:12.279812+0000","last_fullsized":"2024-03-30T01:43:12.279812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:24.494514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.280046+0000","last_change":"2024-03-30T01:43:12.271940+0000","last_active":"2024-03-30T01:43:12.280046+0000","last_peered":"2024-03-30T01:43:12.280046+0000","last_clean":"2024-03-30T01:43:12.280046+0000","last_became_active":"2024-03-30T01:43:12.268234+0000","last_became_peered":"2024-03-30T01:43:12.268234+0000","last_unstale":"2024-03-30T01:43:12.280046+0000","last_undegraded":"2024-03-30T01:43:12.280046+0000","last_fullsized":"2024-03-30T01:43:12.280046+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.278250+0000","last_change":"2024-03-30T01:43:11.803111+0000","last_active":"2024-03-30T01:43:12.278250+0000","last_peered":"2024-03-30T01:43:12.278250+0000","last_clean":"2024-03-30T01:43:12.278250+0000","last_became_active":"2024-03-30T01:43:11.802781+0000","last_became_peered":"2024-03-30T01:43:11.802781+0000","last_unstale":"2024-03-30T01:43:12.278250+0000","last_undegraded":"2024-03-30T01:43:12.278250+0000","last_fullsized":"2024-03-30T01:43:12.278250+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:30.914337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.275437+0000","last_change":"2024-03-30T01:43:12.272623+0000","last_active":"2024-03-30T01:43:12.275437+0000","last_peered":"2024-03-30T01:43:12.275437+0000","last_clean":"2024-03-30T01:43:12.275437+0000","last_became_active":"2024-03-30T01:43:12.272330+0000","last_became_peered":"2024-03-30T01:43:12.272330+0000","last_unstale":"2024-03-30T01:43:12.275437+0000","last_undegraded":"2024-03-30T01:43:12.275437+0000","last_fullsized":"2024-03-30T01:43:12.275437+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:11.229993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.249731+0000","last_change":"2024-03-30T01:43:11.802228+0000","last_active":"2024-03-30T01:43:12.249731+0000","last_peered":"2024-03-30T01:43:12.249731+0000","last_clean":"2024-03-30T01:43:12.249731+0000","last_became_active":"2024-03-30T01:43:11.801742+0000","last_became_peered":"2024-03-30T01:43:11.801742+0000","last_unstale":"2024-03-30T01:43:12.249731+0000","last_undegraded":"2024-03-30T01:43:12.249731+0000","last_fullsized":"2024-03-30T01:43:12.249731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:01.225672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270316+0000","last_change":"2024-03-30T01:43:12.259920+0000","last_active":"2024-03-30T01:43:12.270316+0000","last_peered":"2024-03-30T01:43:12.270316+0000","last_clean":"2024-03-30T01:43:12.270316+0000","last_became_active":"2024-03-30T01:43:12.259636+0000","last_became_peered":"2024-03-30T01:43:12.259636+0000","last_unstale":"2024-03-30T01:43:12.270316+0000","last_undegraded":"2024-03-30T01:43:12.270316+0000","last_fullsized":"2024-03-30T01:43:12.270316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:45:42.926912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270208+0000","last_change":"2024-03-30T01:43:11.804854+0000","last_active":"2024-03-30T01:43:12.270208+0000","last_peered":"2024-03-30T01:43:12.270208+0000","last_clean":"2024-03-30T01:43:12.270208+0000","last_became_active":"2024-03-30T01:43:11.804593+0000","last_became_peered":"2024-03-30T01:43:11.804593+0000","last_unstale":"2024-03-30T01:43:12.270208+0000","last_undegraded":"2024-03-30T01:43:12.270208+0000","last_fullsized":"2024-03-30T01:43:12.270208+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:45:22.282372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.272614+0000","last_change":"2024-03-30T01:43:12.269648+0000","last_active":"2024-03-30T01:43:12.272614+0000","last_peered":"2024-03-30T01:43:12.272614+0000","last_clean":"2024-03-30T01:43:12.272614+0000","last_became_active":"2024-03-30T01:43:12.269295+0000","last_became_peered":"2024-03-30T01:43:12.269295+0000","last_unstale":"2024-03-30T01:43:12.272614+0000","last_undegraded":"2024-03-30T01:43:12.272614+0000","last_fullsized":"2024-03-30T01:43:12.272614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.255416+0000","last_change":"2024-03-30T01:43:11.811090+0000","last_active":"2024-03-30T01:43:12.255416+0000","last_peered":"2024-03-30T01:43:12.255416+0000","last_clean":"2024-03-30T01:43:12.255416+0000","last_became_active":"2024-03-30T01:43:11.810767+0000","last_became_peered":"2024-03-30T01:43:11.810767+0000","last_unstale":"2024-03-30T01:43:12.255416+0000","last_undegraded":"2024-03-30T01:43:12.255416+0000","last_fullsized":"2024-03-30T01:43:12.255416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.267443+0000","last_change":"2024-03-30T01:43:12.265208+0000","last_active":"2024-03-30T01:43:12.267443+0000","last_peered":"2024-03-30T01:43:12.267443+0000","last_clean":"2024-03-30T01:43:12.267443+0000","last_became_active":"2024-03-30T01:43:12.264919+0000","last_became_peered":"2024-03-30T01:43:12.264919+0000","last_unstale":"2024-03-30T01:43:12.267443+0000","last_undegraded":"2024-03-30T01:43:12.267443+0000","last_fullsized":"2024-03-30T01:43:12.267443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:31.918755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.254828+0000","last_change":"2024-03-30T01:43:11.803507+0000","last_active":"2024-03-30T01:43:12.254828+0000","last_peered":"2024-03-30T01:43:12.254828+0000","last_clean":"2024-03-30T01:43:12.254828+0000","last_became_active":"2024-03-30T01:43:11.803197+0000","last_became_peered":"2024-03-30T01:43:11.803197+0000","last_unstale":"2024-03-30T01:43:12.254828+0000","last_undegraded":"2024-03-30T01:43:12.254828+0000","last_fullsized":"2024-03-30T01:43:12.254828+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:41:18.984681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282297+0000","last_change":"2024-03-30T01:43:12.275594+0000","last_active":"2024-03-30T01:43:12.282297+0000","last_peered":"2024-03-30T01:43:12.282297+0000","last_clean":"2024-03-30T01:43:12.282297+0000","last_became_active":"2024-03-30T01:43:12.275249+0000","last_became_peered":"2024-03-30T01:43:12.275249+0000","last_unstale":"2024-03-30T01:43:12.282297+0000","last_undegraded":"2024-03-30T01:43:12.282297+0000","last_fullsized":"2024-03-30T01:43:12.282297+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:09.869640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.275230+0000","last_change":"2024-03-30T01:43:11.810058+0000","last_active":"2024-03-30T01:43:12.275230+0000","last_peered":"2024-03-30T01:43:12.275230+0000","last_clean":"2024-03-30T01:43:12.275230+0000","last_became_active":"2024-03-30T01:43:11.808406+0000","last_became_peered":"2024-03-30T01:43:11.808406+0000","last_unstale":"2024-03-30T01:43:12.275230+0000","last_undegraded":"2024-03-30T01:43:12.275230+0000","last_fullsized":"2024-03-30T01:43:12.275230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:57.117126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.261110+0000","last_change":"2024-03-30T01:43:12.259473+0000","last_active":"2024-03-30T01:43:12.261110+0000","last_peered":"2024-03-30T01:43:12.261110+0000","last_clean":"2024-03-30T01:43:12.261110+0000","last_became_active":"2024-03-30T01:43:12.259194+0000","last_became_peered":"2024-03-30T01:43:12.259194+0000","last_unstale":"2024-03-30T01:43:12.261110+0000","last_undegraded":"2024-03-30T01:43:12.261110+0000","last_fullsized":"2024-03-30T01:43:12.261110+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:14.284517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.260397+0000","last_change":"2024-03-30T01:43:11.810112+0000","last_active":"2024-03-30T01:43:12.260397+0000","last_peered":"2024-03-30T01:43:12.260397+0000","last_clean":"2024-03-30T01:43:12.260397+0000","last_became_active":"2024-03-30T01:43:11.808349+0000","last_became_peered":"2024-03-30T01:43:11.808349+0000","last_unstale":"2024-03-30T01:43:12.260397+0000","last_undegraded":"2024-03-30T01:43:12.260397+0000","last_fullsized":"2024-03-30T01:43:12.260397+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:30.914337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271706+0000","last_change":"2024-03-30T01:43:12.265689+0000","last_active":"2024-03-30T01:43:12.271706+0000","last_peered":"2024-03-30T01:43:12.271706+0000","last_clean":"2024-03-30T01:43:12.271706+0000","last_became_active":"2024-03-30T01:43:12.265382+0000","last_became_peered":"2024-03-30T01:43:12.265382+0000","last_unstale":"2024-03-30T01:43:12.271706+0000","last_undegraded":"2024-03-30T01:43:12.271706+0000","last_fullsized":"2024-03-30T01:43:12.271706+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:17.176008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.256736+0000","last_change":"2024-03-30T01:43:11.811154+0000","last_active":"2024-03-30T01:43:12.256736+0000","last_peered":"2024-03-30T01:43:12.256736+0000","last_clean":"2024-03-30T01:43:12.256736+0000","last_became_active":"2024-03-30T01:43:11.810455+0000","last_became_peered":"2024-03-30T01:43:11.810455+0000","last_unstale":"2024-03-30T01:43:12.256736+0000","last_undegraded":"2024-03-30T01:43:12.256736+0000","last_fullsized":"2024-03-30T01:43:12.256736+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:33.020166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.267892+0000","last_change":"2024-03-30T01:43:12.265758+0000","last_active":"2024-03-30T01:43:12.267892+0000","last_peered":"2024-03-30T01:43:12.267892+0000","last_clean":"2024-03-30T01:43:12.267892+0000","last_became_active":"2024-03-30T01:43:12.265461+0000","last_became_peered":"2024-03-30T01:43:12.265461+0000","last_unstale":"2024-03-30T01:43:12.267892+0000","last_undegraded":"2024-03-30T01:43:12.267892+0000","last_fullsized":"2024-03-30T01:43:12.267892+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:36.552215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.253707+0000","last_change":"2024-03-30T01:43:11.806860+0000","last_active":"2024-03-30T01:43:12.253707+0000","last_peered":"2024-03-30T01:43:12.253707+0000","last_clean":"2024-03-30T01:43:12.253707+0000","last_became_active":"2024-03-30T01:43:11.806563+0000","last_became_peered":"2024-03-30T01:43:11.806563+0000","last_unstale":"2024-03-30T01:43:12.253707+0000","last_undegraded":"2024-03-30T01:43:12.253707+0000","last_fullsized":"2024-03-30T01:43:12.253707+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.274501+0000","last_change":"2024-03-30T01:43:12.269888+0000","last_active":"2024-03-30T01:43:12.274501+0000","last_peered":"2024-03-30T01:43:12.274501+0000","last_clean":"2024-03-30T01:43:12.274501+0000","last_became_active":"2024-03-30T01:43:12.269582+0000","last_became_peered":"2024-03-30T01:43:12.269582+0000","last_unstale":"2024-03-30T01:43:12.274501+0000","last_undegraded":"2024-03-30T01:43:12.274501+0000","last_fullsized":"2024-03-30T01:43:12.274501+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:58.121544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.258139+0000","last_change":"2024-03-30T01:43:11.802541+0000","last_active":"2024-03-30T01:43:12.258139+0000","last_peered":"2024-03-30T01:43:12.258139+0000","last_clean":"2024-03-30T01:43:12.258139+0000","last_became_active":"2024-03-30T01:43:11.802154+0000","last_became_peered":"2024-03-30T01:43:11.802154+0000","last_unstale":"2024-03-30T01:43:12.258139+0000","last_undegraded":"2024-03-30T01:43:12.258139+0000","last_fullsized":"2024-03-30T01:43:12.258139+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282160+0000","last_change":"2024-03-30T01:43:12.279403+0000","last_active":"2024-03-30T01:43:12.282160+0000","last_peered":"2024-03-30T01:43:12.282160+0000","last_clean":"2024-03-30T01:43:12.282160+0000","last_became_active":"2024-03-30T01:43:12.279071+0000","last_became_peered":"2024-03-30T01:43:12.279071+0000","last_unstale":"2024-03-30T01:43:12.282160+0000","last_undegraded":"2024-03-30T01:43:12.282160+0000","last_fullsized":"2024-03-30T01:43:12.282160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.249559+0000","last_change":"2024-03-30T01:43:11.799428+0000","last_active":"2024-03-30T01:43:12.249559+0000","last_peered":"2024-03-30T01:43:12.249559+0000","last_clean":"2024-03-30T01:43:12.249559+0000","last_became_active":"2024-03-30T01:43:11.799147+0000","last_became_peered":"2024-03-30T01:43:11.799147+0000","last_unstale":"2024-03-30T01:43:12.249559+0000","last_undegraded":"2024-03-30T01:43:12.249559+0000","last_fullsized":"2024-03-30T01:43:12.249559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:56.396431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279656+0000","last_change":"2024-03-30T01:43:12.273079+0000","last_active":"2024-03-30T01:43:12.279656+0000","last_peered":"2024-03-30T01:43:12.279656+0000","last_clean":"2024-03-30T01:43:12.279656+0000","last_became_active":"2024-03-30T01:43:12.272766+0000","last_became_peered":"2024-03-30T01:43:12.272766+0000","last_unstale":"2024-03-30T01:43:12.279656+0000","last_undegraded":"2024-03-30T01:43:12.279656+0000","last_fullsized":"2024-03-30T01:43:12.279656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:57.882115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.249145+0000","last_change":"2024-03-30T01:43:11.805262+0000","last_active":"2024-03-30T01:43:12.249145+0000","last_peered":"2024-03-30T01:43:12.249145+0000","last_clean":"2024-03-30T01:43:12.249145+0000","last_became_active":"2024-03-30T01:43:11.804622+0000","last_became_peered":"2024-03-30T01:43:11.804622+0000","last_unstale":"2024-03-30T01:43:12.249145+0000","last_undegraded":"2024-03-30T01:43:12.249145+0000","last_fullsized":"2024-03-30T01:43:12.249145+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:08.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279590+0000","last_change":"2024-03-30T01:43:12.275816+0000","last_active":"2024-03-30T01:43:12.279590+0000","last_peered":"2024-03-30T01:43:12.279590+0000","last_clean":"2024-03-30T01:43:12.279590+0000","last_became_active":"2024-03-30T01:43:12.275547+0000","last_became_peered":"2024-03-30T01:43:12.275547+0000","last_unstale":"2024-03-30T01:43:12.279590+0000","last_undegraded":"2024-03-30T01:43:12.279590+0000","last_fullsized":"2024-03-30T01:43:12.279590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:31.125780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.253921+0000","last_change":"2024-03-30T01:43:11.804870+0000","last_active":"2024-03-30T01:43:12.253921+0000","last_peered":"2024-03-30T01:43:12.253921+0000","last_clean":"2024-03-30T01:43:12.253921+0000","last_became_active":"2024-03-30T01:43:11.804580+0000","last_became_peered":"2024-03-30T01:43:11.804580+0000","last_unstale":"2024-03-30T01:43:12.253921+0000","last_undegraded":"2024-03-30T01:43:12.253921+0000","last_fullsized":"2024-03-30T01:43:12.253921+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.274510+0000","last_change":"2024-03-30T01:43:12.270228+0000","last_active":"2024-03-30T01:43:12.274510+0000","last_peered":"2024-03-30T01:43:12.274510+0000","last_clean":"2024-03-30T01:43:12.274510+0000","last_became_active":"2024-03-30T01:43:12.269910+0000","last_became_peered":"2024-03-30T01:43:12.269910+0000","last_unstale":"2024-03-30T01:43:12.274510+0000","last_undegraded":"2024-03-30T01:43:12.274510+0000","last_fullsized":"2024-03-30T01:43:12.274510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:00:52.920381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.248951+0000","last_change":"2024-03-30T01:43:11.803125+0000","last_active":"2024-03-30T01:43:12.248951+0000","last_peered":"2024-03-30T01:43:12.248951+0000","last_clean":"2024-03-30T01:43:12.248951+0000","last_became_active":"2024-03-30T01:43:11.802822+0000","last_became_peered":"2024-03-30T01:43:11.802822+0000","last_unstale":"2024-03-30T01:43:12.248951+0000","last_undegraded":"2024-03-30T01:43:12.248951+0000","last_fullsized":"2024-03-30T01:43:12.248951+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:21.501027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-03-30T01:43:11.793409+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:43:11.793409+0000","last_peered":"2024-03-30T01:43:11.793409+0000","last_clean":"2024-03-30T01:43:11.793409+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:43:11.793409+0000","last_undegraded":"2024-03-30T01:43:11.793409+0000","last_fullsized":"2024-03-30T01:43:11.793409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1294336,"data_stored":909691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.76400000000000001}]},{"osd":1,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72699999999999998,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.81200000000000006,"15min":0.85599999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72699999999999998,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.68899999999999995}]},{"osd":2,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.76200000000000001,"15min":0.79500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.73199999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73199999999999998,"5min":1.264,"15min":1.264},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.498}]},{"osd":4,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79300000000000004,"5min":1.075,"15min":1.075},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77200000000000002,"5min":1.833,"15min":1.833},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76300000000000001,"5min":1.238,"15min":1.238},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90600000000000003,"5min":1.2,"15min":1.2},"last":0.40600000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85099999999999998,"5min":1.288,"15min":1.288},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.68300000000000005,"15min":0.69799999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82499999999999996,"5min":1.901,"15min":1.901},"last":0.86399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":716800,"data_stored":326452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.94899999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":1.361,"15min":1.361},"last":0.67000000000000004}]},{"osd":1,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96799999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.71699999999999997,"15min":0.74399999999999999},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.54600000000000004}]},{"osd":2,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85599999999999998,"5min":1.27,"15min":1.27},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.409,"15min":1.409},"last":0.70299999999999996}]},{"osd":3,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.379,"5min":0.371,"15min":0.371},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.626}]},{"osd":4,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.72399999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":319323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.432,"15min":0.442},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.503,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":1.111,"15min":1.2270000000000001},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":15.038,"15min":15.038},"last":0.42799999999999999}]},{"osd":2,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.91400000000000003,"15min":1.006},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55500000000000005,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68400000000000005,"5min":1.444,"15min":1.444},"last":0.39400000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":1.474,"15min":1.474},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.624,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64000000000000001,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.35999999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.111,"15min":1.1970000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94499999999999995,"5min":11.99,"15min":11.99},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89900000000000002,"5min":1.407,"15min":1.407},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":1.335,"15min":1.4490000000000001},"min":{"1min":0.48799999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97699999999999998,"5min":15.006,"15min":15.006},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":1.2130000000000001,"15min":1.3069999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83199999999999996,"5min":11.926,"15min":11.926},"last":0.57199999999999995}]},{"osd":6,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.3149999999999999,"15min":1.429},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93100000000000005,"5min":14.974,"15min":14.974},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.1950000000000001,"15min":1.2949999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78100000000000003,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.504,"15min":1.504},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":1.343,"15min":1.4570000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.47499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1290240,"data_stored":908722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.57999999999999996,"15min":0.61499999999999999},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.439,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.54500000000000004,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.95499999999999996,"15min":1.05},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.443,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.86299999999999999,"15min":0.94299999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.503,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.59799999999999998,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.53300000000000003,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.443}]},{"osd":4,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.84699999999999998,"15min":0.875},"min":{"1min":0.47099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":5.431,"15min":5.431},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88300000000000001,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.80100000000000005,"15min":0.82899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77700000000000002,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.83799999999999997,"15min":0.87},"min":{"1min":0.47099999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.53200000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76500000000000001,"5min":1.177,"15min":1.177},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.82399999999999995,"15min":0.85399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.81899999999999995,"15min":0.84799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.88800000000000001,"15min":0.92700000000000005},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81899999999999995,"5min":5.468,"15min":5.468},"last":0.47699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":319323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.89000000000000001,"15min":0.96199999999999997},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.1890000000000001,"5min":12.478999999999999,"15min":12.478999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.753,"15min":0.79600000000000004},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1469999999999998,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.27800000000000002}]},{"osd":1,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53900000000000003,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.93899999999999995,"15min":1.0089999999999999},"min":{"1min":0.28999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.1309999999999998,"5min":12.425000000000001,"15min":12.425000000000001},"last":0.314}]},{"osd":3,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1459999999999999,"5min":1.55,"15min":1.55},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.98299999999999998,"15min":1.0620000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.8759999999999999,"5min":13.217000000000001,"15min":13.217000000000001},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.93899999999999995,"15min":0.97999999999999998},"min":{"1min":0.46400000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.956,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.95499999999999996,"15min":0.997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.0630000000000002,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.45100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89300000000000002,"5min":1.462,"15min":1.462},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.97499999999999998,"15min":1.0229999999999999},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.9159999999999999,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.97899999999999998,"15min":1.0229999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1139999999999999,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.59399999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.72099999999999997,"15min":0.73699999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.524,"15min":1.524},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.0660000000000001,"15min":1.105},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":3.105,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":0.65800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":319323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.47699999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.40400000000000003}]},{"osd":1,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.372,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.251},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41799999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.36799999999999999}]},{"osd":2,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45000000000000001,"15min":0.45400000000000001},"min":{"1min":0.33800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.53600000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.315},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45000000000000001,"15min":0.45400000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.44800000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76200000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91500000000000004,"5min":1.093,"15min":1.093},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.80800000000000005,"5min":1.042,"15min":1.042},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89200000000000002,"5min":1.012,"15min":1.012},"last":0.72899999999999998}]},{"osd":6,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.68899999999999995,"15min":0.69999999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.78900000000000003,"5min":1.024,"15min":1.024},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.78200000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.878,"5min":1.085,"15min":1.085},"last":0.66300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":319323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.67600000000000005,"15min":0.69499999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.87,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.66200000000000003}]},{"osd":1,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":1.0289999999999999,"15min":1.113},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.747,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.75900000000000001}]},{"osd":2,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76800000000000002,"5min":1.494,"15min":1.494},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.71299999999999997}]},{"osd":3,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.86599999999999999,"15min":0.91000000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":1.1279999999999999,"15min":1.218},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73499999999999999,"5min":9.452,"15min":9.452},"last":0.83599999999999997}]},{"osd":5,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78900000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76400000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.68500000000000005,"15min":0.69899999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82499999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89200000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.871}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":716800,"data_stored":326440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.83399999999999996,"15min":0.876},"min":{"1min":0.39800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.88400000000000001,"15min":0.93200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88200000000000001,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.502}]},{"osd":1,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.84999999999999998,"15min":0.89800000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81999999999999995,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.89400000000000002,"15min":0.93999999999999995},"min":{"1min":0.436,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94099999999999995,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.83999999999999997,"15min":0.88600000000000001},"min":{"1min":0.373,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80400000000000005,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77000000000000002,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.88800000000000001,"15min":0.94499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91500000000000004,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86499999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85299999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45400000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89000000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.627,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.63500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:43:16.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:43:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.665+0000 7f9172e49700 1 Processor -- start 2024-03-30T01:43:16.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.665+0000 7f9172e49700 1 -- start start 2024-03-30T01:43:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.665+0000 7f9172e49700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f916c06d5c0 0x7f916c155300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.665+0000 7f9172e49700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c06e0d0 0x7f916c0c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.665+0000 7f9172e49700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c1522e0 con 0x7f916c14e910 2024-03-30T01:43:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.665+0000 7f9172e49700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c152000 con 0x7f916c06d5c0 2024-03-30T01:43:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.665+0000 7f9172e49700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c154b00 con 0x7f916c06e0d0 2024-03-30T01:43:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f916bfff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f916c06d5c0 0x7f916c155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f9170be5700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f916c14e910 0x7f916c0c1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58848/0 (socket says 172.21.15.31:58848) 2024-03-30T01:43:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f9170be5700 1 -- 172.21.15.31:0/697462870 learned_addr learned my addr 172.21.15.31:0/697462870 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f91713e6700 1 --2- 172.21.15.31:0/697462870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c06e0d0 0x7f916c0c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f916bfff700 1 -- 172.21.15.31:0/697462870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c06e0d0 msgr2=0x7f916c0c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f916bfff700 1 --2- 172.21.15.31:0/697462870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c06e0d0 0x7f916c0c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f916bfff700 1 -- 172.21.15.31:0/697462870 >> v1:172.21.15.31:6789/0 conn(0x7f916c14e910 legacy=0x7f916c0c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f916b7fe700 1 -- 172.21.15.31:0/697462870 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302577210 0 0) 0x7f916c1522e0 con 0x7f916c14e910 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f916bfff700 1 -- 172.21.15.31:0/697462870 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c154de0 con 0x7f916c06d5c0 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f916bfff700 1 --2- 172.21.15.31:0/697462870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 0x7f916c155300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9160009840 tx=0x7f9160009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fadeb76a7c55e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.666+0000 7f916b7fe700 1 -- 172.21.15.31:0/697462870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160011830 con 0x7f916c06d5c0 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f916b7fe700 1 -- 172.21.15.31:0/697462870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9160011990 con 0x7f916c06d5c0 2024-03-30T01:43:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f916b7fe700 1 -- 172.21.15.31:0/697462870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160011d20 con 0x7f916c06d5c0 2024-03-30T01:43:16.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f9172e49700 1 -- 172.21.15.31:0/697462870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 msgr2=0x7f916c155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f9172e49700 1 --2- 172.21.15.31:0/697462870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 0x7f916c155300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9160009840 tx=0x7f9160009b50 comp rx=0 tx=0).stop 2024-03-30T01:43:16.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f9172e49700 1 -- 172.21.15.31:0/697462870 shutdown_connections 2024-03-30T01:43:16.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f9172e49700 1 --2- 172.21.15.31:0/697462870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c06e0d0 0x7f916c0c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f9172e49700 1 --2- 172.21.15.31:0/697462870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 0x7f916c155300 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f9172e49700 1 -- 172.21.15.31:0/697462870 >> 172.21.15.31:0/697462870 conn(0x7f916c06ba20 msgr2=0x7f916c06be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:16.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f9172e49700 1 -- 172.21.15.31:0/697462870 shutdown_connections 2024-03-30T01:43:16.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.667+0000 7f9172e49700 1 -- 172.21.15.31:0/697462870 wait complete. 2024-03-30T01:43:16.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.668+0000 7f9172e49700 1 Processor -- start 2024-03-30T01:43:16.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.669+0000 7f9172e49700 1 -- start start 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.669+0000 7f9172e49700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 0x7f916c0a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.669+0000 7f9172e49700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f916c06e0d0 0x7f916c14b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.669+0000 7f9172e49700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c14e910 0x7f916c144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.669+0000 7f9172e49700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0a9db0 con 0x7f916c06d5c0 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.669+0000 7f9172e49700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c1ec130 con 0x7f916c06e0d0 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.669+0000 7f9172e49700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c1eb370 con 0x7f916c14e910 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.669+0000 7f9170be5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 0x7f916c0a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9170be5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 0x7f916c0a9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54414/0 (socket says 172.21.15.31:54414) 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9170be5700 1 -- 172.21.15.31:0/2205354176 learned_addr learned my addr 172.21.15.31:0/2205354176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f91713e6700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c14e910 0x7f916c144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f916bfff700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f916c06e0d0 0x7f916c14b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9170be5700 1 -- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c14e910 msgr2=0x7f916c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9170be5700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c14e910 0x7f916c144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9170be5700 1 -- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f916c06e0d0 msgr2=0x7f916c14b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9170be5700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f916c06e0d0 0x7f916c14b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9170be5700 1 -- 172.21.15.31:0/2205354176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91600094a0 con 0x7f916c06d5c0 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f91713e6700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c14e910 0x7f916c144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f916bfff700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f916c06e0d0 0x7f916c14b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9170be5700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 0x7f916c0a9870 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f915c0011e0 tx=0x7f915c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f91697fa700 1 -- 172.21.15.31:0/2205354176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c004c80 con 0x7f916c06d5c0 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f91697fa700 1 -- 172.21.15.31:0/2205354176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c004e00 con 0x7f916c06d5c0 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f91697fa700 1 -- 172.21.15.31:0/2205354176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c007310 con 0x7f916c06d5c0 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c1eb650 con 0x7f916c06d5c0 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.670+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c1ebc10 con 0x7f916c06d5c0 2024-03-30T01:43:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.671+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c1ebe30 con 0x7f916c06d5c0 2024-03-30T01:43:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.672+0000 7f91697fa700 1 -- 172.21.15.31:0/2205354176 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f915c007470 con 0x7f916c06d5c0 2024-03-30T01:43:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.672+0000 7f91697fa700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f915806ac90 0x7f915806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.673+0000 7f91697fa700 1 -- 172.21.15.31:0/2205354176 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f915c0891d0 con 0x7f916c06d5c0 2024-03-30T01:43:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.673+0000 7f916bfff700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f915806ac90 0x7f915806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.673+0000 7f916bfff700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f915806ac90 0x7f915806d150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f916c1e8d90 tx=0x7f9160008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.677+0000 7f91697fa700 1 -- 172.21.15.31:0/2205354176 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f915c054300 con 0x7f916c06d5c0 2024-03-30T01:43:16.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.888+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f916c1eb4e0 con 0x7f915806ac90 2024-03-30T01:43:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.896+0000 7f91697fa700 1 -- 172.21.15.31:0/2205354176 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289194 (secure 0 0 0) 0x7f916c1eb4e0 con 0x7f915806ac90 2024-03-30T01:43:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.899+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f915806ac90 msgr2=0x7f915806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.899+0000 7f9172e49700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f915806ac90 0x7f915806d150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f916c1e8d90 tx=0x7f9160008030 comp rx=0 tx=0).stop 2024-03-30T01:43:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.899+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 msgr2=0x7f916c0a9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.899+0000 7f9172e49700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 0x7f916c0a9870 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f915c0011e0 tx=0x7f915c00cae0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.900+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 shutdown_connections 2024-03-30T01:43:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.900+0000 7f9172e49700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f916c06d5c0 0x7f916c0a9870 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.900+0000 7f9172e49700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f915806ac90 0x7f915806d150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.900+0000 7f9172e49700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f916c14e910 0x7f916c144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.900+0000 7f9172e49700 1 --2- 172.21.15.31:0/2205354176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f916c06e0d0 0x7f916c14b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:16.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.900+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 >> 172.21.15.31:0/2205354176 conn(0x7f916c06ba20 msgr2=0x7f916c1524c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:16.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.900+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 shutdown_connections 2024-03-30T01:43:16.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:16.900+0000 7f9172e49700 1 -- 172.21.15.31:0/2205354176 wait complete. 2024-03-30T01:43:16.910 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:43:16.920 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-03-30T01:43:15.839398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222852,"kb_used_data":6660,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751868,"statfs":{"total":773094113280,"available":772865912832,"internally_reserved":0,"allocated":6819840,"data_stored":3748597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.057389"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271679+0000","last_change":"2024-03-30T01:43:12.269722+0000","last_active":"2024-03-30T01:43:12.271679+0000","last_peered":"2024-03-30T01:43:12.271679+0000","last_clean":"2024-03-30T01:43:12.271679+0000","last_became_active":"2024-03-30T01:43:12.269445+0000","last_became_peered":"2024-03-30T01:43:12.269445+0000","last_unstale":"2024-03-30T01:43:12.271679+0000","last_undegraded":"2024-03-30T01:43:12.271679+0000","last_fullsized":"2024-03-30T01:43:12.271679+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.269072+0000","last_change":"2024-03-30T01:43:12.261636+0000","last_active":"2024-03-30T01:43:12.269072+0000","last_peered":"2024-03-30T01:43:12.269072+0000","last_clean":"2024-03-30T01:43:12.269072+0000","last_became_active":"2024-03-30T01:43:12.261289+0000","last_became_peered":"2024-03-30T01:43:12.261289+0000","last_unstale":"2024-03-30T01:43:12.269072+0000","last_undegraded":"2024-03-30T01:43:12.269072+0000","last_fullsized":"2024-03-30T01:43:12.269072+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:53.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282260+0000","last_change":"2024-03-30T01:43:12.273399+0000","last_active":"2024-03-30T01:43:12.282260+0000","last_peered":"2024-03-30T01:43:12.282260+0000","last_clean":"2024-03-30T01:43:12.282260+0000","last_became_active":"2024-03-30T01:43:12.273099+0000","last_became_peered":"2024-03-30T01:43:12.273099+0000","last_unstale":"2024-03-30T01:43:12.282260+0000","last_undegraded":"2024-03-30T01:43:12.282260+0000","last_fullsized":"2024-03-30T01:43:12.282260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281552+0000","last_change":"2024-03-30T01:43:12.277385+0000","last_active":"2024-03-30T01:43:12.281552+0000","last_peered":"2024-03-30T01:43:12.281552+0000","last_clean":"2024-03-30T01:43:12.281552+0000","last_became_active":"2024-03-30T01:43:12.277091+0000","last_became_peered":"2024-03-30T01:43:12.277091+0000","last_unstale":"2024-03-30T01:43:12.281552+0000","last_undegraded":"2024-03-30T01:43:12.281552+0000","last_fullsized":"2024-03-30T01:43:12.281552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:09:35.397960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281955+0000","last_change":"2024-03-30T01:43:12.276432+0000","last_active":"2024-03-30T01:43:12.281955+0000","last_peered":"2024-03-30T01:43:12.281955+0000","last_clean":"2024-03-30T01:43:12.281955+0000","last_became_active":"2024-03-30T01:43:12.276157+0000","last_became_peered":"2024-03-30T01:43:12.276157+0000","last_unstale":"2024-03-30T01:43:12.281955+0000","last_undegraded":"2024-03-30T01:43:12.281955+0000","last_fullsized":"2024-03-30T01:43:12.281955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:56.492934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.277103+0000","last_change":"2024-03-30T01:43:12.273311+0000","last_active":"2024-03-30T01:43:12.277103+0000","last_peered":"2024-03-30T01:43:12.277103+0000","last_clean":"2024-03-30T01:43:12.277103+0000","last_became_active":"2024-03-30T01:43:12.272939+0000","last_became_peered":"2024-03-30T01:43:12.272939+0000","last_unstale":"2024-03-30T01:43:12.277103+0000","last_undegraded":"2024-03-30T01:43:12.277103+0000","last_fullsized":"2024-03-30T01:43:12.277103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:38.756747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281349+0000","last_change":"2024-03-30T01:43:12.279192+0000","last_active":"2024-03-30T01:43:12.281349+0000","last_peered":"2024-03-30T01:43:12.281349+0000","last_clean":"2024-03-30T01:43:12.281349+0000","last_became_active":"2024-03-30T01:43:12.278893+0000","last_became_peered":"2024-03-30T01:43:12.278893+0000","last_unstale":"2024-03-30T01:43:12.281349+0000","last_undegraded":"2024-03-30T01:43:12.281349+0000","last_fullsized":"2024-03-30T01:43:12.281349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:21:09.466088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281307+0000","last_change":"2024-03-30T01:43:12.274078+0000","last_active":"2024-03-30T01:43:12.281307+0000","last_peered":"2024-03-30T01:43:12.281307+0000","last_clean":"2024-03-30T01:43:12.281307+0000","last_became_active":"2024-03-30T01:43:12.273703+0000","last_became_peered":"2024-03-30T01:43:12.273703+0000","last_unstale":"2024-03-30T01:43:12.281307+0000","last_undegraded":"2024-03-30T01:43:12.281307+0000","last_fullsized":"2024-03-30T01:43:12.281307+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282502+0000","last_change":"2024-03-30T01:43:12.274182+0000","last_active":"2024-03-30T01:43:12.282502+0000","last_peered":"2024-03-30T01:43:12.282502+0000","last_clean":"2024-03-30T01:43:12.282502+0000","last_became_active":"2024-03-30T01:43:12.273863+0000","last_became_peered":"2024-03-30T01:43:12.273863+0000","last_unstale":"2024-03-30T01:43:12.282502+0000","last_undegraded":"2024-03-30T01:43:12.282502+0000","last_fullsized":"2024-03-30T01:43:12.282502+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282458+0000","last_change":"2024-03-30T01:43:12.277023+0000","last_active":"2024-03-30T01:43:12.282458+0000","last_peered":"2024-03-30T01:43:12.282458+0000","last_clean":"2024-03-30T01:43:12.282458+0000","last_became_active":"2024-03-30T01:43:12.276685+0000","last_became_peered":"2024-03-30T01:43:12.276685+0000","last_unstale":"2024-03-30T01:43:12.282458+0000","last_undegraded":"2024-03-30T01:43:12.282458+0000","last_fullsized":"2024-03-30T01:43:12.282458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:53.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.280490+0000","last_change":"2024-03-30T01:43:12.273450+0000","last_active":"2024-03-30T01:43:12.280490+0000","last_peered":"2024-03-30T01:43:12.280490+0000","last_clean":"2024-03-30T01:43:12.280490+0000","last_became_active":"2024-03-30T01:43:12.273185+0000","last_became_peered":"2024-03-30T01:43:12.273185+0000","last_unstale":"2024-03-30T01:43:12.280490+0000","last_undegraded":"2024-03-30T01:43:12.280490+0000","last_fullsized":"2024-03-30T01:43:12.280490+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.280952+0000","last_change":"2024-03-30T01:43:12.273238+0000","last_active":"2024-03-30T01:43:12.280952+0000","last_peered":"2024-03-30T01:43:12.280952+0000","last_clean":"2024-03-30T01:43:12.280952+0000","last_became_active":"2024-03-30T01:43:12.272970+0000","last_became_peered":"2024-03-30T01:43:12.272970+0000","last_unstale":"2024-03-30T01:43:12.280952+0000","last_undegraded":"2024-03-30T01:43:12.280952+0000","last_fullsized":"2024-03-30T01:43:12.280952+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:49.877712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281107+0000","last_change":"2024-03-30T01:43:12.276660+0000","last_active":"2024-03-30T01:43:12.281107+0000","last_peered":"2024-03-30T01:43:12.281107+0000","last_clean":"2024-03-30T01:43:12.281107+0000","last_became_active":"2024-03-30T01:43:12.276437+0000","last_became_peered":"2024-03-30T01:43:12.276437+0000","last_unstale":"2024-03-30T01:43:12.281107+0000","last_undegraded":"2024-03-30T01:43:12.281107+0000","last_fullsized":"2024-03-30T01:43:12.281107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282427+0000","last_change":"2024-03-30T01:43:12.276823+0000","last_active":"2024-03-30T01:43:12.282427+0000","last_peered":"2024-03-30T01:43:12.282427+0000","last_clean":"2024-03-30T01:43:12.282427+0000","last_became_active":"2024-03-30T01:43:12.272988+0000","last_became_peered":"2024-03-30T01:43:12.272988+0000","last_unstale":"2024-03-30T01:43:12.282427+0000","last_undegraded":"2024-03-30T01:43:12.282427+0000","last_fullsized":"2024-03-30T01:43:12.282427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282708+0000","last_change":"2024-03-30T01:43:12.277931+0000","last_active":"2024-03-30T01:43:12.282708+0000","last_peered":"2024-03-30T01:43:12.282708+0000","last_clean":"2024-03-30T01:43:12.282708+0000","last_became_active":"2024-03-30T01:43:12.277601+0000","last_became_peered":"2024-03-30T01:43:12.277601+0000","last_unstale":"2024-03-30T01:43:12.282708+0000","last_undegraded":"2024-03-30T01:43:12.282708+0000","last_fullsized":"2024-03-30T01:43:12.282708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279537+0000","last_change":"2024-03-30T01:43:12.275724+0000","last_active":"2024-03-30T01:43:12.279537+0000","last_peered":"2024-03-30T01:43:12.279537+0000","last_clean":"2024-03-30T01:43:12.279537+0000","last_became_active":"2024-03-30T01:43:12.275438+0000","last_became_peered":"2024-03-30T01:43:12.275438+0000","last_unstale":"2024-03-30T01:43:12.279537+0000","last_undegraded":"2024-03-30T01:43:12.279537+0000","last_fullsized":"2024-03-30T01:43:12.279537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:18.506294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270111+0000","last_change":"2024-03-30T01:43:12.268431+0000","last_active":"2024-03-30T01:43:12.270111+0000","last_peered":"2024-03-30T01:43:12.270111+0000","last_clean":"2024-03-30T01:43:12.270111+0000","last_became_active":"2024-03-30T01:43:12.268148+0000","last_became_peered":"2024-03-30T01:43:12.268148+0000","last_unstale":"2024-03-30T01:43:12.270111+0000","last_undegraded":"2024-03-30T01:43:12.270111+0000","last_fullsized":"2024-03-30T01:43:12.270111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:49.877712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.280063+0000","last_change":"2024-03-30T01:43:12.275632+0000","last_active":"2024-03-30T01:43:12.280063+0000","last_peered":"2024-03-30T01:43:12.280063+0000","last_clean":"2024-03-30T01:43:12.280063+0000","last_became_active":"2024-03-30T01:43:12.275329+0000","last_became_peered":"2024-03-30T01:43:12.275329+0000","last_unstale":"2024-03-30T01:43:12.280063+0000","last_undegraded":"2024-03-30T01:43:12.280063+0000","last_fullsized":"2024-03-30T01:43:12.280063+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:44.982486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.267611+0000","last_change":"2024-03-30T01:43:12.265240+0000","last_active":"2024-03-30T01:43:12.267611+0000","last_peered":"2024-03-30T01:43:12.267611+0000","last_clean":"2024-03-30T01:43:12.267611+0000","last_became_active":"2024-03-30T01:43:12.264980+0000","last_became_peered":"2024-03-30T01:43:12.264980+0000","last_unstale":"2024-03-30T01:43:12.267611+0000","last_undegraded":"2024-03-30T01:43:12.267611+0000","last_fullsized":"2024-03-30T01:43:12.267611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:15:38.037358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271658+0000","last_change":"2024-03-30T01:43:12.266614+0000","last_active":"2024-03-30T01:43:12.271658+0000","last_peered":"2024-03-30T01:43:12.271658+0000","last_clean":"2024-03-30T01:43:12.271658+0000","last_became_active":"2024-03-30T01:43:12.266325+0000","last_became_peered":"2024-03-30T01:43:12.266325+0000","last_unstale":"2024-03-30T01:43:12.271658+0000","last_undegraded":"2024-03-30T01:43:12.271658+0000","last_fullsized":"2024-03-30T01:43:12.271658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:45:23.286791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.272595+0000","last_change":"2024-03-30T01:43:12.269577+0000","last_active":"2024-03-30T01:43:12.272595+0000","last_peered":"2024-03-30T01:43:12.272595+0000","last_clean":"2024-03-30T01:43:12.272595+0000","last_became_active":"2024-03-30T01:43:12.269263+0000","last_became_peered":"2024-03-30T01:43:12.269263+0000","last_unstale":"2024-03-30T01:43:12.272595+0000","last_undegraded":"2024-03-30T01:43:12.272595+0000","last_fullsized":"2024-03-30T01:43:12.272595+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282429+0000","last_change":"2024-03-30T01:43:12.276626+0000","last_active":"2024-03-30T01:43:12.282429+0000","last_peered":"2024-03-30T01:43:12.282429+0000","last_clean":"2024-03-30T01:43:12.282429+0000","last_became_active":"2024-03-30T01:43:12.276302+0000","last_became_peered":"2024-03-30T01:43:12.276302+0000","last_unstale":"2024-03-30T01:43:12.282429+0000","last_undegraded":"2024-03-30T01:43:12.282429+0000","last_fullsized":"2024-03-30T01:43:12.282429+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:11.027051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282442+0000","last_change":"2024-03-30T01:43:12.276658+0000","last_active":"2024-03-30T01:43:12.282442+0000","last_peered":"2024-03-30T01:43:12.282442+0000","last_clean":"2024-03-30T01:43:12.282442+0000","last_became_active":"2024-03-30T01:43:12.276363+0000","last_became_peered":"2024-03-30T01:43:12.276363+0000","last_unstale":"2024-03-30T01:43:12.282442+0000","last_undegraded":"2024-03-30T01:43:12.282442+0000","last_fullsized":"2024-03-30T01:43:12.282442+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:06.329557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282484+0000","last_change":"2024-03-30T01:43:12.273270+0000","last_active":"2024-03-30T01:43:12.282484+0000","last_peered":"2024-03-30T01:43:12.282484+0000","last_clean":"2024-03-30T01:43:12.282484+0000","last_became_active":"2024-03-30T01:43:12.272973+0000","last_became_peered":"2024-03-30T01:43:12.272973+0000","last_unstale":"2024-03-30T01:43:12.282484+0000","last_undegraded":"2024-03-30T01:43:12.282484+0000","last_fullsized":"2024-03-30T01:43:12.282484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.273511+0000","last_change":"2024-03-30T01:43:12.270679+0000","last_active":"2024-03-30T01:43:12.273511+0000","last_peered":"2024-03-30T01:43:12.273511+0000","last_clean":"2024-03-30T01:43:12.273511+0000","last_became_active":"2024-03-30T01:43:12.270457+0000","last_became_peered":"2024-03-30T01:43:12.270457+0000","last_unstale":"2024-03-30T01:43:12.273511+0000","last_undegraded":"2024-03-30T01:43:12.273511+0000","last_fullsized":"2024-03-30T01:43:12.273511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:45.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.273366+0000","last_change":"2024-03-30T01:43:12.269460+0000","last_active":"2024-03-30T01:43:12.273366+0000","last_peered":"2024-03-30T01:43:12.273366+0000","last_clean":"2024-03-30T01:43:12.273366+0000","last_became_active":"2024-03-30T01:43:12.269159+0000","last_became_peered":"2024-03-30T01:43:12.269159+0000","last_unstale":"2024-03-30T01:43:12.273366+0000","last_undegraded":"2024-03-30T01:43:12.273366+0000","last_fullsized":"2024-03-30T01:43:12.273366+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281445+0000","last_change":"2024-03-30T01:43:12.276051+0000","last_active":"2024-03-30T01:43:12.281445+0000","last_peered":"2024-03-30T01:43:12.281445+0000","last_clean":"2024-03-30T01:43:12.281445+0000","last_became_active":"2024-03-30T01:43:12.275296+0000","last_became_peered":"2024-03-30T01:43:12.275296+0000","last_unstale":"2024-03-30T01:43:12.281445+0000","last_undegraded":"2024-03-30T01:43:12.281445+0000","last_fullsized":"2024-03-30T01:43:12.281445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:17.176008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270043+0000","last_change":"2024-03-30T01:43:12.266882+0000","last_active":"2024-03-30T01:43:12.270043+0000","last_peered":"2024-03-30T01:43:12.270043+0000","last_clean":"2024-03-30T01:43:12.270043+0000","last_became_active":"2024-03-30T01:43:12.266450+0000","last_became_peered":"2024-03-30T01:43:12.266450+0000","last_unstale":"2024-03-30T01:43:12.270043+0000","last_undegraded":"2024-03-30T01:43:12.270043+0000","last_fullsized":"2024-03-30T01:43:12.270043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282693+0000","last_change":"2024-03-30T01:43:12.279605+0000","last_active":"2024-03-30T01:43:12.282693+0000","last_peered":"2024-03-30T01:43:12.282693+0000","last_clean":"2024-03-30T01:43:12.282693+0000","last_became_active":"2024-03-30T01:43:12.279323+0000","last_became_peered":"2024-03-30T01:43:12.279323+0000","last_unstale":"2024-03-30T01:43:12.282693+0000","last_undegraded":"2024-03-30T01:43:12.282693+0000","last_fullsized":"2024-03-30T01:43:12.282693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:32:07.610941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281138+0000","last_change":"2024-03-30T01:43:12.277845+0000","last_active":"2024-03-30T01:43:12.281138+0000","last_peered":"2024-03-30T01:43:12.281138+0000","last_clean":"2024-03-30T01:43:12.281138+0000","last_became_active":"2024-03-30T01:43:12.277482+0000","last_became_peered":"2024-03-30T01:43:12.277482+0000","last_unstale":"2024-03-30T01:43:12.281138+0000","last_undegraded":"2024-03-30T01:43:12.281138+0000","last_fullsized":"2024-03-30T01:43:12.281138+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281528+0000","last_change":"2024-03-30T01:43:12.276064+0000","last_active":"2024-03-30T01:43:12.281528+0000","last_peered":"2024-03-30T01:43:12.281528+0000","last_clean":"2024-03-30T01:43:12.281528+0000","last_became_active":"2024-03-30T01:43:12.275670+0000","last_became_peered":"2024-03-30T01:43:12.275670+0000","last_unstale":"2024-03-30T01:43:12.281528+0000","last_undegraded":"2024-03-30T01:43:12.281528+0000","last_fullsized":"2024-03-30T01:43:12.281528+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:45.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270338+0000","last_change":"2024-03-30T01:43:12.260768+0000","last_active":"2024-03-30T01:43:12.270338+0000","last_peered":"2024-03-30T01:43:12.270338+0000","last_clean":"2024-03-30T01:43:12.270338+0000","last_became_active":"2024-03-30T01:43:12.260512+0000","last_became_peered":"2024-03-30T01:43:12.260512+0000","last_unstale":"2024-03-30T01:43:12.270338+0000","last_undegraded":"2024-03-30T01:43:12.270338+0000","last_fullsized":"2024-03-30T01:43:12.270338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:59.579718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.274705+0000","last_change":"2024-03-30T01:43:12.270602+0000","last_active":"2024-03-30T01:43:12.274705+0000","last_peered":"2024-03-30T01:43:12.274705+0000","last_clean":"2024-03-30T01:43:12.274705+0000","last_became_active":"2024-03-30T01:43:12.270328+0000","last_became_peered":"2024-03-30T01:43:12.270328+0000","last_unstale":"2024-03-30T01:43:12.274705+0000","last_undegraded":"2024-03-30T01:43:12.274705+0000","last_fullsized":"2024-03-30T01:43:12.274705+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:24.040846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282085+0000","last_change":"2024-03-30T01:43:11.800815+0000","last_active":"2024-03-30T01:43:12.282085+0000","last_peered":"2024-03-30T01:43:12.282085+0000","last_clean":"2024-03-30T01:43:12.282085+0000","last_became_active":"2024-03-30T01:43:11.800527+0000","last_became_peered":"2024-03-30T01:43:11.800527+0000","last_unstale":"2024-03-30T01:43:12.282085+0000","last_undegraded":"2024-03-30T01:43:12.282085+0000","last_fullsized":"2024-03-30T01:43:12.282085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:36.821233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279794+0000","last_change":"2024-03-30T01:43:12.268225+0000","last_active":"2024-03-30T01:43:12.279794+0000","last_peered":"2024-03-30T01:43:12.279794+0000","last_clean":"2024-03-30T01:43:12.279794+0000","last_became_active":"2024-03-30T01:43:12.267769+0000","last_became_peered":"2024-03-30T01:43:12.267769+0000","last_unstale":"2024-03-30T01:43:12.279794+0000","last_undegraded":"2024-03-30T01:43:12.279794+0000","last_fullsized":"2024-03-30T01:43:12.279794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282134+0000","last_change":"2024-03-30T01:43:11.806420+0000","last_active":"2024-03-30T01:43:12.282134+0000","last_peered":"2024-03-30T01:43:12.282134+0000","last_clean":"2024-03-30T01:43:12.282134+0000","last_became_active":"2024-03-30T01:43:11.806144+0000","last_became_peered":"2024-03-30T01:43:11.806144+0000","last_unstale":"2024-03-30T01:43:12.282134+0000","last_undegraded":"2024-03-30T01:43:12.282134+0000","last_fullsized":"2024-03-30T01:43:12.282134+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:19.302319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.268410+0000","last_change":"2024-03-30T01:43:12.259615+0000","last_active":"2024-03-30T01:43:12.268410+0000","last_peered":"2024-03-30T01:43:12.268410+0000","last_clean":"2024-03-30T01:43:12.268410+0000","last_became_active":"2024-03-30T01:43:12.259331+0000","last_became_peered":"2024-03-30T01:43:12.259331+0000","last_unstale":"2024-03-30T01:43:12.268410+0000","last_undegraded":"2024-03-30T01:43:12.268410+0000","last_fullsized":"2024-03-30T01:43:12.268410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:10.944027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281404+0000","last_change":"2024-03-30T01:43:11.811391+0000","last_active":"2024-03-30T01:43:12.281404+0000","last_peered":"2024-03-30T01:43:12.281404+0000","last_clean":"2024-03-30T01:43:12.281404+0000","last_became_active":"2024-03-30T01:43:11.810873+0000","last_became_peered":"2024-03-30T01:43:11.810873+0000","last_unstale":"2024-03-30T01:43:12.281404+0000","last_undegraded":"2024-03-30T01:43:12.281404+0000","last_fullsized":"2024-03-30T01:43:12.281404+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:22.774118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282370+0000","last_change":"2024-03-30T01:43:12.276578+0000","last_active":"2024-03-30T01:43:12.282370+0000","last_peered":"2024-03-30T01:43:12.282370+0000","last_clean":"2024-03-30T01:43:12.282370+0000","last_became_active":"2024-03-30T01:43:12.276293+0000","last_became_peered":"2024-03-30T01:43:12.276293+0000","last_unstale":"2024-03-30T01:43:12.282370+0000","last_undegraded":"2024-03-30T01:43:12.282370+0000","last_fullsized":"2024-03-30T01:43:12.282370+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270389+0000","last_change":"2024-03-30T01:43:11.803525+0000","last_active":"2024-03-30T01:43:12.270389+0000","last_peered":"2024-03-30T01:43:12.270389+0000","last_clean":"2024-03-30T01:43:12.270389+0000","last_became_active":"2024-03-30T01:43:11.803115+0000","last_became_peered":"2024-03-30T01:43:11.803115+0000","last_unstale":"2024-03-30T01:43:12.270389+0000","last_undegraded":"2024-03-30T01:43:12.270389+0000","last_fullsized":"2024-03-30T01:43:12.270389+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:10.022632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271495+0000","last_change":"2024-03-30T01:43:12.267451+0000","last_active":"2024-03-30T01:43:12.271495+0000","last_peered":"2024-03-30T01:43:12.271495+0000","last_clean":"2024-03-30T01:43:12.271495+0000","last_became_active":"2024-03-30T01:43:12.267249+0000","last_became_peered":"2024-03-30T01:43:12.267249+0000","last_unstale":"2024-03-30T01:43:12.271495+0000","last_undegraded":"2024-03-30T01:43:12.271495+0000","last_fullsized":"2024-03-30T01:43:12.271495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:49.333290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271580+0000","last_change":"2024-03-30T01:43:11.804981+0000","last_active":"2024-03-30T01:43:12.271580+0000","last_peered":"2024-03-30T01:43:12.271580+0000","last_clean":"2024-03-30T01:43:12.271580+0000","last_became_active":"2024-03-30T01:43:11.804690+0000","last_became_peered":"2024-03-30T01:43:11.804690+0000","last_unstale":"2024-03-30T01:43:12.271580+0000","last_undegraded":"2024-03-30T01:43:12.271580+0000","last_fullsized":"2024-03-30T01:43:12.271580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:25:59.487668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270642+0000","last_change":"2024-03-30T01:43:12.268426+0000","last_active":"2024-03-30T01:43:12.270642+0000","last_peered":"2024-03-30T01:43:12.270642+0000","last_clean":"2024-03-30T01:43:12.270642+0000","last_became_active":"2024-03-30T01:43:12.268106+0000","last_became_peered":"2024-03-30T01:43:12.268106+0000","last_unstale":"2024-03-30T01:43:12.270642+0000","last_undegraded":"2024-03-30T01:43:12.270642+0000","last_fullsized":"2024-03-30T01:43:12.270642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:32:07.610941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271583+0000","last_change":"2024-03-30T01:43:11.798840+0000","last_active":"2024-03-30T01:43:12.271583+0000","last_peered":"2024-03-30T01:43:12.271583+0000","last_clean":"2024-03-30T01:43:12.271583+0000","last_became_active":"2024-03-30T01:43:11.798493+0000","last_became_peered":"2024-03-30T01:43:11.798493+0000","last_unstale":"2024-03-30T01:43:12.271583+0000","last_undegraded":"2024-03-30T01:43:12.271583+0000","last_fullsized":"2024-03-30T01:43:12.271583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:35.032315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.276646+0000","last_change":"2024-03-30T01:43:12.272426+0000","last_active":"2024-03-30T01:43:12.276646+0000","last_peered":"2024-03-30T01:43:12.276646+0000","last_clean":"2024-03-30T01:43:12.276646+0000","last_became_active":"2024-03-30T01:43:12.272160+0000","last_became_peered":"2024-03-30T01:43:12.272160+0000","last_unstale":"2024-03-30T01:43:12.276646+0000","last_undegraded":"2024-03-30T01:43:12.276646+0000","last_fullsized":"2024-03-30T01:43:12.276646+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:37.825651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.249041+0000","last_change":"2024-03-30T01:43:11.805468+0000","last_active":"2024-03-30T01:43:12.249041+0000","last_peered":"2024-03-30T01:43:12.249041+0000","last_clean":"2024-03-30T01:43:12.249041+0000","last_became_active":"2024-03-30T01:43:11.804984+0000","last_became_peered":"2024-03-30T01:43:11.804984+0000","last_unstale":"2024-03-30T01:43:12.249041+0000","last_undegraded":"2024-03-30T01:43:12.249041+0000","last_fullsized":"2024-03-30T01:43:12.249041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:16.171589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271678+0000","last_change":"2024-03-30T01:43:12.265005+0000","last_active":"2024-03-30T01:43:12.271678+0000","last_peered":"2024-03-30T01:43:12.271678+0000","last_clean":"2024-03-30T01:43:12.271678+0000","last_became_active":"2024-03-30T01:43:12.264726+0000","last_became_peered":"2024-03-30T01:43:12.264726+0000","last_unstale":"2024-03-30T01:43:12.271678+0000","last_undegraded":"2024-03-30T01:43:12.271678+0000","last_fullsized":"2024-03-30T01:43:12.271678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:41:19.989099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.256348+0000","last_change":"2024-03-30T01:43:11.799020+0000","last_active":"2024-03-30T01:43:12.256348+0000","last_peered":"2024-03-30T01:43:12.256348+0000","last_clean":"2024-03-30T01:43:12.256348+0000","last_became_active":"2024-03-30T01:43:11.797840+0000","last_became_peered":"2024-03-30T01:43:11.797840+0000","last_unstale":"2024-03-30T01:43:12.256348+0000","last_undegraded":"2024-03-30T01:43:12.256348+0000","last_fullsized":"2024-03-30T01:43:12.256348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:48:17.963383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.269985+0000","last_change":"2024-03-30T01:43:12.265237+0000","last_active":"2024-03-30T01:43:12.269985+0000","last_peered":"2024-03-30T01:43:12.269985+0000","last_clean":"2024-03-30T01:43:12.269985+0000","last_became_active":"2024-03-30T01:43:12.264995+0000","last_became_peered":"2024-03-30T01:43:12.264995+0000","last_unstale":"2024-03-30T01:43:12.269985+0000","last_undegraded":"2024-03-30T01:43:12.269985+0000","last_fullsized":"2024-03-30T01:43:12.269985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.256029+0000","last_change":"2024-03-30T01:43:11.803287+0000","last_active":"2024-03-30T01:43:12.256029+0000","last_peered":"2024-03-30T01:43:12.256029+0000","last_clean":"2024-03-30T01:43:12.256029+0000","last_became_active":"2024-03-30T01:43:11.803067+0000","last_became_peered":"2024-03-30T01:43:11.803067+0000","last_unstale":"2024-03-30T01:43:12.256029+0000","last_undegraded":"2024-03-30T01:43:12.256029+0000","last_fullsized":"2024-03-30T01:43:12.256029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:36.821233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282452+0000","last_change":"2024-03-30T01:43:12.275223+0000","last_active":"2024-03-30T01:43:12.282452+0000","last_peered":"2024-03-30T01:43:12.282452+0000","last_clean":"2024-03-30T01:43:12.282452+0000","last_became_active":"2024-03-30T01:43:12.274949+0000","last_became_peered":"2024-03-30T01:43:12.274949+0000","last_unstale":"2024-03-30T01:43:12.282452+0000","last_undegraded":"2024-03-30T01:43:12.282452+0000","last_fullsized":"2024-03-30T01:43:12.282452+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:34.024584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.266940+0000","last_change":"2024-03-30T01:43:11.802972+0000","last_active":"2024-03-30T01:43:12.266940+0000","last_peered":"2024-03-30T01:43:12.266940+0000","last_clean":"2024-03-30T01:43:12.266940+0000","last_became_active":"2024-03-30T01:43:11.802665+0000","last_became_peered":"2024-03-30T01:43:11.802665+0000","last_unstale":"2024-03-30T01:43:12.266940+0000","last_undegraded":"2024-03-30T01:43:12.266940+0000","last_fullsized":"2024-03-30T01:43:12.266940+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:24.494514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.274408+0000","last_change":"2024-03-30T01:43:12.271580+0000","last_active":"2024-03-30T01:43:12.274408+0000","last_peered":"2024-03-30T01:43:12.274408+0000","last_clean":"2024-03-30T01:43:12.274408+0000","last_became_active":"2024-03-30T01:43:12.271342+0000","last_became_peered":"2024-03-30T01:43:12.271342+0000","last_unstale":"2024-03-30T01:43:12.274408+0000","last_undegraded":"2024-03-30T01:43:12.274408+0000","last_fullsized":"2024-03-30T01:43:12.274408+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:25.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.278679+0000","last_change":"2024-03-30T01:43:11.802252+0000","last_active":"2024-03-30T01:43:12.278679+0000","last_peered":"2024-03-30T01:43:12.278679+0000","last_clean":"2024-03-30T01:43:12.278679+0000","last_became_active":"2024-03-30T01:43:11.801845+0000","last_became_peered":"2024-03-30T01:43:11.801845+0000","last_unstale":"2024-03-30T01:43:12.278679+0000","last_undegraded":"2024-03-30T01:43:12.278679+0000","last_fullsized":"2024-03-30T01:43:12.278679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279509+0000","last_change":"2024-03-30T01:43:12.268100+0000","last_active":"2024-03-30T01:43:12.279509+0000","last_peered":"2024-03-30T01:43:12.279509+0000","last_clean":"2024-03-30T01:43:12.279509+0000","last_became_active":"2024-03-30T01:43:12.267755+0000","last_became_peered":"2024-03-30T01:43:12.267755+0000","last_unstale":"2024-03-30T01:43:12.279509+0000","last_undegraded":"2024-03-30T01:43:12.279509+0000","last_fullsized":"2024-03-30T01:43:12.279509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282865+0000","last_change":"2024-03-30T01:43:11.806909+0000","last_active":"2024-03-30T01:43:12.282865+0000","last_peered":"2024-03-30T01:43:12.282865+0000","last_clean":"2024-03-30T01:43:12.282865+0000","last_became_active":"2024-03-30T01:43:11.806609+0000","last_became_peered":"2024-03-30T01:43:11.806609+0000","last_unstale":"2024-03-30T01:43:12.282865+0000","last_undegraded":"2024-03-30T01:43:12.282865+0000","last_fullsized":"2024-03-30T01:43:12.282865+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282323+0000","last_change":"2024-03-30T01:43:12.280292+0000","last_active":"2024-03-30T01:43:12.282323+0000","last_peered":"2024-03-30T01:43:12.282323+0000","last_clean":"2024-03-30T01:43:12.282323+0000","last_became_active":"2024-03-30T01:43:12.279966+0000","last_became_peered":"2024-03-30T01:43:12.279966+0000","last_unstale":"2024-03-30T01:43:12.282323+0000","last_undegraded":"2024-03-30T01:43:12.282323+0000","last_fullsized":"2024-03-30T01:43:12.282323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.272246+0000","last_change":"2024-03-30T01:43:11.804635+0000","last_active":"2024-03-30T01:43:12.272246+0000","last_peered":"2024-03-30T01:43:12.272246+0000","last_clean":"2024-03-30T01:43:12.272246+0000","last_became_active":"2024-03-30T01:43:11.804355+0000","last_became_peered":"2024-03-30T01:43:11.804355+0000","last_unstale":"2024-03-30T01:43:12.272246+0000","last_undegraded":"2024-03-30T01:43:12.272246+0000","last_fullsized":"2024-03-30T01:43:12.272246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:57.117126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.272575+0000","last_change":"2024-03-30T01:43:12.268978+0000","last_active":"2024-03-30T01:43:12.272575+0000","last_peered":"2024-03-30T01:43:12.272575+0000","last_clean":"2024-03-30T01:43:12.272575+0000","last_became_active":"2024-03-30T01:43:12.268633+0000","last_became_peered":"2024-03-30T01:43:12.268633+0000","last_unstale":"2024-03-30T01:43:12.272575+0000","last_undegraded":"2024-03-30T01:43:12.272575+0000","last_fullsized":"2024-03-30T01:43:12.272575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.255367+0000","last_change":"2024-03-30T01:43:11.811100+0000","last_active":"2024-03-30T01:43:12.255367+0000","last_peered":"2024-03-30T01:43:12.255367+0000","last_clean":"2024-03-30T01:43:12.255367+0000","last_became_active":"2024-03-30T01:43:11.810811+0000","last_became_peered":"2024-03-30T01:43:11.810811+0000","last_unstale":"2024-03-30T01:43:12.255367+0000","last_undegraded":"2024-03-30T01:43:12.255367+0000","last_fullsized":"2024-03-30T01:43:12.255367+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.267805+0000","last_change":"2024-03-30T01:43:12.265729+0000","last_active":"2024-03-30T01:43:12.267805+0000","last_peered":"2024-03-30T01:43:12.267805+0000","last_clean":"2024-03-30T01:43:12.267805+0000","last_became_active":"2024-03-30T01:43:12.265427+0000","last_became_peered":"2024-03-30T01:43:12.265427+0000","last_unstale":"2024-03-30T01:43:12.267805+0000","last_undegraded":"2024-03-30T01:43:12.267805+0000","last_fullsized":"2024-03-30T01:43:12.267805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:40:39.587585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.257366+0000","last_change":"2024-03-30T01:43:11.802626+0000","last_active":"2024-03-30T01:43:12.257366+0000","last_peered":"2024-03-30T01:43:12.257366+0000","last_clean":"2024-03-30T01:43:12.257366+0000","last_became_active":"2024-03-30T01:43:11.802314+0000","last_became_peered":"2024-03-30T01:43:11.802314+0000","last_unstale":"2024-03-30T01:43:12.257366+0000","last_undegraded":"2024-03-30T01:43:12.257366+0000","last_fullsized":"2024-03-30T01:43:12.257366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279701+0000","last_change":"2024-03-30T01:43:12.274115+0000","last_active":"2024-03-30T01:43:12.279701+0000","last_peered":"2024-03-30T01:43:12.279701+0000","last_clean":"2024-03-30T01:43:12.279701+0000","last_became_active":"2024-03-30T01:43:12.273842+0000","last_became_peered":"2024-03-30T01:43:12.273842+0000","last_unstale":"2024-03-30T01:43:12.279701+0000","last_undegraded":"2024-03-30T01:43:12.279701+0000","last_fullsized":"2024-03-30T01:43:12.279701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:44:49.086267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.283586+0000","last_change":"2024-03-30T01:43:11.802309+0000","last_active":"2024-03-30T01:43:12.283586+0000","last_peered":"2024-03-30T01:43:12.283586+0000","last_clean":"2024-03-30T01:43:12.283586+0000","last_became_active":"2024-03-30T01:43:11.802008+0000","last_became_peered":"2024-03-30T01:43:11.802008+0000","last_unstale":"2024-03-30T01:43:12.283586+0000","last_undegraded":"2024-03-30T01:43:12.283586+0000","last_fullsized":"2024-03-30T01:43:12.283586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:11.268414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282670+0000","last_change":"2024-03-30T01:43:12.279554+0000","last_active":"2024-03-30T01:43:12.282670+0000","last_peered":"2024-03-30T01:43:12.282670+0000","last_clean":"2024-03-30T01:43:12.282670+0000","last_became_active":"2024-03-30T01:43:12.279266+0000","last_became_peered":"2024-03-30T01:43:12.279266+0000","last_unstale":"2024-03-30T01:43:12.282670+0000","last_undegraded":"2024-03-30T01:43:12.282670+0000","last_fullsized":"2024-03-30T01:43:12.282670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.273448+0000","last_change":"2024-03-30T01:43:11.809877+0000","last_active":"2024-03-30T01:43:12.273448+0000","last_peered":"2024-03-30T01:43:12.273448+0000","last_clean":"2024-03-30T01:43:12.273448+0000","last_became_active":"2024-03-30T01:43:11.808130+0000","last_became_peered":"2024-03-30T01:43:11.808130+0000","last_unstale":"2024-03-30T01:43:12.273448+0000","last_undegraded":"2024-03-30T01:43:12.273448+0000","last_fullsized":"2024-03-30T01:43:12.273448+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.281930+0000","last_change":"2024-03-30T01:43:12.275996+0000","last_active":"2024-03-30T01:43:12.281930+0000","last_peered":"2024-03-30T01:43:12.281930+0000","last_clean":"2024-03-30T01:43:12.281930+0000","last_became_active":"2024-03-30T01:43:12.275705+0000","last_became_peered":"2024-03-30T01:43:12.275705+0000","last_unstale":"2024-03-30T01:43:12.281930+0000","last_undegraded":"2024-03-30T01:43:12.281930+0000","last_fullsized":"2024-03-30T01:43:12.281930+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:47.117365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279812+0000","last_change":"2024-03-30T01:43:11.806234+0000","last_active":"2024-03-30T01:43:12.279812+0000","last_peered":"2024-03-30T01:43:12.279812+0000","last_clean":"2024-03-30T01:43:12.279812+0000","last_became_active":"2024-03-30T01:43:11.806012+0000","last_became_peered":"2024-03-30T01:43:11.806012+0000","last_unstale":"2024-03-30T01:43:12.279812+0000","last_undegraded":"2024-03-30T01:43:12.279812+0000","last_fullsized":"2024-03-30T01:43:12.279812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:24.494514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.280046+0000","last_change":"2024-03-30T01:43:12.271940+0000","last_active":"2024-03-30T01:43:12.280046+0000","last_peered":"2024-03-30T01:43:12.280046+0000","last_clean":"2024-03-30T01:43:12.280046+0000","last_became_active":"2024-03-30T01:43:12.268234+0000","last_became_peered":"2024-03-30T01:43:12.268234+0000","last_unstale":"2024-03-30T01:43:12.280046+0000","last_undegraded":"2024-03-30T01:43:12.280046+0000","last_fullsized":"2024-03-30T01:43:12.280046+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.278250+0000","last_change":"2024-03-30T01:43:11.803111+0000","last_active":"2024-03-30T01:43:12.278250+0000","last_peered":"2024-03-30T01:43:12.278250+0000","last_clean":"2024-03-30T01:43:12.278250+0000","last_became_active":"2024-03-30T01:43:11.802781+0000","last_became_peered":"2024-03-30T01:43:11.802781+0000","last_unstale":"2024-03-30T01:43:12.278250+0000","last_undegraded":"2024-03-30T01:43:12.278250+0000","last_fullsized":"2024-03-30T01:43:12.278250+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:30.914337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.275437+0000","last_change":"2024-03-30T01:43:12.272623+0000","last_active":"2024-03-30T01:43:12.275437+0000","last_peered":"2024-03-30T01:43:12.275437+0000","last_clean":"2024-03-30T01:43:12.275437+0000","last_became_active":"2024-03-30T01:43:12.272330+0000","last_became_peered":"2024-03-30T01:43:12.272330+0000","last_unstale":"2024-03-30T01:43:12.275437+0000","last_undegraded":"2024-03-30T01:43:12.275437+0000","last_fullsized":"2024-03-30T01:43:12.275437+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:11.229993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.249731+0000","last_change":"2024-03-30T01:43:11.802228+0000","last_active":"2024-03-30T01:43:12.249731+0000","last_peered":"2024-03-30T01:43:12.249731+0000","last_clean":"2024-03-30T01:43:12.249731+0000","last_became_active":"2024-03-30T01:43:11.801742+0000","last_became_peered":"2024-03-30T01:43:11.801742+0000","last_unstale":"2024-03-30T01:43:12.249731+0000","last_undegraded":"2024-03-30T01:43:12.249731+0000","last_fullsized":"2024-03-30T01:43:12.249731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:01.225672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270316+0000","last_change":"2024-03-30T01:43:12.259920+0000","last_active":"2024-03-30T01:43:12.270316+0000","last_peered":"2024-03-30T01:43:12.270316+0000","last_clean":"2024-03-30T01:43:12.270316+0000","last_became_active":"2024-03-30T01:43:12.259636+0000","last_became_peered":"2024-03-30T01:43:12.259636+0000","last_unstale":"2024-03-30T01:43:12.270316+0000","last_undegraded":"2024-03-30T01:43:12.270316+0000","last_fullsized":"2024-03-30T01:43:12.270316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:45:42.926912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.270208+0000","last_change":"2024-03-30T01:43:11.804854+0000","last_active":"2024-03-30T01:43:12.270208+0000","last_peered":"2024-03-30T01:43:12.270208+0000","last_clean":"2024-03-30T01:43:12.270208+0000","last_became_active":"2024-03-30T01:43:11.804593+0000","last_became_peered":"2024-03-30T01:43:11.804593+0000","last_unstale":"2024-03-30T01:43:12.270208+0000","last_undegraded":"2024-03-30T01:43:12.270208+0000","last_fullsized":"2024-03-30T01:43:12.270208+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:45:22.282372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.272614+0000","last_change":"2024-03-30T01:43:12.269648+0000","last_active":"2024-03-30T01:43:12.272614+0000","last_peered":"2024-03-30T01:43:12.272614+0000","last_clean":"2024-03-30T01:43:12.272614+0000","last_became_active":"2024-03-30T01:43:12.269295+0000","last_became_peered":"2024-03-30T01:43:12.269295+0000","last_unstale":"2024-03-30T01:43:12.272614+0000","last_undegraded":"2024-03-30T01:43:12.272614+0000","last_fullsized":"2024-03-30T01:43:12.272614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.255416+0000","last_change":"2024-03-30T01:43:11.811090+0000","last_active":"2024-03-30T01:43:12.255416+0000","last_peered":"2024-03-30T01:43:12.255416+0000","last_clean":"2024-03-30T01:43:12.255416+0000","last_became_active":"2024-03-30T01:43:11.810767+0000","last_became_peered":"2024-03-30T01:43:11.810767+0000","last_unstale":"2024-03-30T01:43:12.255416+0000","last_undegraded":"2024-03-30T01:43:12.255416+0000","last_fullsized":"2024-03-30T01:43:12.255416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.267443+0000","last_change":"2024-03-30T01:43:12.265208+0000","last_active":"2024-03-30T01:43:12.267443+0000","last_peered":"2024-03-30T01:43:12.267443+0000","last_clean":"2024-03-30T01:43:12.267443+0000","last_became_active":"2024-03-30T01:43:12.264919+0000","last_became_peered":"2024-03-30T01:43:12.264919+0000","last_unstale":"2024-03-30T01:43:12.267443+0000","last_undegraded":"2024-03-30T01:43:12.267443+0000","last_fullsized":"2024-03-30T01:43:12.267443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:31.918755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.254828+0000","last_change":"2024-03-30T01:43:11.803507+0000","last_active":"2024-03-30T01:43:12.254828+0000","last_peered":"2024-03-30T01:43:12.254828+0000","last_clean":"2024-03-30T01:43:12.254828+0000","last_became_active":"2024-03-30T01:43:11.803197+0000","last_became_peered":"2024-03-30T01:43:11.803197+0000","last_unstale":"2024-03-30T01:43:12.254828+0000","last_undegraded":"2024-03-30T01:43:12.254828+0000","last_fullsized":"2024-03-30T01:43:12.254828+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:41:18.984681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282297+0000","last_change":"2024-03-30T01:43:12.275594+0000","last_active":"2024-03-30T01:43:12.282297+0000","last_peered":"2024-03-30T01:43:12.282297+0000","last_clean":"2024-03-30T01:43:12.282297+0000","last_became_active":"2024-03-30T01:43:12.275249+0000","last_became_peered":"2024-03-30T01:43:12.275249+0000","last_unstale":"2024-03-30T01:43:12.282297+0000","last_undegraded":"2024-03-30T01:43:12.282297+0000","last_fullsized":"2024-03-30T01:43:12.282297+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:09.869640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.275230+0000","last_change":"2024-03-30T01:43:11.810058+0000","last_active":"2024-03-30T01:43:12.275230+0000","last_peered":"2024-03-30T01:43:12.275230+0000","last_clean":"2024-03-30T01:43:12.275230+0000","last_became_active":"2024-03-30T01:43:11.808406+0000","last_became_peered":"2024-03-30T01:43:11.808406+0000","last_unstale":"2024-03-30T01:43:12.275230+0000","last_undegraded":"2024-03-30T01:43:12.275230+0000","last_fullsized":"2024-03-30T01:43:12.275230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:57.117126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.261110+0000","last_change":"2024-03-30T01:43:12.259473+0000","last_active":"2024-03-30T01:43:12.261110+0000","last_peered":"2024-03-30T01:43:12.261110+0000","last_clean":"2024-03-30T01:43:12.261110+0000","last_became_active":"2024-03-30T01:43:12.259194+0000","last_became_peered":"2024-03-30T01:43:12.259194+0000","last_unstale":"2024-03-30T01:43:12.261110+0000","last_undegraded":"2024-03-30T01:43:12.261110+0000","last_fullsized":"2024-03-30T01:43:12.261110+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:14.284517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.260397+0000","last_change":"2024-03-30T01:43:11.810112+0000","last_active":"2024-03-30T01:43:12.260397+0000","last_peered":"2024-03-30T01:43:12.260397+0000","last_clean":"2024-03-30T01:43:12.260397+0000","last_became_active":"2024-03-30T01:43:11.808349+0000","last_became_peered":"2024-03-30T01:43:11.808349+0000","last_unstale":"2024-03-30T01:43:12.260397+0000","last_undegraded":"2024-03-30T01:43:12.260397+0000","last_fullsized":"2024-03-30T01:43:12.260397+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:30.914337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.271706+0000","last_change":"2024-03-30T01:43:12.265689+0000","last_active":"2024-03-30T01:43:12.271706+0000","last_peered":"2024-03-30T01:43:12.271706+0000","last_clean":"2024-03-30T01:43:12.271706+0000","last_became_active":"2024-03-30T01:43:12.265382+0000","last_became_peered":"2024-03-30T01:43:12.265382+0000","last_unstale":"2024-03-30T01:43:12.271706+0000","last_undegraded":"2024-03-30T01:43:12.271706+0000","last_fullsized":"2024-03-30T01:43:12.271706+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:17.176008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.256736+0000","last_change":"2024-03-30T01:43:11.811154+0000","last_active":"2024-03-30T01:43:12.256736+0000","last_peered":"2024-03-30T01:43:12.256736+0000","last_clean":"2024-03-30T01:43:12.256736+0000","last_became_active":"2024-03-30T01:43:11.810455+0000","last_became_peered":"2024-03-30T01:43:11.810455+0000","last_unstale":"2024-03-30T01:43:12.256736+0000","last_undegraded":"2024-03-30T01:43:12.256736+0000","last_fullsized":"2024-03-30T01:43:12.256736+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:33.020166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.267892+0000","last_change":"2024-03-30T01:43:12.265758+0000","last_active":"2024-03-30T01:43:12.267892+0000","last_peered":"2024-03-30T01:43:12.267892+0000","last_clean":"2024-03-30T01:43:12.267892+0000","last_became_active":"2024-03-30T01:43:12.265461+0000","last_became_peered":"2024-03-30T01:43:12.265461+0000","last_unstale":"2024-03-30T01:43:12.267892+0000","last_undegraded":"2024-03-30T01:43:12.267892+0000","last_fullsized":"2024-03-30T01:43:12.267892+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:36.552215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.253707+0000","last_change":"2024-03-30T01:43:11.806860+0000","last_active":"2024-03-30T01:43:12.253707+0000","last_peered":"2024-03-30T01:43:12.253707+0000","last_clean":"2024-03-30T01:43:12.253707+0000","last_became_active":"2024-03-30T01:43:11.806563+0000","last_became_peered":"2024-03-30T01:43:11.806563+0000","last_unstale":"2024-03-30T01:43:12.253707+0000","last_undegraded":"2024-03-30T01:43:12.253707+0000","last_fullsized":"2024-03-30T01:43:12.253707+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.274501+0000","last_change":"2024-03-30T01:43:12.269888+0000","last_active":"2024-03-30T01:43:12.274501+0000","last_peered":"2024-03-30T01:43:12.274501+0000","last_clean":"2024-03-30T01:43:12.274501+0000","last_became_active":"2024-03-30T01:43:12.269582+0000","last_became_peered":"2024-03-30T01:43:12.269582+0000","last_unstale":"2024-03-30T01:43:12.274501+0000","last_undegraded":"2024-03-30T01:43:12.274501+0000","last_fullsized":"2024-03-30T01:43:12.274501+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:58.121544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.258139+0000","last_change":"2024-03-30T01:43:11.802541+0000","last_active":"2024-03-30T01:43:12.258139+0000","last_peered":"2024-03-30T01:43:12.258139+0000","last_clean":"2024-03-30T01:43:12.258139+0000","last_became_active":"2024-03-30T01:43:11.802154+0000","last_became_peered":"2024-03-30T01:43:11.802154+0000","last_unstale":"2024-03-30T01:43:12.258139+0000","last_undegraded":"2024-03-30T01:43:12.258139+0000","last_fullsized":"2024-03-30T01:43:12.258139+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.282160+0000","last_change":"2024-03-30T01:43:12.279403+0000","last_active":"2024-03-30T01:43:12.282160+0000","last_peered":"2024-03-30T01:43:12.282160+0000","last_clean":"2024-03-30T01:43:12.282160+0000","last_became_active":"2024-03-30T01:43:12.279071+0000","last_became_peered":"2024-03-30T01:43:12.279071+0000","last_unstale":"2024-03-30T01:43:12.282160+0000","last_undegraded":"2024-03-30T01:43:12.282160+0000","last_fullsized":"2024-03-30T01:43:12.282160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.249559+0000","last_change":"2024-03-30T01:43:11.799428+0000","last_active":"2024-03-30T01:43:12.249559+0000","last_peered":"2024-03-30T01:43:12.249559+0000","last_clean":"2024-03-30T01:43:12.249559+0000","last_became_active":"2024-03-30T01:43:11.799147+0000","last_became_peered":"2024-03-30T01:43:11.799147+0000","last_unstale":"2024-03-30T01:43:12.249559+0000","last_undegraded":"2024-03-30T01:43:12.249559+0000","last_fullsized":"2024-03-30T01:43:12.249559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:56.396431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279656+0000","last_change":"2024-03-30T01:43:12.273079+0000","last_active":"2024-03-30T01:43:12.279656+0000","last_peered":"2024-03-30T01:43:12.279656+0000","last_clean":"2024-03-30T01:43:12.279656+0000","last_became_active":"2024-03-30T01:43:12.272766+0000","last_became_peered":"2024-03-30T01:43:12.272766+0000","last_unstale":"2024-03-30T01:43:12.279656+0000","last_undegraded":"2024-03-30T01:43:12.279656+0000","last_fullsized":"2024-03-30T01:43:12.279656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:57.882115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.249145+0000","last_change":"2024-03-30T01:43:11.805262+0000","last_active":"2024-03-30T01:43:12.249145+0000","last_peered":"2024-03-30T01:43:12.249145+0000","last_clean":"2024-03-30T01:43:12.249145+0000","last_became_active":"2024-03-30T01:43:11.804622+0000","last_became_peered":"2024-03-30T01:43:11.804622+0000","last_unstale":"2024-03-30T01:43:12.249145+0000","last_undegraded":"2024-03-30T01:43:12.249145+0000","last_fullsized":"2024-03-30T01:43:12.249145+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:08.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.279590+0000","last_change":"2024-03-30T01:43:12.275816+0000","last_active":"2024-03-30T01:43:12.279590+0000","last_peered":"2024-03-30T01:43:12.279590+0000","last_clean":"2024-03-30T01:43:12.279590+0000","last_became_active":"2024-03-30T01:43:12.275547+0000","last_became_peered":"2024-03-30T01:43:12.275547+0000","last_unstale":"2024-03-30T01:43:12.279590+0000","last_undegraded":"2024-03-30T01:43:12.279590+0000","last_fullsized":"2024-03-30T01:43:12.279590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:31.125780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.253921+0000","last_change":"2024-03-30T01:43:11.804870+0000","last_active":"2024-03-30T01:43:12.253921+0000","last_peered":"2024-03-30T01:43:12.253921+0000","last_clean":"2024-03-30T01:43:12.253921+0000","last_became_active":"2024-03-30T01:43:11.804580+0000","last_became_peered":"2024-03-30T01:43:11.804580+0000","last_unstale":"2024-03-30T01:43:12.253921+0000","last_undegraded":"2024-03-30T01:43:12.253921+0000","last_fullsized":"2024-03-30T01:43:12.253921+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.274510+0000","last_change":"2024-03-30T01:43:12.270228+0000","last_active":"2024-03-30T01:43:12.274510+0000","last_peered":"2024-03-30T01:43:12.274510+0000","last_clean":"2024-03-30T01:43:12.274510+0000","last_became_active":"2024-03-30T01:43:12.269910+0000","last_became_peered":"2024-03-30T01:43:12.269910+0000","last_unstale":"2024-03-30T01:43:12.274510+0000","last_undegraded":"2024-03-30T01:43:12.274510+0000","last_fullsized":"2024-03-30T01:43:12.274510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:00:52.920381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.248951+0000","last_change":"2024-03-30T01:43:11.803125+0000","last_active":"2024-03-30T01:43:12.248951+0000","last_peered":"2024-03-30T01:43:12.248951+0000","last_clean":"2024-03-30T01:43:12.248951+0000","last_became_active":"2024-03-30T01:43:11.802822+0000","last_became_peered":"2024-03-30T01:43:11.802822+0000","last_unstale":"2024-03-30T01:43:12.248951+0000","last_undegraded":"2024-03-30T01:43:12.248951+0000","last_fullsized":"2024-03-30T01:43:12.248951+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:21.501027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-03-30T01:43:11.793409+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:43:11.793409+0000","last_peered":"2024-03-30T01:43:11.793409+0000","last_clean":"2024-03-30T01:43:11.793409+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:43:11.793409+0000","last_undegraded":"2024-03-30T01:43:11.793409+0000","last_fullsized":"2024-03-30T01:43:11.793409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1294336,"data_stored":909691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.76400000000000001}]},{"osd":1,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72699999999999998,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.81200000000000006,"15min":0.85599999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72699999999999998,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.68899999999999995}]},{"osd":2,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.76200000000000001,"15min":0.79500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.73199999999999998}]},{"osd":3,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73199999999999998,"5min":1.264,"15min":1.264},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.498}]},{"osd":4,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79300000000000004,"5min":1.075,"15min":1.075},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77200000000000002,"5min":1.833,"15min":1.833},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76300000000000001,"5min":1.238,"15min":1.238},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90600000000000003,"5min":1.2,"15min":1.2},"last":0.40600000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85099999999999998,"5min":1.288,"15min":1.288},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.68300000000000005,"15min":0.69799999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82499999999999996,"5min":1.901,"15min":1.901},"last":0.86399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":716800,"data_stored":326452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.94899999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":1.361,"15min":1.361},"last":0.67000000000000004}]},{"osd":1,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96799999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.71699999999999997,"15min":0.74399999999999999},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.54600000000000004}]},{"osd":2,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85599999999999998,"5min":1.27,"15min":1.27},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.409,"15min":1.409},"last":0.70299999999999996}]},{"osd":3,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.379,"5min":0.371,"15min":0.371},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.626}]},{"osd":4,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.72399999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":319323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.432,"15min":0.442},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.503,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":1.111,"15min":1.2270000000000001},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":15.038,"15min":15.038},"last":0.42799999999999999}]},{"osd":2,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.91400000000000003,"15min":1.006},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55500000000000005,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68400000000000005,"5min":1.444,"15min":1.444},"last":0.39400000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":1.474,"15min":1.474},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.624,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64000000000000001,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.35999999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.111,"15min":1.1970000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94499999999999995,"5min":11.99,"15min":11.99},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89900000000000002,"5min":1.407,"15min":1.407},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":1.335,"15min":1.4490000000000001},"min":{"1min":0.48799999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97699999999999998,"5min":15.006,"15min":15.006},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":1.2130000000000001,"15min":1.3069999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83199999999999996,"5min":11.926,"15min":11.926},"last":0.57199999999999995}]},{"osd":6,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.3149999999999999,"15min":1.429},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93100000000000005,"5min":14.974,"15min":14.974},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.1950000000000001,"15min":1.2949999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78100000000000003,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.504,"15min":1.504},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":1.343,"15min":1.4570000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.47499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1290240,"data_stored":908722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.57999999999999996,"15min":0.61499999999999999},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.439,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.54500000000000004,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.95499999999999996,"15min":1.05},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.443,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.86299999999999999,"15min":0.94299999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.503,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.59799999999999998,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.53300000000000003,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.443}]},{"osd":4,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.84699999999999998,"15min":0.875},"min":{"1min":0.47099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":5.431,"15min":5.431},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88300000000000001,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.80100000000000005,"15min":0.82899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77700000000000002,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.83799999999999997,"15min":0.87},"min":{"1min":0.47099999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.53200000000000003}]},{"osd":6,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76500000000000001,"5min":1.177,"15min":1.177},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.82399999999999995,"15min":0.85399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.81899999999999995,"15min":0.84799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.88800000000000001,"15min":0.92700000000000005},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81899999999999995,"5min":5.468,"15min":5.468},"last":0.47699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":319323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.89000000000000001,"15min":0.96199999999999997},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.1890000000000001,"5min":12.478999999999999,"15min":12.478999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.753,"15min":0.79600000000000004},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1469999999999998,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.27800000000000002}]},{"osd":1,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53900000000000003,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.93899999999999995,"15min":1.0089999999999999},"min":{"1min":0.28999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.1309999999999998,"5min":12.425000000000001,"15min":12.425000000000001},"last":0.314}]},{"osd":3,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1459999999999999,"5min":1.55,"15min":1.55},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.98299999999999998,"15min":1.0620000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.8759999999999999,"5min":13.217000000000001,"15min":13.217000000000001},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.93899999999999995,"15min":0.97999999999999998},"min":{"1min":0.46400000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.956,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.95499999999999996,"15min":0.997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.0630000000000002,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.45100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89300000000000002,"5min":1.462,"15min":1.462},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.97499999999999998,"15min":1.0229999999999999},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.9159999999999999,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.97899999999999998,"15min":1.0229999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1139999999999999,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.59399999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.72099999999999997,"15min":0.73699999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.524,"15min":1.524},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.0660000000000001,"15min":1.105},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":3.105,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":0.65800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":319323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.47699999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.40400000000000003}]},{"osd":1,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.372,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.251},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41799999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.36799999999999999}]},{"osd":2,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45000000000000001,"15min":0.45400000000000001},"min":{"1min":0.33800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.53600000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.315},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45000000000000001,"15min":0.45400000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.44800000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76200000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91500000000000004,"5min":1.093,"15min":1.093},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.80800000000000005,"5min":1.042,"15min":1.042},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89200000000000002,"5min":1.012,"15min":1.012},"last":0.72899999999999998}]},{"osd":6,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.68899999999999995,"15min":0.69999999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.78900000000000003,"5min":1.024,"15min":1.024},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.78200000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.878,"5min":1.085,"15min":1.085},"last":0.66300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":319323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.67600000000000005,"15min":0.69499999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.87,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.66200000000000003}]},{"osd":1,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":1.0289999999999999,"15min":1.113},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.747,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.75900000000000001}]},{"osd":2,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76800000000000002,"5min":1.494,"15min":1.494},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.71299999999999997}]},{"osd":3,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.86599999999999999,"15min":0.91000000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":1.1279999999999999,"15min":1.218},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73499999999999999,"5min":9.452,"15min":9.452},"last":0.83599999999999997}]},{"osd":5,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78900000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76400000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.68500000000000005,"15min":0.69899999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82499999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89200000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.871}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":716800,"data_stored":326440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.83399999999999996,"15min":0.876},"min":{"1min":0.39800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.88400000000000001,"15min":0.93200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88200000000000001,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.502}]},{"osd":1,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.84999999999999998,"15min":0.89800000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81999999999999995,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.89400000000000002,"15min":0.93999999999999995},"min":{"1min":0.436,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94099999999999995,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.83999999999999997,"15min":0.88600000000000001},"min":{"1min":0.373,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80400000000000005,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77000000000000002,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.88800000000000001,"15min":0.94499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91500000000000004,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86499999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85299999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45400000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89000000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.627,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.63500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:43:16.924 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T01:43:16.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T01:43:17.008 INFO:tasks.ceph.mds.d.smithi059.stderr:2024-03-30T01:43:17.002+0000 7f43abf06700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T01:43:17.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.101+0000 7f0be58a4700 1 Processor -- start 2024-03-30T01:43:17.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.101+0000 7f0be58a4700 1 -- start start 2024-03-30T01:43:17.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0be58a4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0be0072090 0x7f0be006c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0be58a4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be00c2430 0x7f0be00c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0be58a4700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be00c58d0 con 0x7f0be00bc820 2024-03-30T01:43:17.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0be58a4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be006d000 con 0x7f0be0072090 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0be58a4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be00c5350 con 0x7f0be00c2430 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bdeffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0be00bc820 0x7f0be00c1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58860/0 (socket says 172.21.15.31:58860) 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bdeffd700 1 -- 172.21.15.31:0/2119134355 learned_addr learned my addr 172.21.15.31:0/2119134355 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bdf7fe700 1 --2- 172.21.15.31:0/2119134355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be00c2430 0x7f0be00c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bde7fc700 1 --2- 172.21.15.31:0/2119134355 >> v2:172.21.15.59:3300/0 conn(0x7f0be0072090 0x7f0be006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bddffb700 1 -- 172.21.15.31:0/2119134355 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852940448 0 0) 0x7f0be00c58d0 con 0x7f0be00bc820 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bddffb700 1 -- 172.21.15.31:0/2119134355 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc80035b0 con 0x7f0be00bc820 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bdf7fe700 1 -- 172.21.15.31:0/2119134355 >> v2:172.21.15.59:3300/0 conn(0x7f0be0072090 msgr2=0x7f0be006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bdf7fe700 1 --2- 172.21.15.31:0/2119134355 >> v2:172.21.15.59:3300/0 conn(0x7f0be0072090 0x7f0be006c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bdf7fe700 1 -- 172.21.15.31:0/2119134355 >> v1:172.21.15.31:6789/0 conn(0x7f0be00bc820 legacy=0x7f0be00c1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.102+0000 7f0bdf7fe700 1 -- 172.21.15.31:0/2119134355 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be00c55f0 con 0x7f0be00c2430 2024-03-30T01:43:17.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0bdf7fe700 1 --2- 172.21.15.31:0/2119134355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be00c2430 0x7f0be00c28a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009d90 tx=0x7f0bd800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feafbeff8d8e0d62 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0bddffb700 1 -- 172.21.15.31:0/2119134355 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800e070 con 0x7f0be00c2430 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0bddffb700 1 -- 172.21.15.31:0/2119134355 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd800bb20 con 0x7f0be00c2430 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0bddffb700 1 -- 172.21.15.31:0/2119134355 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8012430 con 0x7f0be00c2430 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0be58a4700 1 -- 172.21.15.31:0/2119134355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be00c2430 msgr2=0x7f0be00c28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0be58a4700 1 --2- 172.21.15.31:0/2119134355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be00c2430 0x7f0be00c28a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009d90 tx=0x7f0bd800b330 comp rx=0 tx=0).stop 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0be58a4700 1 -- 172.21.15.31:0/2119134355 shutdown_connections 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0be58a4700 1 --2- 172.21.15.31:0/2119134355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be00c2430 0x7f0be00c28a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0be58a4700 1 --2- 172.21.15.31:0/2119134355 >> v2:172.21.15.59:3300/0 conn(0x7f0be0072090 0x7f0be006c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0be58a4700 1 -- 172.21.15.31:0/2119134355 >> 172.21.15.31:0/2119134355 conn(0x7f0be006b110 msgr2=0x7f0be00a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0be58a4700 1 -- 172.21.15.31:0/2119134355 shutdown_connections 2024-03-30T01:43:17.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.103+0000 7f0be58a4700 1 -- 172.21.15.31:0/2119134355 wait complete. 2024-03-30T01:43:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.105+0000 7f0be58a4700 1 Processor -- start 2024-03-30T01:43:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.105+0000 7f0be58a4700 1 -- start start 2024-03-30T01:43:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.106+0000 7f0be58a4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be0072090 0x7f0be00a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.106+0000 7f0be58a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0be00bc820 0x7f0be00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.106+0000 7f0be58a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be00c2430 0x7f0be00a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.106+0000 7f0be58a4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01e8100 con 0x7f0be00bc820 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.106+0000 7f0be58a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01e7830 con 0x7f0be00c2430 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.106+0000 7f0be58a4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0071c40 con 0x7f0be0072090 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.106+0000 7f0bde7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0be00bc820 0x7f0be00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.106+0000 7f0bde7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0be00bc820 0x7f0be00a3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54422/0 (socket says 172.21.15.31:54422) 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bde7fc700 1 -- 172.21.15.31:0/3854951484 learned_addr learned my addr 172.21.15.31:0/3854951484 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bdeffd700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be0072090 0x7f0be00a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bdf7fe700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be00c2430 0x7f0be00a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bde7fc700 1 -- 172.21.15.31:0/3854951484 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be0072090 msgr2=0x7f0be00a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bde7fc700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be0072090 0x7f0be00a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bde7fc700 1 -- 172.21.15.31:0/3854951484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be00c2430 msgr2=0x7f0be00a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bde7fc700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be00c2430 0x7f0be00a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bde7fc700 1 -- 172.21.15.31:0/3854951484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd8009a70 con 0x7f0be00bc820 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bdf7fe700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be00c2430 0x7f0be00a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bde7fc700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0be00bc820 0x7f0be00a3850 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e8a0 tx=0x7f0bd000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bc7fff700 1 -- 172.21.15.31:0/3854951484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0009590 con 0x7f0be00bc820 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bc7fff700 1 -- 172.21.15.31:0/3854951484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd000eea0 con 0x7f0be00bc820 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be00a47c0 con 0x7f0be00bc820 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0bc7fff700 1 -- 172.21.15.31:0/3854951484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000ca50 con 0x7f0be00bc820 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.107+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be01e7ab0 con 0x7f0be00bc820 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.108+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be00747d0 con 0x7f0be00bc820 2024-03-30T01:43:17.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.109+0000 7f0bc7fff700 1 -- 172.21.15.31:0/3854951484 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0bd000cbb0 con 0x7f0be00bc820 2024-03-30T01:43:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.109+0000 7f0bc7fff700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0bcc06ac90 0x7f0bcc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.109+0000 7f0bc7fff700 1 -- 172.21.15.31:0/3854951484 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0bd0089380 con 0x7f0be00bc820 2024-03-30T01:43:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.109+0000 7f0bdeffd700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0bcc06ac90 0x7f0bcc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.110+0000 7f0bdeffd700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0bcc06ac90 0x7f0bcc06d150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40011e0 tx=0x7f0bd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.114+0000 7f0bc7fff700 1 -- 172.21.15.31:0/3854951484 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0bd00544b0 con 0x7f0be00bc820 2024-03-30T01:43:17.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.417+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0be00c40a0 con 0x7f0be00bc820 2024-03-30T01:43:17.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.421+0000 7f0bc7fff700 1 -- 172.21.15.31:0/3854951484 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0bd0057ad0 con 0x7f0be00bc820 2024-03-30T01:43:17.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0bcc06ac90 msgr2=0x7f0bcc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0bcc06ac90 0x7f0bcc06d150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40011e0 tx=0x7f0bd4009040 comp rx=0 tx=0).stop 2024-03-30T01:43:17.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0be00bc820 msgr2=0x7f0be00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0be00bc820 0x7f0be00a3850 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e8a0 tx=0x7f0bd000ec60 comp rx=0 tx=0).stop 2024-03-30T01:43:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 shutdown_connections 2024-03-30T01:43:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f0bcc06ac90 0x7f0bcc06d150 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0be00c2430 0x7f0be00a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0be0072090 0x7f0be00a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 --2- 172.21.15.31:0/3854951484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0be00bc820 0x7f0be00a3850 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 >> 172.21.15.31:0/3854951484 conn(0x7f0be006b110 msgr2=0x7f0be0070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 shutdown_connections 2024-03-30T01:43:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.424+0000 7f0be58a4700 1 -- 172.21.15.31:0/3854951484 wait complete. 2024-03-30T01:43:17.431 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2024-03-30T01:43:17.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:43:17.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.573+0000 7f390146e700 1 Processor -- start 2024-03-30T01:43:17.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.573+0000 7f390146e700 1 -- start start 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f390146e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 0x7f38fc155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f390146e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f38fc0bd7f0 0x7f38fc06b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f390146e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f38fc154760 0x7f38fc0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f390146e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc06bef0 con 0x7f38fc154760 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f390146e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0c8d10 con 0x7f38fc0bd7f0 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f390146e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0c8100 con 0x7f38fc06b170 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f38fb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f38fc154760 0x7f38fc0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f38fb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f38fc154760 0x7f38fc0c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54424/0 (socket says 172.21.15.31:54424) 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f38fb7fe700 1 -- 172.21.15.31:0/3584690903 learned_addr learned my addr 172.21.15.31:0/3584690903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f38faffd700 1 --2- 172.21.15.31:0/3584690903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 0x7f38fc155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f38fb7fe700 1 -- 172.21.15.31:0/3584690903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 msgr2=0x7f38fc155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f38fb7fe700 1 --2- 172.21.15.31:0/3584690903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 0x7f38fc155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f38fa7fc700 1 --2- 172.21.15.31:0/3584690903 >> v2:172.21.15.59:3300/0 conn(0x7f38fc0bd7f0 0x7f38fc06b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.574+0000 7f38fb7fe700 1 -- 172.21.15.31:0/3584690903 >> v2:172.21.15.59:3300/0 conn(0x7f38fc0bd7f0 msgr2=0x7f38fc06b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f38fb7fe700 1 --2- 172.21.15.31:0/3584690903 >> v2:172.21.15.59:3300/0 conn(0x7f38fc0bd7f0 0x7f38fc06b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f38fb7fe700 1 -- 172.21.15.31:0/3584690903 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0c83e0 con 0x7f38fc154760 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f38fa7fc700 1 --2- 172.21.15.31:0/3584690903 >> v2:172.21.15.59:3300/0 conn(0x7f38fc0bd7f0 0x7f38fc06b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f38fb7fe700 1 --2- 172.21.15.31:0/3584690903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 0x7f38fc0c91a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0acba0 tx=0x7f38f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e424760f1af7897d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f38f9ffb700 1 -- 172.21.15.31:0/3584690903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400e070 con 0x7f38fc154760 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f38f9ffb700 1 -- 172.21.15.31:0/3584690903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f400b9c0 con 0x7f38fc154760 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f38f9ffb700 1 -- 172.21.15.31:0/3584690903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400bd70 con 0x7f38fc154760 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f390146e700 1 -- 172.21.15.31:0/3584690903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 msgr2=0x7f38fc0c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f390146e700 1 --2- 172.21.15.31:0/3584690903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 0x7f38fc0c91a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0acba0 tx=0x7f38f400b330 comp rx=0 tx=0).stop 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f390146e700 1 -- 172.21.15.31:0/3584690903 shutdown_connections 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f390146e700 1 --2- 172.21.15.31:0/3584690903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 0x7f38fc0c91a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f390146e700 1 --2- 172.21.15.31:0/3584690903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 0x7f38fc155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f390146e700 1 --2- 172.21.15.31:0/3584690903 >> v2:172.21.15.59:3300/0 conn(0x7f38fc0bd7f0 0x7f38fc06b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.575+0000 7f390146e700 1 -- 172.21.15.31:0/3584690903 >> 172.21.15.31:0/3584690903 conn(0x7f38fc0ca0d0 msgr2=0x7f38fc0ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.576+0000 7f390146e700 1 -- 172.21.15.31:0/3584690903 shutdown_connections 2024-03-30T01:43:17.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.576+0000 7f390146e700 1 -- 172.21.15.31:0/3584690903 wait complete. 2024-03-30T01:43:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.577+0000 7f390146e700 1 Processor -- start 2024-03-30T01:43:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.577+0000 7f390146e700 1 -- start start 2024-03-30T01:43:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f390146e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 0x7f38fc0be2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f390146e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38fc0bd7f0 0x7f38fc0ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f390146e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 0x7f38fc0c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f390146e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0c3bd0 con 0x7f38fc154760 2024-03-30T01:43:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f390146e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0bde30 con 0x7f38fc0bd7f0 2024-03-30T01:43:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f390146e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0be800 con 0x7f38fc06b170 2024-03-30T01:43:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 0x7f38fc0c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 0x7f38fc0c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54426/0 (socket says 172.21.15.31:54426) 2024-03-30T01:43:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fb7fe700 1 -- 172.21.15.31:0/1425189844 learned_addr learned my addr 172.21.15.31:0/1425189844 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38faffd700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 0x7f38fc0be2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fa7fc700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38fc0bd7f0 0x7f38fc0ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fb7fe700 1 -- 172.21.15.31:0/1425189844 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 msgr2=0x7f38fc0be2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fb7fe700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 0x7f38fc0be2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fb7fe700 1 -- 172.21.15.31:0/1425189844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38fc0bd7f0 msgr2=0x7f38fc0ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fb7fe700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38fc0bd7f0 0x7f38fc0ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fb7fe700 1 -- 172.21.15.31:0/1425189844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4009b30 con 0x7f38fc154760 2024-03-30T01:43:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fa7fc700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38fc0bd7f0 0x7f38fc0ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.578+0000 7f38fb7fe700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 0x7f38fc0c2570 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f38fc067230 tx=0x7f38f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.579+0000 7f38e37fe700 1 -- 172.21.15.31:0/1425189844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4004250 con 0x7f38fc154760 2024-03-30T01:43:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.579+0000 7f38e37fe700 1 -- 172.21.15.31:0/1425189844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f40043d0 con 0x7f38fc154760 2024-03-30T01:43:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.579+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc1ea430 con 0x7f38fc154760 2024-03-30T01:43:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.579+0000 7f38e37fe700 1 -- 172.21.15.31:0/1425189844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4004780 con 0x7f38fc154760 2024-03-30T01:43:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.579+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc0aeab0 con 0x7f38fc154760 2024-03-30T01:43:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.580+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc0bdfa0 con 0x7f38fc154760 2024-03-30T01:43:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.580+0000 7f38e37fe700 1 -- 172.21.15.31:0/1425189844 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f38f401b430 con 0x7f38fc154760 2024-03-30T01:43:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.581+0000 7f38e37fe700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38e806ac90 0x7f38e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.581+0000 7f38e37fe700 1 -- 172.21.15.31:0/1425189844 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f38f408a940 con 0x7f38fc154760 2024-03-30T01:43:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.581+0000 7f38faffd700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38e806ac90 0x7f38e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.581+0000 7f38faffd700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38e806ac90 0x7f38e806d150 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f38fc151e20 tx=0x7f38ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:17.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.585+0000 7f38e37fe700 1 -- 172.21.15.31:0/1425189844 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f38f4055aa0 con 0x7f38fc154760 2024-03-30T01:43:17.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.848+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38fc04ea80 con 0x7f38fc154760 2024-03-30T01:43:17.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.848+0000 7f38e37fe700 1 -- 172.21.15.31:0/1425189844 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f38f40590c0 con 0x7f38fc154760 2024-03-30T01:43:17.855 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:17.855 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3969533250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3969533250},{"type":"v1","addr":"172.21.15.31:6836","nonce":3969533250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3308862528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3308862528},{"type":"v1","addr":"172.21.15.59:6835","nonce":3308862528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3281184299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3281184299},{"type":"v1","addr":"172.21.15.31:6837","nonce":3281184299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:43:12.245458+0000","modified":"2024-03-30T01:43:15.261882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6000":{"gid":6000,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3498371628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3498371628},{"type":"v1","addr":"172.21.15.59:6834","nonce":3498371628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-30T01:43:17.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38e806ac90 msgr2=0x7f38e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38e806ac90 0x7f38e806d150 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f38fc151e20 tx=0x7f38ec008040 comp rx=0 tx=0).stop 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 msgr2=0x7f38fc0c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 0x7f38fc0c2570 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f38fc067230 tx=0x7f38f4004ad0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 shutdown_connections 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38fc154760 0x7f38fc0c2570 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f38e806ac90 0x7f38e806d150 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38fc06b170 0x7f38fc0be2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 --2- 172.21.15.31:0/1425189844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38fc0bd7f0 0x7f38fc0ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 >> 172.21.15.31:0/1425189844 conn(0x7f38fc0ca0d0 msgr2=0x7f38fc156c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 shutdown_connections 2024-03-30T01:43:17.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:17.851+0000 7f390146e700 1 -- 172.21.15.31:0/1425189844 wait complete. 2024-03-30T01:43:17.859 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 31 2024-03-30T01:43:18.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T01:43:12.245458+0000', 'modified': '2024-03-30T01:43:15.261882+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6000}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6000': {'gid': 6000, 'name': 'd', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6834/3498371628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6832', 'nonce': 3498371628}, {'type': 'v1', 'addr': '172.21.15.59:6834', 'nonce': 3498371628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T01:43:18.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T01:43:18.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-30T01:43:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.415+0000 7f581bb13700 1 Processor -- start 2024-03-30T01:43:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.415+0000 7f581bb13700 1 -- start start 2024-03-30T01:43:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f581bb13700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f581414e5f0 0x7f58140c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f581bb13700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5814071240 0x7f5814154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f581bb13700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5814072160 con 0x7f58140c4100 2024-03-30T01:43:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f581bb13700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140c2bd0 con 0x7f581414e5f0 2024-03-30T01:43:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f581bb13700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140c2e90 con 0x7f5814071240 2024-03-30T01:43:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f58198af700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f581414e5f0 0x7f58140c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f58198af700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f581414e5f0 0x7f58140c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54434/0 (socket says 172.21.15.31:54434) 2024-03-30T01:43:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f58198af700 1 -- 172.21.15.31:0/3767889140 learned_addr learned my addr 172.21.15.31:0/3767889140 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:18.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f58190ae700 1 --2- 172.21.15.31:0/3767889140 >> v2:172.21.15.59:3300/0 conn(0x7f5814071240 0x7f5814154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f58198af700 1 -- 172.21.15.31:0/3767889140 >> v2:172.21.15.59:3300/0 conn(0x7f5814071240 msgr2=0x7f5814154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f58198af700 1 --2- 172.21.15.31:0/3767889140 >> v2:172.21.15.59:3300/0 conn(0x7f5814071240 0x7f5814154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f58198af700 1 -- 172.21.15.31:0/3767889140 >> v1:172.21.15.59:6789/0 conn(0x7f58140c4100 legacy=0x7f5814154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.416+0000 7f58198af700 1 -- 172.21.15.31:0/3767889140 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140723b0 con 0x7f581414e5f0 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f58198af700 1 --2- 172.21.15.31:0/3767889140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f581414e5f0 0x7f58140c3bc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f58140c2b00 tx=0x7f5804009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11a1e450f0cd1cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f58188ad700 1 -- 172.21.15.31:0/3767889140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804004230 con 0x7f581414e5f0 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f58188ad700 1 -- 172.21.15.31:0/3767889140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5804004390 con 0x7f581414e5f0 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f58188ad700 1 -- 172.21.15.31:0/3767889140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58040036a0 con 0x7f581414e5f0 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f581bb13700 1 -- 172.21.15.31:0/3767889140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f581414e5f0 msgr2=0x7f58140c3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f581bb13700 1 --2- 172.21.15.31:0/3767889140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f581414e5f0 0x7f58140c3bc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f58140c2b00 tx=0x7f5804009bd0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f581bb13700 1 -- 172.21.15.31:0/3767889140 shutdown_connections 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f581bb13700 1 --2- 172.21.15.31:0/3767889140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f581414e5f0 0x7f58140c3bc0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f581bb13700 1 --2- 172.21.15.31:0/3767889140 >> v2:172.21.15.59:3300/0 conn(0x7f5814071240 0x7f5814154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f581bb13700 1 -- 172.21.15.31:0/3767889140 >> 172.21.15.31:0/3767889140 conn(0x7f581406ba20 msgr2=0x7f581406be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.417+0000 7f581bb13700 1 -- 172.21.15.31:0/3767889140 shutdown_connections 2024-03-30T01:43:18.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.418+0000 7f581bb13700 1 -- 172.21.15.31:0/3767889140 wait complete. 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.419+0000 7f581bb13700 1 Processor -- start 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.419+0000 7f581bb13700 1 -- start start 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.419+0000 7f581bb13700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5814071240 0x7f58141ea440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.419+0000 7f581bb13700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58140c4100 0x7f5814147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.419+0000 7f581bb13700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f581414e5f0 0x7f5814147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.419+0000 7f581bb13700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5814155760 con 0x7f5814071240 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.419+0000 7f581bb13700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58141ea980 con 0x7f581414e5f0 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.419+0000 7f581bb13700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5814158710 con 0x7f58140c4100 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58198af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5814071240 0x7f58141ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58198af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5814071240 0x7f58141ea440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54446/0 (socket says 172.21.15.31:54446) 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58198af700 1 -- 172.21.15.31:0/3441225241 learned_addr learned my addr 172.21.15.31:0/3441225241 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58190ae700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58140c4100 0x7f5814147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f581a0b0700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f581414e5f0 0x7f5814147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58198af700 1 -- 172.21.15.31:0/3441225241 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58140c4100 msgr2=0x7f5814147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58198af700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58140c4100 0x7f5814147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58198af700 1 -- 172.21.15.31:0/3441225241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f581414e5f0 msgr2=0x7f5814147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58198af700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f581414e5f0 0x7f5814147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58198af700 1 -- 172.21.15.31:0/3441225241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5804009550 con 0x7f5814071240 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f58198af700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5814071240 0x7f58141ea440 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f58140c4a20 tx=0x7f5804004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f580a7fc700 1 -- 172.21.15.31:0/3441225241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804011630 con 0x7f5814071240 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f580a7fc700 1 -- 172.21.15.31:0/3441225241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5804011790 con 0x7f5814071240 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f581bb13700 1 -- 172.21.15.31:0/3441225241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58141589f0 con 0x7f5814071240 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f580a7fc700 1 -- 172.21.15.31:0/3441225241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804011b40 con 0x7f5814071240 2024-03-30T01:43:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.420+0000 7f581bb13700 1 -- 172.21.15.31:0/3441225241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5814155c90 con 0x7f5814071240 2024-03-30T01:43:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.425+0000 7f580a7fc700 1 -- 172.21.15.31:0/3441225241 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5804011ca0 con 0x7f5814071240 2024-03-30T01:43:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.425+0000 7f580a7fc700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57f806ac90 0x7f57f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.425+0000 7f5827ca9b80 1 Processor -- start 2024-03-30T01:43:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.425+0000 7f580a7fc700 1 -- 172.21.15.31:0/3441225241 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f580408aab0 con 0x7f5814071240 2024-03-30T01:43:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.426+0000 7f5827ca9b80 1 -- start start 2024-03-30T01:43:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.426+0000 7f58190ae700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57f806ac90 0x7f57f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.426+0000 7f58190ae700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57f806ac90 0x7f57f806d150 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5814148940 tx=0x7f5810008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.426+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa39b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.426+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a1090 0x55dffa3ddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.426+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3a03d0 0x55dffa3bde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.426+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa2f7940 con 0x55dffa3a1090 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.426+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa2f5300 con 0x55dffa39fff0 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.426+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dffa2f0bf0 con 0x55dffa3a03d0 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58190ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a1090 0x55dffa3ddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58190ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a1090 0x55dffa3ddf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54460/0 (socket says 172.21.15.31:54460) 2024-03-30T01:43:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58190ae700 1 -- 172.21.15.31:0/4247876941 learned_addr learned my addr 172.21.15.31:0/4247876941 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f581a0b0700 1 --2- 172.21.15.31:0/4247876941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3a03d0 0x55dffa3bde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58198af700 1 --2- 172.21.15.31:0/4247876941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa39b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58190ae700 1 -- 172.21.15.31:0/4247876941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3a03d0 msgr2=0x55dffa3bde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58190ae700 1 --2- 172.21.15.31:0/4247876941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3a03d0 0x55dffa3bde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58190ae700 1 -- 172.21.15.31:0/4247876941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 msgr2=0x55dffa39b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58190ae700 1 --2- 172.21.15.31:0/4247876941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa39b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58190ae700 1 -- 172.21.15.31:0/4247876941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dffa11eb10 con 0x55dffa3a1090 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f58190ae700 1 --2- 172.21.15.31:0/4247876941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a1090 0x55dffa3ddf80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f581000fb80 tx=0x7f5810012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a581165be5c3ffe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f57f77fe700 1 -- 172.21.15.31:0/4247876941 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581000f700 con 0x55dffa3a1090 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f57f77fe700 1 -- 172.21.15.31:0/4247876941 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581000c040 con 0x55dffa3a1090 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.427+0000 7f57f77fe700 1 -- 172.21.15.31:0/4247876941 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810004470 con 0x55dffa3a1090 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.428+0000 7f5827ca9b80 1 -- 172.21.15.31:0/4247876941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a1090 msgr2=0x55dffa3ddf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.428+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/4247876941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a1090 0x55dffa3ddf80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f581000fb80 tx=0x7f5810012000 comp rx=0 tx=0).stop 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.428+0000 7f5827ca9b80 1 -- 172.21.15.31:0/4247876941 shutdown_connections 2024-03-30T01:43:18.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.428+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/4247876941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3a03d0 0x55dffa3bde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.428+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/4247876941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa39b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.428+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/4247876941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a1090 0x55dffa3ddf80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.428+0000 7f5827ca9b80 1 -- 172.21.15.31:0/4247876941 >> 172.21.15.31:0/4247876941 conn(0x55dffa2fe650 msgr2=0x55dffa39dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.428+0000 7f5827ca9b80 1 -- 172.21.15.31:0/4247876941 shutdown_connections 2024-03-30T01:43:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.428+0000 7f5827ca9b80 1 -- 172.21.15.31:0/4247876941 wait complete. 2024-03-30T01:43:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.429+0000 7f5827ca9b80 1 -- start start 2024-03-30T01:43:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.429+0000 7f5827ca9b80 1 Processor -- start 2024-03-30T01:43:18.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa3d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3ec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa3bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa28cd30 con 0x55dffa3a03d0 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa28c9d0 con 0x55dffa39fff0 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dffa174650 con 0x55dffa3bc210 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f5827ca9b80 10 client.0 _ll_register_callbacks cb 0x55dffa296420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58190ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3ec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58190ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3ec1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54470/0 (socket says 172.21.15.31:54470) 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58190ae700 1 -- 172.21.15.31:0/1249606323 learned_addr learned my addr 172.21.15.31:0/1249606323 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f581a0b0700 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa3bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58198af700 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa3d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58190ae700 1 -- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 msgr2=0x55dffa3bc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58190ae700 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa3bc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58190ae700 1 -- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 msgr2=0x55dffa3d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58190ae700 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa3d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58190ae700 1 -- 172.21.15.31:0/1249606323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581000f860 con 0x55dffa3a03d0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.430+0000 7f58190ae700 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3ec1a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55dffa39da60 tx=0x7f58100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5660e76e0e718eff server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f57f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f57f4ff9700 1 -- 172.21.15.31:0/1249606323 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581002c070 con 0x55dffa3a03d0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f57f4ff9700 1 -- 172.21.15.31:0/1249606323 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f581001a040 con 0x55dffa3a03d0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1249606323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dffa2b1330 con 0x55dffa3a03d0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f57f4ff9700 1 -- 172.21.15.31:0/1249606323 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5810015970 con 0x55dffa3a03d0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f5827ca9b80 10 client.6144 fetch_fsmap learned FSMap version 31 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1249606323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55dffa2b6e50 con 0x55dffa3a03d0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f57f4ff9700 1 -- 172.21.15.31:0/1249606323 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dffa2b1330 con 0x55dffa3a03d0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f57f4ff9700 1 -- 172.21.15.31:0/1249606323 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5810015ad0 con 0x55dffa3a03d0 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f5827ca9b80 10 client.6144 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T01:43:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f5827ca9b80 10 client.6144 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f5827ca9b80 20 client.6144 populate_metadata read hostname 'smithi031' 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] conn(0x55dffa3ef840 0x55dffa3ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f5827ca9b80 4 client.6144 mds_command: new command op to 6000 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.431+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1249606323 --> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dffa2e96b0 con 0x55dffa3ef840 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.432+0000 7f58198af700 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] conn(0x55dffa3ef840 0x55dffa3ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.432+0000 7f57f4ff9700 10 client.6144 ms_handle_connect on v2:172.21.15.59:6832/3498371628 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.432+0000 7f58198af700 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] conn(0x55dffa3ef840 0x55dffa3ba940 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ae96888982b317 server_cookie=b74b6c959460aceb in_seq=0 out_seq=0 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f57f4ff9700 1 -- 172.21.15.31:0/1249606323 <== mds.0 v2:172.21.15.59:6832/3498371628 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dffa2e96b0 con 0x55dffa3ef840 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f57f4ff9700 10 client.6144 handle_command_reply: tid=0 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 client.6144 shutdown 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1249606323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 msgr2=0x55dffa3ec1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3ec1a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55dffa39da60 tx=0x7f58100047c0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1249606323 shutdown_connections 2024-03-30T01:43:18.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] conn(0x55dffa3ef840 0x55dffa3ba940 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa3bc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa3d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1249606323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3ec1a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1249606323 >> 172.21.15.31:0/1249606323 conn(0x55dffa2fe650 msgr2=0x55dffa3c0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:18.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f57f4ff9700 0 client.6144 ms_handle_reset on v2:172.21.15.59:6832/3498371628 2024-03-30T01:43:18.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1249606323 shutdown_connections 2024-03-30T01:43:18.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1249606323 wait complete. 2024-03-30T01:43:18.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.433+0000 7f5827ca9b80 20 client.6144 trim_cache size 0 max 16384 2024-03-30T01:43:18.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.452+0000 7f5827ca9b80 1 Processor -- start 2024-03-30T01:43:18.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.452+0000 7f5827ca9b80 1 -- start start 2024-03-30T01:43:18.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa39c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3e0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa3d88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa3e8ee0 con 0x55dffa3a03d0 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa3c02d0 con 0x55dffa39fff0 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dffa3c00b0 con 0x55dffa3bc210 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58190ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3e0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58190ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3e0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54474/0 (socket says 172.21.15.31:54474) 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58190ae700 1 -- 172.21.15.31:0/1496901772 learned_addr learned my addr 172.21.15.31:0/1496901772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58198af700 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa39c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f581a0b0700 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa3d88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58190ae700 1 -- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 msgr2=0x55dffa3d88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58190ae700 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa3d88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58190ae700 1 -- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 msgr2=0x55dffa39c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58190ae700 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa39c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58190ae700 1 -- 172.21.15.31:0/1496901772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dffa39acf0 con 0x55dffa3a03d0 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f581a0b0700 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa3d88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58198af700 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa39c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f58190ae700 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3e0010 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5810008010 tx=0x7f5810029050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8054024b157759db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f57f7fff700 1 -- 172.21.15.31:0/1496901772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810015970 con 0x55dffa3a03d0 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f57f7fff700 1 -- 172.21.15.31:0/1496901772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810015ad0 con 0x55dffa3a03d0 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.453+0000 7f57f7fff700 1 -- 172.21.15.31:0/1496901772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810003a00 con 0x55dffa3a03d0 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.454+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1496901772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 msgr2=0x55dffa3e0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.454+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3e0010 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5810008010 tx=0x7f5810029050 comp rx=0 tx=0).stop 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.454+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1496901772 shutdown_connections 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.454+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa3d88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.454+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa39c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.454+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/1496901772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3e0010 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.454+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1496901772 >> 172.21.15.31:0/1496901772 conn(0x55dffa2fe650 msgr2=0x55dffa3965c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:18.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.454+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1496901772 shutdown_connections 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.454+0000 7f5827ca9b80 1 -- 172.21.15.31:0/1496901772 wait complete. 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.455+0000 7f5827ca9b80 1 -- start start 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.455+0000 7f5827ca9b80 1 Processor -- start 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa3d8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f5827ca9b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa39dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa3ebbb0 con 0x55dffa3a03d0 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa3b4260 con 0x55dffa39fff0 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f5827ca9b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dffa399d00 con 0x55dffa3bc210 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f5827ca9b80 10 client.0 _ll_register_callbacks cb 0x55dffa296420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58190ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58190ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54488/0 (socket says 172.21.15.31:54488) 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58190ae700 1 -- 172.21.15.31:0/413915105 learned_addr learned my addr 172.21.15.31:0/413915105 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:18.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58198af700 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa3d8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f581a0b0700 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa39dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58190ae700 1 -- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 msgr2=0x55dffa39dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58190ae700 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa39dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58190ae700 1 -- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 msgr2=0x55dffa3d8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58190ae700 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa3d8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58190ae700 1 -- 172.21.15.31:0/413915105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dffa2b6e50 con 0x55dffa3a03d0 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f58190ae700 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3997c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5810042040 tx=0x7f5810019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc59e409b1b0fec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f57f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.456+0000 7f57f4ff9700 1 -- 172.21.15.31:0/413915105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810015970 con 0x55dffa3a03d0 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f57f4ff9700 1 -- 172.21.15.31:0/413915105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5810013620 con 0x55dffa3a03d0 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f5827ca9b80 1 -- 172.21.15.31:0/413915105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f581000c040 con 0x55dffa3a03d0 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f57f4ff9700 1 -- 172.21.15.31:0/413915105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5810015c40 con 0x55dffa3a03d0 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f5827ca9b80 10 client.6150 fetch_fsmap learned FSMap version 31 2024-03-30T01:43:18.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f5827ca9b80 1 -- 172.21.15.31:0/413915105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x7f5814154510 con 0x55dffa3a03d0 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f57f4ff9700 1 -- 172.21.15.31:0/413915105 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581000c040 con 0x55dffa3a03d0 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f57f4ff9700 1 -- 172.21.15.31:0/413915105 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5810005a20 con 0x55dffa3a03d0 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f5827ca9b80 10 client.6150 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f5827ca9b80 10 client.6150 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f5827ca9b80 20 client.6150 populate_metadata read hostname 'smithi031' 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] conn(0x55dffa3d3880 0x55dffa3d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f5827ca9b80 4 client.6150 mds_command: new command op to 6000 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f5827ca9b80 1 -- 172.21.15.31:0/413915105 --> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dffa13fca0 con 0x55dffa3d3880 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.457+0000 7f58198af700 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] conn(0x55dffa3d3880 0x55dffa3d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f57f4ff9700 10 client.6150 ms_handle_connect on v2:172.21.15.59:6832/3498371628 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f58198af700 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] conn(0x55dffa3d3880 0x55dffa3d3c60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1cb4abd0c3affd server_cookie=882046e6dd9ec94d in_seq=0 out_seq=0 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f57f4ff9700 1 -- 172.21.15.31:0/413915105 <== mds.0 v2:172.21.15.59:6832/3498371628 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55dffa13fca0 con 0x55dffa3d3880 2024-03-30T01:43:18.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f57f4ff9700 10 client.6150 handle_command_reply: tid=0 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 client.6150 shutdown 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 -- 172.21.15.31:0/413915105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 msgr2=0x55dffa3997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3997c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5810042040 tx=0x7f5810019000 comp rx=0 tx=0).stop 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 -- 172.21.15.31:0/413915105 shutdown_connections 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:6832/3498371628,v1:172.21.15.59:6834/3498371628] conn(0x55dffa3d3880 0x55dffa3d3c60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dffa3bc210 0x55dffa39dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dffa39fff0 0x55dffa3d8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 --2- 172.21.15.31:0/413915105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dffa3a03d0 0x55dffa3997c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f57f4ff9700 0 client.6150 ms_handle_reset on v2:172.21.15.59:6832/3498371628 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 -- 172.21.15.31:0/413915105 >> 172.21.15.31:0/413915105 conn(0x55dffa2fe650 msgr2=0x55dffa3e0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 -- 172.21.15.31:0/413915105 shutdown_connections 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 1 -- 172.21.15.31:0/413915105 wait complete. 2024-03-30T01:43:18.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.458+0000 7f5827ca9b80 20 client.6150 trim_cache size 0 max 16384 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "9739890c-57ec-4503-9fd1-d4b0145023f4", 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6000, 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.2117962530000002, 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 31, 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 60, 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 60, 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 12.312425523 2024-03-30T01:43:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-03-30T01:43:18.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 -- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57f806ac90 msgr2=0x7f57f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57f806ac90 0x7f57f806d150 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5814148940 tx=0x7f5810008040 comp rx=0 tx=0).stop 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 -- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5814071240 msgr2=0x7f58141ea440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5814071240 0x7f58141ea440 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f58140c4a20 tx=0x7f5804004750 comp rx=0 tx=0).stop 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 -- 172.21.15.31:0/3441225241 shutdown_connections 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5814071240 0x7f58141ea440 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f57f806ac90 0x7f57f806d150 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f581414e5f0 0x7f5814147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 --2- 172.21.15.31:0/3441225241 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f58140c4100 0x7f5814147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 -- 172.21.15.31:0/3441225241 >> 172.21.15.31:0/3441225241 conn(0x7f581406ba20 msgr2=0x7f58140c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 -- 172.21.15.31:0/3441225241 shutdown_connections 2024-03-30T01:43:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.459+0000 7f581bb13700 1 -- 172.21.15.31:0/3441225241 wait complete. 2024-03-30T01:43:18.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:43:18.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.609+0000 7f1c0e491700 1 Processor -- start 2024-03-30T01:43:18.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.609+0000 7f1c0e491700 1 -- start start 2024-03-30T01:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c0e491700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1c080ab100 0x7f1c08156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c0e491700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c080722f0 0x7f1c08152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c0e491700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c08071ee0 con 0x7f1c0814fa40 2024-03-30T01:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c0e491700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c08152570 con 0x7f1c080ab100 2024-03-30T01:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c0e491700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0806d5c0 con 0x7f1c080722f0 2024-03-30T01:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c077fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1c080ab100 0x7f1c08156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c077fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1c080ab100 0x7f1c08156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54492/0 (socket says 172.21.15.31:54492) 2024-03-30T01:43:18.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c077fe700 1 -- 172.21.15.31:0/314584160 learned_addr learned my addr 172.21.15.31:0/314584160 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c0ca2e700 1 --2- 172.21.15.31:0/314584160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c080722f0 0x7f1c08152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c077fe700 1 -- 172.21.15.31:0/314584160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c080722f0 msgr2=0x7f1c08152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c077fe700 1 --2- 172.21.15.31:0/314584160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c080722f0 0x7f1c08152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c077fe700 1 -- 172.21.15.31:0/314584160 >> v1:172.21.15.59:6789/0 conn(0x7f1c0814fa40 legacy=0x7f1c080c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c077fe700 1 -- 172.21.15.31:0/314584160 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0806d860 con 0x7f1c080ab100 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c077fe700 1 --2- 172.21.15.31:0/314584160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080ab100 0x7f1c08156570 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0098f0 tx=0x7f1bfc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bb48bfe85ea710 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c06ffd700 1 -- 172.21.15.31:0/314584160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00b8a0 con 0x7f1c080ab100 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.610+0000 7f1c06ffd700 1 -- 172.21.15.31:0/314584160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc00ba00 con 0x7f1c080ab100 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.611+0000 7f1c06ffd700 1 -- 172.21.15.31:0/314584160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00bd90 con 0x7f1c080ab100 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.611+0000 7f1c0e491700 1 -- 172.21.15.31:0/314584160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080ab100 msgr2=0x7f1c08156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.611+0000 7f1c0e491700 1 --2- 172.21.15.31:0/314584160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080ab100 0x7f1c08156570 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0098f0 tx=0x7f1bfc009c00 comp rx=0 tx=0).stop 2024-03-30T01:43:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.611+0000 7f1c0e491700 1 -- 172.21.15.31:0/314584160 shutdown_connections 2024-03-30T01:43:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.611+0000 7f1c0e491700 1 --2- 172.21.15.31:0/314584160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c080722f0 0x7f1c08152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.611+0000 7f1c0e491700 1 --2- 172.21.15.31:0/314584160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080ab100 0x7f1c08156570 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.611+0000 7f1c0e491700 1 -- 172.21.15.31:0/314584160 >> 172.21.15.31:0/314584160 conn(0x7f1c0806ba20 msgr2=0x7f1c0806be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.611+0000 7f1c0e491700 1 -- 172.21.15.31:0/314584160 shutdown_connections 2024-03-30T01:43:18.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.611+0000 7f1c0e491700 1 -- 172.21.15.31:0/314584160 wait complete. 2024-03-30T01:43:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.613+0000 7f1c0e491700 1 Processor -- start 2024-03-30T01:43:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.613+0000 7f1c0e491700 1 -- start start 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c0e491700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080722f0 0x7f1c08154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c0e491700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c080ab100 0x7f1c080a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c0e491700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c0814fa40 0x7f1c081459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c0e491700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080a9110 con 0x7f1c080722f0 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c0e491700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080c1520 con 0x7f1c080ab100 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c0e491700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080c0f20 con 0x7f1c0814fa40 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c07fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080722f0 0x7f1c08154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c07fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080722f0 0x7f1c08154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54506/0 (socket says 172.21.15.31:54506) 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c07fff700 1 -- 172.21.15.31:0/2186569278 learned_addr learned my addr 172.21.15.31:0/2186569278 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c0ca2e700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c0814fa40 0x7f1c081459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c077fe700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c080ab100 0x7f1c080a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c07fff700 1 -- 172.21.15.31:0/2186569278 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c0814fa40 msgr2=0x7f1c081459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c07fff700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c0814fa40 0x7f1c081459d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c07fff700 1 -- 172.21.15.31:0/2186569278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c080ab100 msgr2=0x7f1c080a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c07fff700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c080ab100 0x7f1c080a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c07fff700 1 -- 172.21.15.31:0/2186569278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc009550 con 0x7f1c080722f0 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.614+0000 7f1c0ca2e700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c0814fa40 0x7f1c081459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.615+0000 7f1c07fff700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080722f0 0x7f1c08154d50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800c450 tx=0x7f1bf800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.615+0000 7f1c04ff9700 1 -- 172.21.15.31:0/2186569278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80052d0 con 0x7f1c080722f0 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.615+0000 7f1c04ff9700 1 -- 172.21.15.31:0/2186569278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8005430 con 0x7f1c080722f0 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.615+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c081523b0 con 0x7f1c080722f0 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.615+0000 7f1c04ff9700 1 -- 172.21.15.31:0/2186569278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80038c0 con 0x7f1c080722f0 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.615+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c080c0bb0 con 0x7f1c080722f0 2024-03-30T01:43:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.616+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0814f5f0 con 0x7f1c080722f0 2024-03-30T01:43:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.616+0000 7f1c04ff9700 1 -- 172.21.15.31:0/2186569278 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1bf8003a20 con 0x7f1c080722f0 2024-03-30T01:43:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.617+0000 7f1c04ff9700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1bf406ac90 0x7f1bf406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.617+0000 7f1c04ff9700 1 -- 172.21.15.31:0/2186569278 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1bf80888c0 con 0x7f1c080722f0 2024-03-30T01:43:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.617+0000 7f1c077fe700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1bf406ac90 0x7f1bf406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:18.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.617+0000 7f1c077fe700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1bf406ac90 0x7f1bf406d150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1c080c7e60 tx=0x7f1bfc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:18.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.621+0000 7f1c04ff9700 1 -- 172.21.15.31:0/2186569278 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1bf8053950 con 0x7f1c080722f0 2024-03-30T01:43:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.886+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1c080c0d40 con 0x7f1c080722f0 2024-03-30T01:43:18.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.887+0000 7f1c04ff9700 1 -- 172.21.15.31:0/2186569278 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1bf8017060 con 0x7f1c080722f0 2024-03-30T01:43:18.893 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T01:43:18.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1bf406ac90 msgr2=0x7f1bf406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1bf406ac90 0x7f1bf406d150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1c080c7e60 tx=0x7f1bfc008030 comp rx=0 tx=0).stop 2024-03-30T01:43:18.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080722f0 msgr2=0x7f1c08154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:18.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080722f0 0x7f1c08154d50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800c450 tx=0x7f1bf800c810 comp rx=0 tx=0).stop 2024-03-30T01:43:18.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 shutdown_connections 2024-03-30T01:43:18.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c080722f0 0x7f1c08154d50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1bf406ac90 0x7f1bf406d150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c0814fa40 0x7f1c081459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 --2- 172.21.15.31:0/2186569278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c080ab100 0x7f1c080a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:18.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 >> 172.21.15.31:0/2186569278 conn(0x7f1c0806ba20 msgr2=0x7f1c0806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:18.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.889+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 shutdown_connections 2024-03-30T01:43:18.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:18.890+0000 7f1c0e491700 1 -- 172.21.15.31:0/2186569278 wait complete. 2024-03-30T01:43:18.910 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T01:43:18.910 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:43:18.910 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-03-30T01:43:18.910 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T01:43:18.910 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T01:43:18.910 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T01:43:18.910 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-30T01:43:18.910 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T01:43:18.910 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:43:18.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:18.911 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-03-30T01:43:18.930 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:43:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:43:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:43:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42126sec preferred_lft 42126sec 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:43:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:37:22:42:e7:ee brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::2cb6:4cff:fefa:d1a5/64 scope link 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 3e:37:22:42:e7:ee brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::3c37:22ff:fe42:e7ee/64 scope link 2024-03-30T01:43:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:18.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:18.933 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:43:19.001 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-03-30T01:43:19.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:19.003 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-03-30T01:43:19.067 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-03-30T01:43:19.067 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:43:19.067 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:43:19.068 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:43:19.068 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:43:19.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:19.068 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2024-03-30T01:43:19.068 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-03-30T01:43:19.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:43:19.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:43:19.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:43:19.069 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-03-30T01:43:19.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:19.192 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-03-30T01:43:19.212 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T01:43:19.212 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T01:43:19.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:19.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-03-30T01:43:19.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:43:19.288 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T01:43:19.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:43:19.350 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T01:43:19.411 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-30T01:43:19.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:19.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:43:19.509 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:43:19.509 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-30T01:43:19.509 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:43:19.520 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:43:19.520 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-30T01:43:19.520 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:43:19.520 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:43:19.520 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:43:19.539 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-30T01:43:19.539 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-30T01:43:19.539 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-03-30T01:43:19.540 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:43:19.540 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:43:19.540 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790" 2024-03-30T01:43:19.540 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790:/ 2024-03-30T01:43:19.540 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9739890c-57ec-4503-9fd1-d4b0145023f4" will pass to kernel 2024-03-30T01:43:19.543 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T01:43:19.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:19.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T01:43:19.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:19.576 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:43:19.677 INFO:teuthology.orchestra.run.smithi031.stdout:client5423 2024-03-30T01:43:19.678 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:43:19.678 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:43:19.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:43:19.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.880+0000 7f9adb041700 1 Processor -- start 2024-03-30T01:43:19.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.880+0000 7f9adb041700 1 -- start start 2024-03-30T01:43:19.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9adb041700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad40c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:19.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9adb041700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ad40c2870 0x7f9ad40a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:19.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9adb041700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad41556d0 con 0x7f9ad40c5df0 2024-03-30T01:43:19.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9adb041700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad41559b0 con 0x7f9ad40c2870 2024-03-30T01:43:19.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9adb041700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40c1f70 con 0x7f9ad40c8310 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9ad8ddd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad40c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9ad95de700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ad40c2870 0x7f9ad40a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9ad95de700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ad40c2870 0x7f9ad40a6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:48602/0 (socket says 172.21.15.31:48602) 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9ad8ddd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad40c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:48720/0 (socket says 172.21.15.31:48720) 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9ad3fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9ad40c5df0 0x7f9ad4151d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:45760/0 (socket says 172.21.15.31:45760) 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9ad95de700 1 -- 172.21.15.31:0/917365312 learned_addr learned my addr 172.21.15.31:0/917365312 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9ad95de700 1 -- 172.21.15.31:0/917365312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 msgr2=0x7f9ad40c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.881+0000 7f9ad95de700 1 --2- 172.21.15.31:0/917365312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad40c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.882+0000 7f9ad95de700 1 -- 172.21.15.31:0/917365312 >> v1:172.21.15.59:6789/0 conn(0x7f9ad40c5df0 legacy=0x7f9ad4151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.882+0000 7f9ad95de700 1 -- 172.21.15.31:0/917365312 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad40c2250 con 0x7f9ad40c2870 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.882+0000 7f9ad8ddd700 1 --2- 172.21.15.31:0/917365312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad40c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.882+0000 7f9ad37fe700 1 -- 172.21.15.31:0/917365312 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000896262 0 0) 0x7f9ad41556d0 con 0x7f9ad40c5df0 2024-03-30T01:43:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.882+0000 7f9ad95de700 1 --2- 172.21.15.31:0/917365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad40c2870 0x7f9ad40a6ad0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009e50 tx=0x7f9acc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52429d4e5dc1ff2b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.882+0000 7f9ad37fe700 1 -- 172.21.15.31:0/917365312 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00e070 con 0x7f9ad40c2870 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.882+0000 7f9ad37fe700 1 -- 172.21.15.31:0/917365312 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc00b910 con 0x7f9ad40c2870 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.882+0000 7f9ad37fe700 1 -- 172.21.15.31:0/917365312 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00bcc0 con 0x7f9ad40c2870 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.883+0000 7f9adb041700 1 -- 172.21.15.31:0/917365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad40c2870 msgr2=0x7f9ad40a6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.883+0000 7f9adb041700 1 --2- 172.21.15.31:0/917365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad40c2870 0x7f9ad40a6ad0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009e50 tx=0x7f9acc00b330 comp rx=0 tx=0).stop 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.883+0000 7f9adb041700 1 -- 172.21.15.31:0/917365312 shutdown_connections 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.883+0000 7f9adb041700 1 --2- 172.21.15.31:0/917365312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad40c2870 0x7f9ad40a6ad0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.883+0000 7f9adb041700 1 --2- 172.21.15.31:0/917365312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad40c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.883+0000 7f9adb041700 1 -- 172.21.15.31:0/917365312 >> 172.21.15.31:0/917365312 conn(0x7f9ad406ab20 msgr2=0x7f9ad406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.883+0000 7f9adb041700 1 -- 172.21.15.31:0/917365312 shutdown_connections 2024-03-30T01:43:19.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.883+0000 7f9adb041700 1 -- 172.21.15.31:0/917365312 wait complete. 2024-03-30T01:43:19.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.884+0000 7f9adb041700 1 Processor -- start 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.884+0000 7f9adb041700 1 -- start start 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9adb041700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad40c2870 0x7f9ad4153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9adb041700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad40c5df0 0x7f9ad414e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9adb041700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9adb041700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad414eb50 con 0x7f9ad40c5df0 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9adb041700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40a8cd0 con 0x7f9ad40c2870 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9adb041700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad41547e0 con 0x7f9ad40c8310 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9ad3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad40c5df0 0x7f9ad414e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9ad3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad40c5df0 0x7f9ad414e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54512/0 (socket says 172.21.15.31:54512) 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9ad3fff700 1 -- 172.21.15.31:0/1507984515 learned_addr learned my addr 172.21.15.31:0/1507984515 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9ad95de700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.885+0000 7f9ad8ddd700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad40c2870 0x7f9ad4153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:19.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9ad3fff700 1 -- 172.21.15.31:0/1507984515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 msgr2=0x7f9ad414e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9ad3fff700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad414e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9ad3fff700 1 -- 172.21.15.31:0/1507984515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad40c2870 msgr2=0x7f9ad4153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9ad3fff700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad40c2870 0x7f9ad4153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9ad3fff700 1 -- 172.21.15.31:0/1507984515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac40094a0 con 0x7f9ad40c5df0 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9ad3fff700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad40c5df0 0x7f9ad414e0d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800c750 tx=0x7f9ac800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9ad17fa700 1 -- 172.21.15.31:0/1507984515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8004c80 con 0x7f9ad40c5df0 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9ad17fa700 1 -- 172.21.15.31:0/1507984515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac8004e00 con 0x7f9ad40c5df0 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc009b30 con 0x7f9ad40c5df0 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9ad17fa700 1 -- 172.21.15.31:0/1507984515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8007310 con 0x7f9ad40c5df0 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.886+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad41e9f00 con 0x7f9ad40c5df0 2024-03-30T01:43:19.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.887+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad4073760 con 0x7f9ad40c5df0 2024-03-30T01:43:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.888+0000 7f9ad17fa700 1 -- 172.21.15.31:0/1507984515 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9ac8007470 con 0x7f9ad40c5df0 2024-03-30T01:43:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.888+0000 7f9ad17fa700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ac006ac90 0x7f9ac006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.888+0000 7f9ad17fa700 1 -- 172.21.15.31:0/1507984515 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9ac8089010 con 0x7f9ad40c5df0 2024-03-30T01:43:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.888+0000 7f9ad8ddd700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ac006ac90 0x7f9ac006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:19.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.889+0000 7f9ad8ddd700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ac006ac90 0x7f9ac006d150 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009cd0 tx=0x7f9ac4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:19.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:19.893+0000 7f9ad17fa700 1 -- 172.21.15.31:0/1507984515 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9ac8054140 con 0x7f9ad40c5df0 2024-03-30T01:43:20.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.158+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ad404ea80 con 0x7f9ad40c5df0 2024-03-30T01:43:20.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.158+0000 7f9ad17fa700 1 -- 172.21.15.31:0/1507984515 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9ac8057760 con 0x7f9ad40c5df0 2024-03-30T01:43:20.165 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T01:43:20.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ac006ac90 msgr2=0x7f9ac006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ac006ac90 0x7f9ac006d150 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009cd0 tx=0x7f9ac4009210 comp rx=0 tx=0).stop 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad40c5df0 msgr2=0x7f9ad414e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad40c5df0 0x7f9ad414e0d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800c750 tx=0x7f9ac800ca60 comp rx=0 tx=0).stop 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 shutdown_connections 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ac006ac90 0x7f9ac006d150 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ad40c8310 0x7f9ad414e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ad40c2870 0x7f9ad4153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 --2- 172.21.15.31:0/1507984515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad40c5df0 0x7f9ad414e0d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 >> 172.21.15.31:0/1507984515 conn(0x7f9ad406ab20 msgr2=0x7f9ad41540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 shutdown_connections 2024-03-30T01:43:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:20.161+0000 7f9adb041700 1 -- 172.21.15.31:0/1507984515 wait complete. 2024-03-30T01:43:20.181 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T01:43:20.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:43:20.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-03-30T01:43:20.181 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T01:43:20.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T01:43:20.182 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T01:43:20.182 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-30T01:43:20.182 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T01:43:20.182 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:43:20.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:20.182 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-03-30T01:43:20.205 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:43:20.206 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42141sec preferred_lft 42141sec 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether e6:5a:8b:06:76:44 brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::5055:d3ff:fee5:dc2a/64 scope link 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:20.207 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T01:43:20.208 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether e6:5a:8b:06:76:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T01:43:20.208 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::e45a:8bff:fe06:7644/64 scope link 2024-03-30T01:43:20.208 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:20.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:20.208 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:43:20.282 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-03-30T01:43:20.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:20.284 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-03-30T01:43:20.349 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-03-30T01:43:20.349 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:43:20.349 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:43:20.349 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:43:20.349 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:43:20.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:20.349 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2024-03-30T01:43:20.350 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-03-30T01:43:20.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:43:20.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:43:20.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:43:20.350 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-03-30T01:43:20.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:20.474 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-03-30T01:43:20.495 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T01:43:20.496 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:43:20.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:20.496 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-03-30T01:43:20.571 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:43:20.571 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T01:43:20.632 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:43:20.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:43:20.633 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:43:20.694 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-30T01:43:20.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:20.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:43:20.791 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:43:20.791 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-30T01:43:20.791 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:43:20.801 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:43:20.802 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-30T01:43:20.802 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:43:20.802 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:43:20.802 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:43:20.821 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-30T01:43:20.821 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-30T01:43:20.821 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:43:20.821 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:43:20.821 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:43:20.821 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790" 2024-03-30T01:43:20.821 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790:/ 2024-03-30T01:43:20.821 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9739890c-57ec-4503-9fd1-d4b0145023f4" will pass to kernel 2024-03-30T01:43:20.823 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T01:43:20.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:20.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:43:20.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:20.854 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:43:20.937 INFO:teuthology.orchestra.run.smithi059.stdout:client5947 2024-03-30T01:43:20.938 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:43:20.938 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:43:21.000 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth rm client.testuser 2024-03-30T01:43:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.139+0000 7f9cc8e44700 1 Processor -- start 2024-03-30T01:43:21.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.139+0000 7f9cc8e44700 1 -- start start 2024-03-30T01:43:21.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.140+0000 7f9cc8e44700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9cc4151d20 0x7f9cc40705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.140+0000 7f9cc8e44700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc4152100 0x7f9cc40c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.140+0000 7f9cc8e44700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc4070ae0 con 0x7f9cc40c5dc0 2024-03-30T01:43:21.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.140+0000 7f9cc8e44700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc4070d60 con 0x7f9cc4151d20 2024-03-30T01:43:21.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.140+0000 7f9cc8e44700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc4157e90 con 0x7f9cc4152100 2024-03-30T01:43:21.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc1d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc4152100 0x7f9cc40c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9cc4151d20 0x7f9cc40705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc1d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc4152100 0x7f9cc40c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:48740/0 (socket says 172.21.15.31:48740) 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc1d9b700 1 -- 172.21.15.31:0/3286185280 learned_addr learned my addr 172.21.15.31:0/3286185280 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc2d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9cc40c5dc0 0x7f9cc4154880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:45770/0 (socket says 172.21.15.31:45770) 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc159a700 1 -- 172.21.15.31:0/3286185280 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143852394 0 0) 0x7f9cc4070ae0 con 0x7f9cc40c5dc0 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc1d9b700 1 -- 172.21.15.31:0/3286185280 >> v2:172.21.15.59:3300/0 conn(0x7f9cc4151d20 msgr2=0x7f9cc40705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc1d9b700 1 --2- 172.21.15.31:0/3286185280 >> v2:172.21.15.59:3300/0 conn(0x7f9cc4151d20 0x7f9cc40705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc1d9b700 1 -- 172.21.15.31:0/3286185280 >> v1:172.21.15.59:6789/0 conn(0x7f9cc40c5dc0 legacy=0x7f9cc4154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc1d9b700 1 -- 172.21.15.31:0/3286185280 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4158170 con 0x7f9cc4152100 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc259c700 1 --2- 172.21.15.31:0/3286185280 >> v2:172.21.15.59:3300/0 conn(0x7f9cc4151d20 0x7f9cc40705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.141+0000 7f9cc1d9b700 1 --2- 172.21.15.31:0/3286185280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc4152100 0x7f9cc40c4850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098f0 tx=0x7f9cb8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89bec8f871ff9a2b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc159a700 1 -- 172.21.15.31:0/3286185280 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8011830 con 0x7f9cc4152100 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc159a700 1 -- 172.21.15.31:0/3286185280 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb8011990 con 0x7f9cc4152100 2024-03-30T01:43:21.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc159a700 1 -- 172.21.15.31:0/3286185280 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8011d20 con 0x7f9cc4152100 2024-03-30T01:43:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc8e44700 1 -- 172.21.15.31:0/3286185280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc4152100 msgr2=0x7f9cc40c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc8e44700 1 --2- 172.21.15.31:0/3286185280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc4152100 0x7f9cc40c4850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098f0 tx=0x7f9cb8009c00 comp rx=0 tx=0).stop 2024-03-30T01:43:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc8e44700 1 -- 172.21.15.31:0/3286185280 shutdown_connections 2024-03-30T01:43:21.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc8e44700 1 --2- 172.21.15.31:0/3286185280 >> v2:172.21.15.59:3300/0 conn(0x7f9cc4151d20 0x7f9cc40705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc8e44700 1 --2- 172.21.15.31:0/3286185280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc4152100 0x7f9cc40c4850 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc8e44700 1 -- 172.21.15.31:0/3286185280 >> 172.21.15.31:0/3286185280 conn(0x7f9cc406ad30 msgr2=0x7f9cc40a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc8e44700 1 -- 172.21.15.31:0/3286185280 shutdown_connections 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.142+0000 7f9cc8e44700 1 -- 172.21.15.31:0/3286185280 wait complete. 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.144+0000 7f9cc8e44700 1 Processor -- start 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.144+0000 7f9cc8e44700 1 -- start start 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.144+0000 7f9cc8e44700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc40c5dc0 0x7f9cc41ed840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc8e44700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cc4151d20 0x7f9cc406d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc8e44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cc4152100 0x7f9cc414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc8e44700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc406f6e0 con 0x7f9cc4152100 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc8e44700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc41edd80 con 0x7f9cc4151d20 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc8e44700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc41524e0 con 0x7f9cc40c5dc0 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cc4152100 0x7f9cc414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cc4152100 0x7f9cc414e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54522/0 (socket says 172.21.15.31:54522) 2024-03-30T01:43:21.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc2d9d700 1 -- 172.21.15.31:0/2619310748 learned_addr learned my addr 172.21.15.31:0/2619310748 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc1d9b700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cc4151d20 0x7f9cc406d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc259c700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc40c5dc0 0x7f9cc41ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc2d9d700 1 -- 172.21.15.31:0/2619310748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc40c5dc0 msgr2=0x7f9cc41ed840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc2d9d700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc40c5dc0 0x7f9cc41ed840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc2d9d700 1 -- 172.21.15.31:0/2619310748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cc4151d20 msgr2=0x7f9cc406d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc2d9d700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cc4151d20 0x7f9cc406d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc2d9d700 1 -- 172.21.15.31:0/2619310748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb40094a0 con 0x7f9cc4152100 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc1d9b700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cc4151d20 0x7f9cc406d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc2d9d700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cc4152100 0x7f9cc414e1c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00cba0 tx=0x7f9cbc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cb2ffd700 1 -- 172.21.15.31:0/2619310748 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00ab80 con 0x7f9cc4152100 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cb2ffd700 1 -- 172.21.15.31:0/2619310748 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc00ad00 con 0x7f9cc4152100 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb8009550 con 0x7f9cc4152100 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cb2ffd700 1 -- 172.21.15.31:0/2619310748 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc007310 con 0x7f9cc4152100 2024-03-30T01:43:21.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.145+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc4152830 con 0x7f9cc4152100 2024-03-30T01:43:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.147+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc41e9930 con 0x7f9cc4152100 2024-03-30T01:43:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.147+0000 7f9cb2ffd700 1 -- 172.21.15.31:0/2619310748 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9cbc007470 con 0x7f9cc4152100 2024-03-30T01:43:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.147+0000 7f9cb2ffd700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ca406ac90 0x7f9ca406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.147+0000 7f9cb2ffd700 1 -- 172.21.15.31:0/2619310748 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9cbc088eb0 con 0x7f9cc4152100 2024-03-30T01:43:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.148+0000 7f9cc259c700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ca406ac90 0x7f9ca406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.148+0000 7f9cc259c700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ca406ac90 0x7f9ca406d150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009c20 tx=0x7f9cb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:21.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.152+0000 7f9cb2ffd700 1 -- 172.21.15.31:0/2619310748 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9cc41e9930 con 0x7f9cc4152100 2024-03-30T01:43:21.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.455+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.testuser"} v 0) v1 -- 0x7f9cc41e9930 con 0x7f9cc4152100 2024-03-30T01:43:21.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.455+0000 7f9cb2ffd700 1 -- 172.21.15.31:0/2619310748 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.testuser"}]=0 v17) v1 ==== 84+0+0 (secure 0 0 0) 0x7f9cc41e9930 con 0x7f9cc4152100 2024-03-30T01:43:21.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ca406ac90 msgr2=0x7f9ca406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ca406ac90 0x7f9ca406d150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009c20 tx=0x7f9cb4008040 comp rx=0 tx=0).stop 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cc4152100 msgr2=0x7f9cc414e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cc4152100 0x7f9cc414e1c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00cba0 tx=0x7f9cbc00cf60 comp rx=0 tx=0).stop 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 shutdown_connections 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cc4152100 0x7f9cc414e1c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f9ca406ac90 0x7f9ca406d150 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cc40c5dc0 0x7f9cc41ed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 --2- 172.21.15.31:0/2619310748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cc4151d20 0x7f9cc406d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 >> 172.21.15.31:0/2619310748 conn(0x7f9cc406ad30 msgr2=0x7f9cc40c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 shutdown_connections 2024-03-30T01:43:21.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.458+0000 7f9cc8e44700 1 -- 172.21.15.31:0/2619310748 wait complete. 2024-03-30T01:43:21.478 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:21.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.608+0000 7f63e357a700 1 Processor -- start 2024-03-30T01:43:21.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.608+0000 7f63e357a700 1 -- start start 2024-03-30T01:43:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e357a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63dc14fd30 0x7f63dc156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e357a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63dc150110 0x7f63dc0c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e357a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc0c88c0 0x7f63dc0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e357a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63dc0c4d60 con 0x7f63dc14fd30 2024-03-30T01:43:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e357a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63dc0c5040 con 0x7f63dc150110 2024-03-30T01:43:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e357a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc0c5320 con 0x7f63dc0c88c0 2024-03-30T01:43:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1316700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63dc14fd30 0x7f63dc156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1316700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63dc14fd30 0x7f63dc156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54538/0 (socket says 172.21.15.31:54538) 2024-03-30T01:43:21.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1316700 1 -- 172.21.15.31:0/2108957057 learned_addr learned my addr 172.21.15.31:0/2108957057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e0b15700 1 --2- 172.21.15.31:0/2108957057 >> v2:172.21.15.59:3300/0 conn(0x7f63dc150110 0x7f63dc0c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1b17700 1 --2- 172.21.15.31:0/2108957057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc0c88c0 0x7f63dc0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1316700 1 -- 172.21.15.31:0/2108957057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc0c88c0 msgr2=0x7f63dc0ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1316700 1 --2- 172.21.15.31:0/2108957057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc0c88c0 0x7f63dc0ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1316700 1 -- 172.21.15.31:0/2108957057 >> v2:172.21.15.59:3300/0 conn(0x7f63dc150110 msgr2=0x7f63dc0c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1316700 1 --2- 172.21.15.31:0/2108957057 >> v2:172.21.15.59:3300/0 conn(0x7f63dc150110 0x7f63dc0c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1316700 1 -- 172.21.15.31:0/2108957057 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc0c5600 con 0x7f63dc14fd30 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e0b15700 1 --2- 172.21.15.31:0/2108957057 >> v2:172.21.15.59:3300/0 conn(0x7f63dc150110 0x7f63dc0c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1b17700 1 --2- 172.21.15.31:0/2108957057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc0c88c0 0x7f63dc0ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.609+0000 7f63e1316700 1 --2- 172.21.15.31:0/2108957057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc14fd30 0x7f63dc156800 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0aac20 tx=0x7f63d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64965419007ec9a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63d3fff700 1 -- 172.21.15.31:0/2108957057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d800b870 con 0x7f63dc14fd30 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63d3fff700 1 -- 172.21.15.31:0/2108957057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d800b9d0 con 0x7f63dc14fd30 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63d3fff700 1 -- 172.21.15.31:0/2108957057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d800bd60 con 0x7f63dc14fd30 2024-03-30T01:43:21.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63e357a700 1 -- 172.21.15.31:0/2108957057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc14fd30 msgr2=0x7f63dc156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63e357a700 1 --2- 172.21.15.31:0/2108957057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc14fd30 0x7f63dc156800 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0aac20 tx=0x7f63d8009bd0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63e357a700 1 -- 172.21.15.31:0/2108957057 shutdown_connections 2024-03-30T01:43:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63e357a700 1 --2- 172.21.15.31:0/2108957057 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc0c88c0 0x7f63dc0ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63e357a700 1 --2- 172.21.15.31:0/2108957057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc14fd30 0x7f63dc156800 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63e357a700 1 --2- 172.21.15.31:0/2108957057 >> v2:172.21.15.59:3300/0 conn(0x7f63dc150110 0x7f63dc0c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63e357a700 1 -- 172.21.15.31:0/2108957057 >> 172.21.15.31:0/2108957057 conn(0x7f63dc06ab20 msgr2=0x7f63dc06af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63e357a700 1 -- 172.21.15.31:0/2108957057 shutdown_connections 2024-03-30T01:43:21.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.610+0000 7f63e357a700 1 -- 172.21.15.31:0/2108957057 wait complete. 2024-03-30T01:43:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e357a700 1 Processor -- start 2024-03-30T01:43:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e357a700 1 -- start start 2024-03-30T01:43:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e357a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc0c88c0 0x7f63dc14cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e357a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63dc14fd30 0x7f63dc1e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e357a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc150110 0x7f63dc14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e357a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1dede0 con 0x7f63dc0c88c0 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e357a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1df190 con 0x7f63dc14fd30 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e357a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc1eb600 con 0x7f63dc150110 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e1316700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc0c88c0 0x7f63dc14cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e1316700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc0c88c0 0x7f63dc14cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54548/0 (socket says 172.21.15.31:54548) 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.612+0000 7f63e1316700 1 -- 172.21.15.31:0/2086279387 learned_addr learned my addr 172.21.15.31:0/2086279387 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63e0b15700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63dc14fd30 0x7f63dc1e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63e1b17700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc150110 0x7f63dc14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63e1316700 1 -- 172.21.15.31:0/2086279387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc150110 msgr2=0x7f63dc14ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63e1316700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc150110 0x7f63dc14ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63e1316700 1 -- 172.21.15.31:0/2086279387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63dc14fd30 msgr2=0x7f63dc1e8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63e1316700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63dc14fd30 0x7f63dc1e8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63e1316700 1 -- 172.21.15.31:0/2086279387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8009550 con 0x7f63dc0c88c0 2024-03-30T01:43:21.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63e1316700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc0c88c0 0x7f63dc14cb20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63d1ffb700 1 -- 172.21.15.31:0/2086279387 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8004480 con 0x7f63dc0c88c0 2024-03-30T01:43:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63d1ffb700 1 -- 172.21.15.31:0/2086279387 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d80045e0 con 0x7f63dc0c88c0 2024-03-30T01:43:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63d1ffb700 1 -- 172.21.15.31:0/2086279387 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80038e0 con 0x7f63dc0c88c0 2024-03-30T01:43:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.613+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63dc1df680 con 0x7f63dc0c88c0 2024-03-30T01:43:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.614+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63dc1eafb0 con 0x7f63dc0c88c0 2024-03-30T01:43:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.614+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63dc073760 con 0x7f63dc0c88c0 2024-03-30T01:43:21.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.615+0000 7f63d1ffb700 1 -- 172.21.15.31:0/2086279387 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f63d8021070 con 0x7f63dc0c88c0 2024-03-30T01:43:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.616+0000 7f63d1ffb700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f63c006ac90 0x7f63c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.616+0000 7f63e0b15700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f63c006ac90 0x7f63c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.616+0000 7f63d1ffb700 1 -- 172.21.15.31:0/2086279387 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f63d808a930 con 0x7f63dc0c88c0 2024-03-30T01:43:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.616+0000 7f63e0b15700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f63c006ac90 0x7f63c006d150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f63dc154bf0 tx=0x7f63cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:21.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.620+0000 7f63d1ffb700 1 -- 172.21.15.31:0/2086279387 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f63d8055b80 con 0x7f63dc0c88c0 2024-03-30T01:43:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.836+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63dc04ea80 con 0x7f63dc0c88c0 2024-03-30T01:43:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.837+0000 7f63d1ffb700 1 -- 172.21.15.31:0/2086279387 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16200 (secure 0 0 0) 0x7f63d8015020 con 0x7f63dc0c88c0 2024-03-30T01:43:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:21.844 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":60,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:13.247511+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:43:09.911033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T01:43:11.216248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f63c006ac90 msgr2=0x7f63c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f63c006ac90 0x7f63c006d150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f63dc154bf0 tx=0x7f63cc008040 comp rx=0 tx=0).stop 2024-03-30T01:43:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc0c88c0 msgr2=0x7f63dc14cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc0c88c0 0x7f63dc14cb20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d8004750 comp rx=0 tx=0).stop 2024-03-30T01:43:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 shutdown_connections 2024-03-30T01:43:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63dc0c88c0 0x7f63dc14cb20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f63c006ac90 0x7f63c006d150 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f63dc150110 0x7f63dc14ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 --2- 172.21.15.31:0/2086279387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63dc14fd30 0x7f63dc1e8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.839+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 >> 172.21.15.31:0/2086279387 conn(0x7f63dc06ab20 msgr2=0x7f63dc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.840+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 shutdown_connections 2024-03-30T01:43:21.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.840+0000 7f63e357a700 1 -- 172.21.15.31:0/2086279387 wait complete. 2024-03-30T01:43:21.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:43:09.911033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:43:11.216248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:43:21.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:43:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.994+0000 7f6a2250f700 1 Processor -- start 2024-03-30T01:43:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.994+0000 7f6a2250f700 1 -- start start 2024-03-30T01:43:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.995+0000 7f6a2250f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a1c1549a0 0x7f6a1c072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.995+0000 7f6a2250f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1c14ee60 con 0x7f6a1c06d900 2024-03-30T01:43:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.995+0000 7f6a2250f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1c0c2290 con 0x7f6a1c06d040 2024-03-30T01:43:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.995+0000 7f6a2250f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c155490 con 0x7f6a1c1549a0 2024-03-30T01:43:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.995+0000 7f6a20aac700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6a1c06d900 0x7f6a1c06dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58866/0 (socket says 172.21.15.31:58866) 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.995+0000 7f6a20aac700 1 -- 172.21.15.31:0/2858640588 learned_addr learned my addr 172.21.15.31:0/2858640588 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.995+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232836099 0 0) 0x7f6a1c14ee60 con 0x7f6a1c06d900 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.995+0000 7f6a1bfff700 1 --2- 172.21.15.31:0/2858640588 >> v2:172.21.15.59:3300/0 conn(0x7f6a1c1549a0 0x7f6a1c072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.995+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a080035b0 con 0x7f6a1c06d900 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665039771 0 0) 0x7f6a080035b0 con 0x7f6a1c06d900 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 >> v2:172.21.15.59:3300/0 conn(0x7f6a1c1549a0 msgr2=0x7f6a1c072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 --2- 172.21.15.31:0/2858640588 >> v2:172.21.15.59:3300/0 conn(0x7f6a1c1549a0 0x7f6a1c072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 >> v1:172.21.15.59:6789/0 conn(0x7f6a1c06d040 legacy=0x7f6a1c154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c155720 con 0x7f6a1c06d900 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f6a140019f0 con 0x7f6a1c06d900 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 >> v1:172.21.15.31:6789/0 conn(0x7f6a1c06d900 legacy=0x7f6a1c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a08003ae0 0x7f6a08003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a08004850 0x7f6a08004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a080052e0 0x7f6a0800dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c155490 con 0x7f6a08003ae0 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0800e440 con 0x7f6a080052e0 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0800e600 con 0x7f6a08004850 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1bfff700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a08003ae0 0x7f6a08003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a20aac700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a080052e0 0x7f6a0800dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1b7fe700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a08004850 0x7f6a08004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1bfff700 1 -- 172.21.15.31:0/2858640588 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a08004850 msgr2=0x7f6a08004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1bfff700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a08004850 0x7f6a08004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1bfff700 1 -- 172.21.15.31:0/2858640588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a080052e0 msgr2=0x7f6a0800dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1bfff700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a080052e0 0x7f6a0800dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1bfff700 1 -- 172.21.15.31:0/2858640588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a080035b0 con 0x7f6a08003ae0 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a20aac700 1 -- 172.21.15.31:0/2858640588 reap_dead start 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.996+0000 7f6a1bfff700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a08003ae0 0x7f6a08003f00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009b60 tx=0x7f6a08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac27c4f81d76a3d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00e6c0 con 0x7f6a08003ae0 2024-03-30T01:43:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c00e820 con 0x7f6a08003ae0 2024-03-30T01:43:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a1affd700 1 -- 172.21.15.31:0/2858640588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00eb90 con 0x7f6a08003ae0 2024-03-30T01:43:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a2250f700 1 -- 172.21.15.31:0/2858640588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a08003ae0 msgr2=0x7f6a08003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a2250f700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a08003ae0 0x7f6a08003f00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009b60 tx=0x7f6a08005880 comp rx=0 tx=0).stop 2024-03-30T01:43:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a2250f700 1 -- 172.21.15.31:0/2858640588 shutdown_connections 2024-03-30T01:43:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a2250f700 1 --2- 172.21.15.31:0/2858640588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a08003ae0 0x7f6a08003f00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a2250f700 1 -- 172.21.15.31:0/2858640588 >> 172.21.15.31:0/2858640588 conn(0x7f6a1c0c3980 msgr2=0x7f6a1c0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a2250f700 1 -- 172.21.15.31:0/2858640588 shutdown_connections 2024-03-30T01:43:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.997+0000 7f6a2250f700 1 -- 172.21.15.31:0/2858640588 wait complete. 2024-03-30T01:43:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.998+0000 7f6a2250f700 1 Processor -- start 2024-03-30T01:43:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.999+0000 7f6a2250f700 1 -- start start 2024-03-30T01:43:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.999+0000 7f6a2250f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a08003ae0 0x7f6a1c147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.999+0000 7f6a2250f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a1c147fc0 0x7f6a1c0c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.999+0000 7f6a2250f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a1c145b80 0x7f6a1c143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.999+0000 7f6a2250f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c148700 con 0x7f6a1c145b80 2024-03-30T01:43:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.999+0000 7f6a2250f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c1eab20 con 0x7f6a08003ae0 2024-03-30T01:43:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.999+0000 7f6a2250f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c0abc60 con 0x7f6a1c147fc0 2024-03-30T01:43:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:21.999+0000 7f6a20aac700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a1c145b80 0x7f6a1c143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a20aac700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a1c145b80 0x7f6a1c143780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54560/0 (socket says 172.21.15.31:54560) 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a20aac700 1 -- 172.21.15.31:0/1053763049 learned_addr learned my addr 172.21.15.31:0/1053763049 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a1bfff700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a08003ae0 0x7f6a1c147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a1b7fe700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a1c147fc0 0x7f6a1c0c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a20aac700 1 -- 172.21.15.31:0/1053763049 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a1c147fc0 msgr2=0x7f6a1c0c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a20aac700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a1c147fc0 0x7f6a1c0c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a20aac700 1 -- 172.21.15.31:0/1053763049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a08003ae0 msgr2=0x7f6a1c147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a20aac700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a08003ae0 0x7f6a1c147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a20aac700 1 -- 172.21.15.31:0/1053763049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c0abf40 con 0x7f6a1c145b80 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a20aac700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a1c145b80 0x7f6a1c143780 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6a14002230 tx=0x7f6a1400e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a18ff9700 1 -- 172.21.15.31:0/1053763049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1400ec50 con 0x7f6a1c145b80 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a18ff9700 1 -- 172.21.15.31:0/1053763049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1400edb0 con 0x7f6a1c145b80 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c0ac220 con 0x7f6a1c145b80 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a18ff9700 1 -- 172.21.15.31:0/1053763049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14013660 con 0x7f6a1c145b80 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.000+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c1ea760 con 0x7f6a1c145b80 2024-03-30T01:43:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.002+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1c0747d0 con 0x7f6a1c145b80 2024-03-30T01:43:22.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.002+0000 7f6a18ff9700 1 -- 172.21.15.31:0/1053763049 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6a140137c0 con 0x7f6a1c145b80 2024-03-30T01:43:22.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.002+0000 7f6a18ff9700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69fc06ac90 0x7f69fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.002+0000 7f6a18ff9700 1 -- 172.21.15.31:0/1053763049 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6a14089980 con 0x7f6a1c145b80 2024-03-30T01:43:22.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.002+0000 7f6a1bfff700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69fc06ac90 0x7f69fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.003+0000 7f6a1bfff700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69fc06ac90 0x7f69fc06d150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0090b0 tx=0x7f6a0c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:22.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.007+0000 7f6a18ff9700 1 -- 172.21.15.31:0/1053763049 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6a14054ab0 con 0x7f6a1c145b80 2024-03-30T01:43:22.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.220+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a1c0c3500 con 0x7f6a1c145b80 2024-03-30T01:43:22.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.221+0000 7f6a18ff9700 1 -- 172.21.15.31:0/1053763049 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a140580d0 con 0x7f6a1c145b80 2024-03-30T01:43:22.227 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:43:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69fc06ac90 msgr2=0x7f69fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69fc06ac90 0x7f69fc06d150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0090b0 tx=0x7f6a0c008030 comp rx=0 tx=0).stop 2024-03-30T01:43:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a1c145b80 msgr2=0x7f6a1c143780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a1c145b80 0x7f6a1c143780 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6a14002230 tx=0x7f6a1400e440 comp rx=0 tx=0).stop 2024-03-30T01:43:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 shutdown_connections 2024-03-30T01:43:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a1c145b80 0x7f6a1c143780 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f69fc06ac90 0x7f69fc06d150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a08003ae0 0x7f6a1c147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 --2- 172.21.15.31:0/1053763049 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a1c147fc0 0x7f6a1c0c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 >> 172.21.15.31:0/1053763049 conn(0x7f6a1c0c3980 msgr2=0x7f6a1c14c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:22.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 shutdown_connections 2024-03-30T01:43:22.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.223+0000 7f6a2250f700 1 -- 172.21.15.31:0/1053763049 wait complete. 2024-03-30T01:43:22.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:43:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.376+0000 7ff17dd39700 1 Processor -- start 2024-03-30T01:43:22.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.377+0000 7ff17dd39700 1 -- start start 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.377+0000 7ff17dd39700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1781549a0 0x7ff178072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.377+0000 7ff17dd39700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff17806d900 0x7ff17806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.377+0000 7ff17dd39700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17814ee60 con 0x7ff17806d040 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.377+0000 7ff17dd39700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1780c2290 con 0x7ff17806d900 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.377+0000 7ff17dd39700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff178155490 con 0x7ff1781549a0 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.377+0000 7ff176ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff17806d040 0x7ff178154d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58874/0 (socket says 172.21.15.31:58874) 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.377+0000 7ff176ffd700 1 -- 172.21.15.31:0/3027421647 learned_addr learned my addr 172.21.15.31:0/3027421647 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff177fff700 1 --2- 172.21.15.31:0/3027421647 >> v2:172.21.15.59:3300/0 conn(0x7ff17806d900 0x7ff17806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff1777fe700 1 --2- 172.21.15.31:0/3027421647 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1781549a0 0x7ff178072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff1767fc700 1 -- 172.21.15.31:0/3027421647 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848554195 0 0) 0x7ff17814ee60 con 0x7ff17806d040 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff1767fc700 1 -- 172.21.15.31:0/3027421647 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1600035b0 con 0x7ff17806d040 2024-03-30T01:43:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff177fff700 1 -- 172.21.15.31:0/3027421647 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1781549a0 msgr2=0x7ff178072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff177fff700 1 --2- 172.21.15.31:0/3027421647 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1781549a0 0x7ff178072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff177fff700 1 -- 172.21.15.31:0/3027421647 >> v1:172.21.15.31:6789/0 conn(0x7ff17806d040 legacy=0x7ff178154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff177fff700 1 -- 172.21.15.31:0/3027421647 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17806e2d0 con 0x7ff17806d900 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff177fff700 1 --2- 172.21.15.31:0/3027421647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17806d900 0x7ff17806dd90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff170009d90 tx=0x7ff17000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884c0041ca1beec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff1767fc700 1 -- 172.21.15.31:0/3027421647 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17000e070 con 0x7ff17806d900 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff1767fc700 1 -- 172.21.15.31:0/3027421647 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17000bb20 con 0x7ff17806d900 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.378+0000 7ff1767fc700 1 -- 172.21.15.31:0/3027421647 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170012430 con 0x7ff17806d900 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.379+0000 7ff17dd39700 1 -- 172.21.15.31:0/3027421647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17806d900 msgr2=0x7ff17806dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.379+0000 7ff17dd39700 1 --2- 172.21.15.31:0/3027421647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17806d900 0x7ff17806dd90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff170009d90 tx=0x7ff17000b330 comp rx=0 tx=0).stop 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.379+0000 7ff17dd39700 1 -- 172.21.15.31:0/3027421647 shutdown_connections 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.379+0000 7ff17dd39700 1 --2- 172.21.15.31:0/3027421647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17806d900 0x7ff17806dd90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.379+0000 7ff17dd39700 1 --2- 172.21.15.31:0/3027421647 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1781549a0 0x7ff178072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.379+0000 7ff17dd39700 1 -- 172.21.15.31:0/3027421647 >> 172.21.15.31:0/3027421647 conn(0x7ff1780c3980 msgr2=0x7ff1780c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.379+0000 7ff17dd39700 1 -- 172.21.15.31:0/3027421647 shutdown_connections 2024-03-30T01:43:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.379+0000 7ff17dd39700 1 -- 172.21.15.31:0/3027421647 wait complete. 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.380+0000 7ff17dd39700 1 Processor -- start 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff17dd39700 1 -- start start 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff17dd39700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff17806d040 0x7ff178146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff17dd39700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff17806d900 0x7ff1781538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff17dd39700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1781549a0 0x7ff178147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff17dd39700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17806f5a0 con 0x7ff17806d040 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff17dd39700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1781dd180 con 0x7ff1781549a0 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff17dd39700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1781dd460 con 0x7ff17806d900 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff1777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff17806d040 0x7ff178146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff1777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff17806d040 0x7ff178146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54572/0 (socket says 172.21.15.31:54572) 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.381+0000 7ff1777fe700 1 -- 172.21.15.31:0/1578050841 learned_addr learned my addr 172.21.15.31:0/1578050841 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:22.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff177fff700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1781549a0 0x7ff178147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff1777fe700 1 -- 172.21.15.31:0/1578050841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff17806d900 msgr2=0x7ff1781538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff1777fe700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff17806d900 0x7ff1781538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff1777fe700 1 -- 172.21.15.31:0/1578050841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1781549a0 msgr2=0x7ff178147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff1777fe700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1781549a0 0x7ff178147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff1777fe700 1 -- 172.21.15.31:0/1578050841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff170009a70 con 0x7ff17806d040 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff1777fe700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff17806d040 0x7ff178146550 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff16800e870 tx=0x7ff16800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff15ffff700 1 -- 172.21.15.31:0/1578050841 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff168009590 con 0x7ff17806d040 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff15ffff700 1 -- 172.21.15.31:0/1578050841 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16800ee70 con 0x7ff17806d040 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1781dbcb0 con 0x7ff17806d040 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff15ffff700 1 -- 172.21.15.31:0/1578050841 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16800ca50 con 0x7ff17806d040 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.382+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1781dc270 con 0x7ff17806d040 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.383+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1781dc5e0 con 0x7ff17806d040 2024-03-30T01:43:22.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.383+0000 7ff15ffff700 1 -- 172.21.15.31:0/1578050841 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff16800cbb0 con 0x7ff17806d040 2024-03-30T01:43:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.384+0000 7ff15ffff700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff16406ac90 0x7ff16406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.384+0000 7ff15ffff700 1 -- 172.21.15.31:0/1578050841 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff168089480 con 0x7ff17806d040 2024-03-30T01:43:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.384+0000 7ff176ffd700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff16406ac90 0x7ff16406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.384+0000 7ff176ffd700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff16406ac90 0x7ff16406d150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff17806d460 tx=0x7ff16c0079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.389+0000 7ff15ffff700 1 -- 172.21.15.31:0/1578050841 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff168054520 con 0x7ff17806d040 2024-03-30T01:43:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.603+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1780c4870 con 0x7ff17806d040 2024-03-30T01:43:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.603+0000 7ff15ffff700 1 -- 172.21.15.31:0/1578050841 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff16800ce60 con 0x7ff17806d040 2024-03-30T01:43:22.610 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:43:22.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff16406ac90 msgr2=0x7ff16406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff16406ac90 0x7ff16406d150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff17806d460 tx=0x7ff16c0079c0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff17806d040 msgr2=0x7ff178146550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff17806d040 0x7ff178146550 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff16800e870 tx=0x7ff16800ec30 comp rx=0 tx=0).stop 2024-03-30T01:43:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 shutdown_connections 2024-03-30T01:43:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff17806d040 0x7ff178146550 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff16406ac90 0x7ff16406d150 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1781549a0 0x7ff178147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 --2- 172.21.15.31:0/1578050841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff17806d900 0x7ff1781538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 >> 172.21.15.31:0/1578050841 conn(0x7ff1780c3980 msgr2=0x7ff17814e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 shutdown_connections 2024-03-30T01:43:22.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.606+0000 7ff17dd39700 1 -- 172.21.15.31:0/1578050841 wait complete. 2024-03-30T01:43:22.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:43:22.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.759+0000 7f557b1e3700 1 Processor -- start 2024-03-30T01:43:22.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.759+0000 7f557b1e3700 1 -- start start 2024-03-30T01:43:22.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.759+0000 7f557b1e3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5574155490 0x7f557406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.759+0000 7f557b1e3700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5574155000 con 0x7f55740721f0 2024-03-30T01:43:22.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.759+0000 7f557b1e3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557406e650 con 0x7f5574151ea0 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.759+0000 7f557b1e3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557406e930 con 0x7f5574155490 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f5573fff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f55740721f0 0x7f557406a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58882/0 (socket says 172.21.15.31:58882) 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f5573fff700 1 -- 172.21.15.31:0/19776806 learned_addr learned my addr 172.21.15.31:0/19776806 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f5579780700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5574155490 0x7f557406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186116937 0 0) 0x7f5574155000 con 0x7f55740721f0 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55640035b0 con 0x7f55740721f0 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276306745 0 0) 0x7f557406e650 con 0x7f5574151ea0 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5574155000 con 0x7f5574151ea0 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538386016 0 0) 0x7f55640035b0 con 0x7f55740721f0 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5574155490 msgr2=0x7f557406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5574155490 0x7f557406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 >> v1:172.21.15.59:6789/0 conn(0x7f5574151ea0 legacy=0x7f55740c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55741549b0 con 0x7f55740721f0 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f5560001670 con 0x7f55740721f0 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f5579780700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5574155490 0x7f557406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 >> v1:172.21.15.31:6789/0 conn(0x7f55740721f0 legacy=0x7f557406a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5564003ca0 0x7f55640040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564004850 0x7f5564004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5574155490 0x7f55640052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557406e930 con 0x7f5564004850 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556400bc00 con 0x7f5564003ca0 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556400bdc0 con 0x7f5574155490 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f5573fff700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564004850 0x7f5564004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.760+0000 7f5578f7f700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5564003ca0 0x7f55640040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f5579780700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5574155490 0x7f55640052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f5573fff700 1 -- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5574155490 msgr2=0x7f55640052f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:22.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f5573fff700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5574155490 0x7f55640052f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f5573fff700 1 -- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5564003ca0 msgr2=0x7f55640040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f5573fff700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5564003ca0 0x7f55640040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f5573fff700 1 -- 172.21.15.31:0/19776806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c009a70 con 0x7f5564004850 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f5573fff700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564004850 0x7f5564004cd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5560001eb0 tx=0x7f5564005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba5486691d60df9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556000cd20 con 0x7f5564004850 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5560005960 con 0x7f5564004850 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.761+0000 7f55737fe700 1 -- 172.21.15.31:0/19776806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556000cea0 con 0x7f5564004850 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.762+0000 7f557b1e3700 1 -- 172.21.15.31:0/19776806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564004850 msgr2=0x7f5564004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.762+0000 7f557b1e3700 1 --2- 172.21.15.31:0/19776806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5564004850 0x7f5564004cd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5560001eb0 tx=0x7f5564005880 comp rx=0 tx=0).stop 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.762+0000 7f5579780700 1 -- 172.21.15.31:0/19776806 reap_dead start 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.762+0000 7f557b1e3700 1 -- 172.21.15.31:0/19776806 shutdown_connections 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.762+0000 7f557b1e3700 1 -- 172.21.15.31:0/19776806 >> 172.21.15.31:0/19776806 conn(0x7f55740c3980 msgr2=0x7f55740c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.762+0000 7f557b1e3700 1 -- 172.21.15.31:0/19776806 shutdown_connections 2024-03-30T01:43:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.762+0000 7f557b1e3700 1 -- 172.21.15.31:0/19776806 wait complete. 2024-03-30T01:43:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.763+0000 7f557b1e3700 1 Processor -- start 2024-03-30T01:43:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.763+0000 7f557b1e3700 1 -- start start 2024-03-30T01:43:22.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f557b1e3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5564004850 0x7f55741575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f557b1e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f557414d060 0x7f55740c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f557b1e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55741454b0 0x7f5574144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f557b1e3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5574149db0 con 0x7f55741454b0 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f557b1e3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5574157e60 con 0x7f557414d060 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f557b1e3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5574145990 con 0x7f5564004850 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5579780700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55741454b0 0x7f5574144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5579780700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55741454b0 0x7f5574144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54582/0 (socket says 172.21.15.31:54582) 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5579780700 1 -- 172.21.15.31:0/2296026744 learned_addr learned my addr 172.21.15.31:0/2296026744 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5573fff700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f557414d060 0x7f55740c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5578f7f700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5564004850 0x7f55741575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5579780700 1 -- 172.21.15.31:0/2296026744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5564004850 msgr2=0x7f55741575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5579780700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5564004850 0x7f55741575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5579780700 1 -- 172.21.15.31:0/2296026744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f557414d060 msgr2=0x7f55740c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5579780700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f557414d060 0x7f55740c1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.764+0000 7f5579780700 1 -- 172.21.15.31:0/2296026744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5574153800 con 0x7f55741454b0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.765+0000 7f5579780700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55741454b0 0x7f5574144470 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f557406d6a0 tx=0x7f556c00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.765+0000 7f55717fa700 1 -- 172.21.15.31:0/2296026744 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c01c070 con 0x7f55741454b0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.765+0000 7f55717fa700 1 -- 172.21.15.31:0/2296026744 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c00f040 con 0x7f55741454b0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.765+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55741451d0 con 0x7f55741454b0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.765+0000 7f55717fa700 1 -- 172.21.15.31:0/2296026744 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c00e040 con 0x7f55741454b0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.765+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5574153a70 con 0x7f55741454b0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.766+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5574149930 con 0x7f55741454b0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.767+0000 7f55717fa700 1 -- 172.21.15.31:0/2296026744 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f556c015430 con 0x7f55741454b0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.767+0000 7f55717fa700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f555806ac90 0x7f555806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.767+0000 7f55717fa700 1 -- 172.21.15.31:0/2296026744 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f556c089130 con 0x7f55741454b0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.767+0000 7f5578f7f700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f555806ac90 0x7f555806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:22.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.768+0000 7f5578f7f700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f555806ac90 0x7f555806d150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5568007fd0 tx=0x7f5568009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:22.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.772+0000 7f55717fa700 1 -- 172.21.15.31:0/2296026744 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5574149930 con 0x7f55741454b0 2024-03-30T01:43:22.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.985+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5574149930 con 0x7f55741454b0 2024-03-30T01:43:22.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.986+0000 7f55717fa700 1 -- 172.21.15.31:0/2296026744 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5574149930 con 0x7f55741454b0 2024-03-30T01:43:22.993 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:43:22.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.988+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f555806ac90 msgr2=0x7f555806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.988+0000 7f557b1e3700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f555806ac90 0x7f555806d150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5568007fd0 tx=0x7f5568009040 comp rx=0 tx=0).stop 2024-03-30T01:43:22.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.988+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55741454b0 msgr2=0x7f5574144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.988+0000 7f557b1e3700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55741454b0 0x7f5574144470 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f557406d6a0 tx=0x7f556c00f490 comp rx=0 tx=0).stop 2024-03-30T01:43:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.989+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 shutdown_connections 2024-03-30T01:43:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.989+0000 7f557b1e3700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55741454b0 0x7f5574144470 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.989+0000 7f557b1e3700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f555806ac90 0x7f555806d150 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.989+0000 7f557b1e3700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5564004850 0x7f55741575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.989+0000 7f557b1e3700 1 --2- 172.21.15.31:0/2296026744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f557414d060 0x7f55740c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.989+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 >> 172.21.15.31:0/2296026744 conn(0x7f55740c3980 msgr2=0x7f55740c2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.989+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 shutdown_connections 2024-03-30T01:43:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:22.989+0000 7f557b1e3700 1 -- 172.21.15.31:0/2296026744 wait complete. 2024-03-30T01:43:23.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T01:43:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.138+0000 7f7421177700 1 Processor -- start 2024-03-30T01:43:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.138+0000 7f7421177700 1 -- start start 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7421177700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0bd7f0 0x7f741c155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7421177700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c0c8d10 con 0x7f741c06b170 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7421177700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c0c98b0 con 0x7f741c154760 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7421177700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c06bef0 con 0x7f741c0bd7f0 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f741ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f741c06b170 0x7f741c0c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58886/0 (socket says 172.21.15.31:58886) 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f741ad9d700 1 -- 172.21.15.31:0/4131845898 learned_addr learned my addr 172.21.15.31:0/4131845898 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f741a59c700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0bd7f0 0x7f741c155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84322728 0 0) 0x7f741c0c8d10 con 0x7f741c06b170 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74040035b0 con 0x7f741c06b170 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1431941907 0 0) 0x7f74040035b0 con 0x7f741c06b170 2024-03-30T01:43:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0bd7f0 msgr2=0x7f741c155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0bd7f0 0x7f741c155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 >> v1:172.21.15.59:6789/0 conn(0x7f741c154760 legacy=0x7f741c15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0c8100 con 0x7f741c06b170 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f740c001670 con 0x7f741c06b170 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 >> v1:172.21.15.31:6789/0 conn(0x7f741c06b170 legacy=0x7f741c0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.139+0000 7f7419d9b700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404003a30 0x7f7404003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f7419d9b700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f7404004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f7419d9b700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0bd7f0 0x7f7404005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740400bb10 con 0x7f7404004850 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740400bc80 con 0x7f7404003a30 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740400be40 con 0x7f741c0bd7f0 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741a59c700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f7404004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741ad9d700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404003a30 0x7f7404003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741b59e700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0bd7f0 0x7f7404005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741a59c700 1 -- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0bd7f0 msgr2=0x7f7404005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741a59c700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0bd7f0 0x7f7404005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741a59c700 1 -- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404003a30 msgr2=0x7f7404003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741a59c700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404003a30 0x7f7404003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741a59c700 1 -- 172.21.15.31:0/4131845898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74040035b0 con 0x7f7404004850 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741b59e700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0bd7f0 0x7f7404005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f741a59c700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f7404004cd0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f741c08b840 tx=0x7f7404005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c96f0a200c5da66 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741000e6c0 con 0x7f7404004850 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741000e820 con 0x7f7404004850 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.140+0000 7f7419d9b700 1 -- 172.21.15.31:0/4131845898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741000eb90 con 0x7f7404004850 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.141+0000 7f7421177700 1 -- 172.21.15.31:0/4131845898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 msgr2=0x7f7404004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.141+0000 7f7421177700 1 --2- 172.21.15.31:0/4131845898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f7404004cd0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f741c08b840 tx=0x7f7404005880 comp rx=0 tx=0).stop 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.141+0000 7f741b59e700 1 -- 172.21.15.31:0/4131845898 reap_dead start 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.141+0000 7f7421177700 1 -- 172.21.15.31:0/4131845898 shutdown_connections 2024-03-30T01:43:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.141+0000 7f7421177700 1 -- 172.21.15.31:0/4131845898 >> 172.21.15.31:0/4131845898 conn(0x7f741c0ca0d0 msgr2=0x7f741c0ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.141+0000 7f7421177700 1 -- 172.21.15.31:0/4131845898 shutdown_connections 2024-03-30T01:43:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.141+0000 7f7421177700 1 -- 172.21.15.31:0/4131845898 wait complete. 2024-03-30T01:43:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f7421177700 1 Processor -- start 2024-03-30T01:43:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f7421177700 1 -- start start 2024-03-30T01:43:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f7421177700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f741c156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f7421177700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f741c0c1ae0 0x7f741c1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f7421177700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0c1f60 0x7f741c0bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f7421177700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c1eda60 con 0x7f7404004850 2024-03-30T01:43:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f7421177700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c1edd40 con 0x7f741c0c1ae0 2024-03-30T01:43:23.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f7421177700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0bdc00 con 0x7f741c0c1f60 2024-03-30T01:43:23.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f741ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f741c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f741ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f741c156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54602/0 (socket says 172.21.15.31:54602) 2024-03-30T01:43:23.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.143+0000 7f741ad9d700 1 -- 172.21.15.31:0/1517965679 learned_addr learned my addr 172.21.15.31:0/1517965679 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:23.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f741b59e700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0c1f60 0x7f741c0bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f741a59c700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f741c0c1ae0 0x7f741c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f741ad9d700 1 -- 172.21.15.31:0/1517965679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0c1f60 msgr2=0x7f741c0bfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f741ad9d700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0c1f60 0x7f741c0bfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f741ad9d700 1 -- 172.21.15.31:0/1517965679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f741c0c1ae0 msgr2=0x7f741c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f741ad9d700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f741c0c1ae0 0x7f741c1502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f741ad9d700 1 -- 172.21.15.31:0/1517965679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0bdee0 con 0x7f7404004850 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f741ad9d700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f741c156740 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f740c007290 tx=0x7f740c0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f74037fe700 1 -- 172.21.15.31:0/1517965679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00f6c0 con 0x7f7404004850 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f74037fe700 1 -- 172.21.15.31:0/1517965679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c00f860 con 0x7f7404004850 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c0be1c0 con 0x7f7404004850 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f74037fe700 1 -- 172.21.15.31:0/1517965679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00fc10 con 0x7f7404004850 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.144+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c156f50 con 0x7f7404004850 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.145+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741c1edf00 con 0x7f7404004850 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.146+0000 7f74037fe700 1 -- 172.21.15.31:0/1517965679 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f740c00fd70 con 0x7f7404004850 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.146+0000 7f74037fe700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f740806ac90 0x7f740806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.146+0000 7f74037fe700 1 -- 172.21.15.31:0/1517965679 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f740c089330 con 0x7f7404004850 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.146+0000 7f741a59c700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f740806ac90 0x7f740806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.147+0000 7f741a59c700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f740806ac90 0x7f740806d150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f74100090b0 tx=0x7f7410008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:23.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.151+0000 7f74037fe700 1 -- 172.21.15.31:0/1517965679 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f740c054460 con 0x7f7404004850 2024-03-30T01:43:23.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.366+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f741c0be3b0 con 0x7f7404004850 2024-03-30T01:43:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.366+0000 7f74037fe700 1 -- 172.21.15.31:0/1517965679 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+208 (secure 0 0 0) 0x7f740c057a80 con 0x7f7404004850 2024-03-30T01:43:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:23.373 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-03-30T01:43:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-03-30T01:43:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-03-30T01:43:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:23.373 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 9, 2024-03-30T01:43:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "cephfs_metadata" 2024-03-30T01:43:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-03-30T01:43:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-03-30T01:43:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 10, 2024-03-30T01:43:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": "cephfs_data" 2024-03-30T01:43:23.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-03-30T01:43:23.374 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-03-30T01:43:23.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.368+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f740806ac90 msgr2=0x7f740806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.368+0000 7f7421177700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f740806ac90 0x7f740806d150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f74100090b0 tx=0x7f7410008030 comp rx=0 tx=0).stop 2024-03-30T01:43:23.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.368+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 msgr2=0x7f741c156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.368+0000 7f7421177700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f741c156740 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f740c007290 tx=0x7f740c0075a0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.368+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 shutdown_connections 2024-03-30T01:43:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.368+0000 7f7421177700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7404004850 0x7f741c156740 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.368+0000 7f7421177700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f740806ac90 0x7f740806d150 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.368+0000 7f7421177700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f741c0c1f60 0x7f741c0bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.368+0000 7f7421177700 1 --2- 172.21.15.31:0/1517965679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f741c0c1ae0 0x7f741c1502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.369+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 >> 172.21.15.31:0/1517965679 conn(0x7f741c0ca0d0 msgr2=0x7f741c06b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.369+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 shutdown_connections 2024-03-30T01:43:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.369+0000 7f7421177700 1 -- 172.21.15.31:0/1517965679 wait complete. 2024-03-30T01:43:23.389 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-30T01:43:23.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-30T01:43:23.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.523+0000 7f73311ae700 1 Processor -- start 2024-03-30T01:43:23.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f73311ae700 1 -- start start 2024-03-30T01:43:23.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f73311ae700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c0ad6e0 0x7f732c0c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f73311ae700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f732c0adc30 0x7f732c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f73311ae700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c156c10 con 0x7f732c156830 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f73311ae700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c0aae80 con 0x7f732c0adc30 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f73311ae700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0c29f0 con 0x7f732c0ad6e0 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f732b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f732c0adc30 0x7f732c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f732ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f732c156830 0x7f732c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58894/0 (socket says 172.21.15.31:58894) 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f732ad9d700 1 -- 172.21.15.31:0/1427172969 learned_addr learned my addr 172.21.15.31:0/1427172969 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.524+0000 7f732b59e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f732c0adc30 0x7f732c0c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54610/0 (socket says 172.21.15.31:54610) 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f732a59c700 1 --2- 172.21.15.31:0/1427172969 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c0ad6e0 0x7f732c0c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f732b59e700 1 -- 172.21.15.31:0/1427172969 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c0ad6e0 msgr2=0x7f732c0c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f732b59e700 1 --2- 172.21.15.31:0/1427172969 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c0ad6e0 0x7f732c0c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f732b59e700 1 -- 172.21.15.31:0/1427172969 >> v1:172.21.15.31:6789/0 conn(0x7f732c156830 legacy=0x7f732c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f732b59e700 1 -- 172.21.15.31:0/1427172969 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0c3ff0 con 0x7f732c0adc30 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f7329d9b700 1 -- 172.21.15.31:0/1427172969 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104982852 0 0) 0x7f732c156c10 con 0x7f732c156830 2024-03-30T01:43:23.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f732b59e700 1 --2- 172.21.15.31:0/1427172969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 0x7f732c0c3660 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7324009d90 tx=0x7f732400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3c42cffd9c73e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f7329d9b700 1 -- 172.21.15.31:0/1427172969 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732400e070 con 0x7f732c0adc30 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f7329d9b700 1 -- 172.21.15.31:0/1427172969 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732400bbd0 con 0x7f732c0adc30 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.525+0000 7f7329d9b700 1 -- 172.21.15.31:0/1427172969 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324012430 con 0x7f732c0adc30 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.526+0000 7f73311ae700 1 -- 172.21.15.31:0/1427172969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 msgr2=0x7f732c0c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.526+0000 7f73311ae700 1 --2- 172.21.15.31:0/1427172969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 0x7f732c0c3660 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7324009d90 tx=0x7f732400b330 comp rx=0 tx=0).stop 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.526+0000 7f73311ae700 1 -- 172.21.15.31:0/1427172969 shutdown_connections 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.526+0000 7f73311ae700 1 --2- 172.21.15.31:0/1427172969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 0x7f732c0c3660 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.526+0000 7f73311ae700 1 --2- 172.21.15.31:0/1427172969 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c0ad6e0 0x7f732c0c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.526+0000 7f73311ae700 1 -- 172.21.15.31:0/1427172969 >> 172.21.15.31:0/1427172969 conn(0x7f732c0c48d0 msgr2=0x7f732c0c4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.526+0000 7f73311ae700 1 -- 172.21.15.31:0/1427172969 shutdown_connections 2024-03-30T01:43:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.526+0000 7f73311ae700 1 -- 172.21.15.31:0/1427172969 wait complete. 2024-03-30T01:43:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.527+0000 7f73311ae700 1 Processor -- start 2024-03-30T01:43:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f73311ae700 1 -- start start 2024-03-30T01:43:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f73311ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 0x7f732c1eb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f73311ae700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c156830 0x7f732c1eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f73311ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c1542f0 0x7f732c14db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f73311ae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c1dbcc0 con 0x7f732c0adc30 2024-03-30T01:43:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f73311ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0a71d0 con 0x7f732c1542f0 2024-03-30T01:43:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f73311ae700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c1e9440 con 0x7f732c156830 2024-03-30T01:43:23.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f732ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 0x7f732c1eb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f732ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 0x7f732c1eb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54622/0 (socket says 172.21.15.31:54622) 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.528+0000 7f732ad9d700 1 -- 172.21.15.31:0/2821110291 learned_addr learned my addr 172.21.15.31:0/2821110291 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f732b59e700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c1542f0 0x7f732c14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f732a59c700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c156830 0x7f732c1eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f732ad9d700 1 -- 172.21.15.31:0/2821110291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c156830 msgr2=0x7f732c1eb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f732ad9d700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c156830 0x7f732c1eb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f732ad9d700 1 -- 172.21.15.31:0/2821110291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c1542f0 msgr2=0x7f732c14db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f732ad9d700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c1542f0 0x7f732c14db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f732ad9d700 1 -- 172.21.15.31:0/2821110291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7324009a70 con 0x7f732c0adc30 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f732b59e700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c1542f0 0x7f732c14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f732ad9d700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 0x7f732c1eb3a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f731c00c580 tx=0x7f731c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f73137fe700 1 -- 172.21.15.31:0/2821110291 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c004c80 con 0x7f732c0adc30 2024-03-30T01:43:23.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f73137fe700 1 -- 172.21.15.31:0/2821110291 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c004e00 con 0x7f732c0adc30 2024-03-30T01:43:23.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c1e9140 con 0x7f732c0adc30 2024-03-30T01:43:23.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f73137fe700 1 -- 172.21.15.31:0/2821110291 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c007310 con 0x7f732c0adc30 2024-03-30T01:43:23.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.529+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c1dc180 con 0x7f732c0adc30 2024-03-30T01:43:23.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.530+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c04ea80 con 0x7f732c0adc30 2024-03-30T01:43:23.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.531+0000 7f73137fe700 1 -- 172.21.15.31:0/2821110291 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f731c007470 con 0x7f732c0adc30 2024-03-30T01:43:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.531+0000 7f73137fe700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f731806ac90 0x7f731806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.531+0000 7f73137fe700 1 -- 172.21.15.31:0/2821110291 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f731c089080 con 0x7f732c0adc30 2024-03-30T01:43:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.531+0000 7f732a59c700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f731806ac90 0x7f731806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.532+0000 7f732a59c700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f731806ac90 0x7f731806d150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f732c1561d0 tx=0x7f7320008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.536+0000 7f73137fe700 1 -- 172.21.15.31:0/2821110291 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f731c0541b0 con 0x7f732c0adc30 2024-03-30T01:43:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.750+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f732c06e5a0 con 0x7f732c0adc30 2024-03-30T01:43:23.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.808+0000 7f73137fe700 1 -- 172.21.15.31:0/2821110291 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v61) v1 ==== 140+0+0 (secure 0 0 0) 0x7f731c0577d0 con 0x7f732c0adc30 2024-03-30T01:43:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f731806ac90 msgr2=0x7f731806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f731806ac90 0x7f731806d150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f732c1561d0 tx=0x7f7320008040 comp rx=0 tx=0).stop 2024-03-30T01:43:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 msgr2=0x7f732c1eb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 0x7f732c1eb3a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f731c00c580 tx=0x7f731c00c940 comp rx=0 tx=0).stop 2024-03-30T01:43:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 shutdown_connections 2024-03-30T01:43:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732c0adc30 0x7f732c1eb3a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f731806ac90 0x7f731806d150 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f732c1542f0 0x7f732c14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 --2- 172.21.15.31:0/2821110291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f732c156830 0x7f732c1eb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.812+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 >> 172.21.15.31:0/2821110291 conn(0x7f732c0c48d0 msgr2=0x7f732c0a8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:23.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.813+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 shutdown_connections 2024-03-30T01:43:23.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.813+0000 7f73311ae700 1 -- 172.21.15.31:0/2821110291 wait complete. 2024-03-30T01:43:23.821 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs2_metadata' created 2024-03-30T01:43:23.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T01:43:23.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.975+0000 7f5e6a070700 1 Processor -- start 2024-03-30T01:43:23.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.975+0000 7f5e6a070700 1 -- start start 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.975+0000 7f5e6a070700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640c2670 0x7f5e6406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.975+0000 7f5e6a070700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e640a8460 con 0x7f5e6406f310 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.975+0000 7f5e6a070700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e640a86b0 con 0x7f5e640c18f0 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.975+0000 7f5e6a070700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641526a0 con 0x7f5e640c2670 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e637fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5e6406f310 0x7f5e64152dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:58900/0 (socket says 172.21.15.31:58900) 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e637fe700 1 -- 172.21.15.31:0/1372040169 learned_addr learned my addr 172.21.15.31:0/1372040169 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e63fff700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640c2670 0x7f5e6406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977002754 0 0) 0x7f5e640a8460 con 0x7f5e6406f310 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e4c0035b0 con 0x7f5e6406f310 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1515911242 0 0) 0x7f5e4c0035b0 con 0x7f5e6406f310 2024-03-30T01:43:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640c2670 msgr2=0x7f5e6406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640c2670 0x7f5e6406f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 >> v1:172.21.15.59:6789/0 conn(0x7f5e640c18f0 legacy=0x7f5e6415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e640a8990 con 0x7f5e6406f310 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f5e54001670 con 0x7f5e6406f310 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 >> v1:172.21.15.31:6789/0 conn(0x7f5e6406f310 legacy=0x7f5e64152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e4c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e4c004850 0x7f5e4c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640c2670 0x7f5e4c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641526a0 con 0x7f5e4c003b10 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c00bb20 con 0x7f5e4c004850 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.976+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c00bce0 con 0x7f5e640c2670 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e637fe700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e4c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e63fff700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640c2670 0x7f5e4c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e62ffd700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e4c004850 0x7f5e4c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e637fe700 1 -- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640c2670 msgr2=0x7f5e4c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e637fe700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640c2670 0x7f5e4c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e637fe700 1 -- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e4c004850 msgr2=0x7f5e4c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e637fe700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e4c004850 0x7f5e4c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e637fe700 1 -- 172.21.15.31:0/1372040169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c0035b0 con 0x7f5e4c003b10 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e637fe700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e4c003ef0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5e54001eb0 tx=0x7f5e4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da557e60dc87752 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5400ce00 con 0x7f5e4c003b10 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54005960 con 0x7f5e4c003b10 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.977+0000 7f5e627fc700 1 -- 172.21.15.31:0/1372040169 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54005cf0 con 0x7f5e4c003b10 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.978+0000 7f5e6a070700 1 -- 172.21.15.31:0/1372040169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 msgr2=0x7f5e4c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.978+0000 7f5e6a070700 1 --2- 172.21.15.31:0/1372040169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e4c003ef0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5e54001eb0 tx=0x7f5e4c005880 comp rx=0 tx=0).stop 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.978+0000 7f5e63fff700 1 -- 172.21.15.31:0/1372040169 reap_dead start 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.978+0000 7f5e6a070700 1 -- 172.21.15.31:0/1372040169 shutdown_connections 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.978+0000 7f5e6a070700 1 -- 172.21.15.31:0/1372040169 >> 172.21.15.31:0/1372040169 conn(0x7f5e640c3980 msgr2=0x7f5e640c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.978+0000 7f5e6a070700 1 -- 172.21.15.31:0/1372040169 shutdown_connections 2024-03-30T01:43:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.978+0000 7f5e6a070700 1 -- 172.21.15.31:0/1372040169 wait complete. 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.979+0000 7f5e6a070700 1 Processor -- start 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e6a070700 1 -- start start 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e6a070700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e6414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e6a070700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640ac120 0x7f5e640aa890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e6a070700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e64131270 0x7f5e64150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e6a070700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64134200 con 0x7f5e4c003b10 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e6a070700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64133cd0 con 0x7f5e64131270 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e6a070700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64133f20 con 0x7f5e640ac120 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e6414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e637fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e6414f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54638/0 (socket says 172.21.15.31:54638) 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.980+0000 7f5e637fe700 1 -- 172.21.15.31:0/2987410260 learned_addr learned my addr 172.21.15.31:0/2987410260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e62ffd700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640ac120 0x7f5e640aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e63fff700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e64131270 0x7f5e64150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e637fe700 1 -- 172.21.15.31:0/2987410260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640ac120 msgr2=0x7f5e640aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e637fe700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640ac120 0x7f5e640aa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e637fe700 1 -- 172.21.15.31:0/2987410260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e64131270 msgr2=0x7f5e64150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e637fe700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e64131270 0x7f5e64150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e637fe700 1 -- 172.21.15.31:0/2987410260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e64133070 con 0x7f5e4c003b10 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e637fe700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e6414f5b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00baf0 tx=0x7f5e54001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e4bfff700 1 -- 172.21.15.31:0/2987410260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004a80 con 0x7f5e4c003b10 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e4bfff700 1 -- 172.21.15.31:0/2987410260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54004be0 con 0x7f5e4c003b10 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e64133350 con 0x7f5e4c003b10 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e4bfff700 1 -- 172.21.15.31:0/2987410260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e540070d0 con 0x7f5e4c003b10 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.981+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e641316f0 con 0x7f5e4c003b10 2024-03-30T01:43:23.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.983+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e641318e0 con 0x7f5e4c003b10 2024-03-30T01:43:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.983+0000 7f5e4bfff700 1 -- 172.21.15.31:0/2987410260 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5e54007230 con 0x7f5e4c003b10 2024-03-30T01:43:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.983+0000 7f5e4bfff700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e5006ac90 0x7f5e5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.984+0000 7f5e4bfff700 1 -- 172.21.15.31:0/2987410260 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6400+0+0 (secure 0 0 0) 0x7f5e5408aab0 con 0x7f5e4c003b10 2024-03-30T01:43:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.984+0000 7f5e62ffd700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e5006ac90 0x7f5e5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:23.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.984+0000 7f5e62ffd700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e5006ac90 0x7f5e5006d150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e64133c60 tx=0x7f5e58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:23.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:23.988+0000 7f5e4bfff700 1 -- 172.21.15.31:0/2987410260 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5e54055b30 con 0x7f5e4c003b10 2024-03-30T01:43:24.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.204+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5e640aa6e0 con 0x7f5e4c003b10 2024-03-30T01:43:24.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.817+0000 7f5e4bfff700 1 -- 172.21.15.31:0/2987410260 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v62) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5e540170d0 con 0x7f5e4c003b10 2024-03-30T01:43:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.828+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e5006ac90 msgr2=0x7f5e5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.828+0000 7f5e6a070700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e5006ac90 0x7f5e5006d150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e64133c60 tx=0x7f5e58009040 comp rx=0 tx=0).stop 2024-03-30T01:43:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.828+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 msgr2=0x7f5e6414f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.828+0000 7f5e6a070700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e6414f5b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00baf0 tx=0x7f5e54001a00 comp rx=0 tx=0).stop 2024-03-30T01:43:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.829+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 shutdown_connections 2024-03-30T01:43:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.829+0000 7f5e6a070700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e4c003b10 0x7f5e6414f5b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.829+0000 7f5e6a070700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f5e5006ac90 0x7f5e5006d150 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:24.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.829+0000 7f5e6a070700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e64131270 0x7f5e64150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:24.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.829+0000 7f5e6a070700 1 --2- 172.21.15.31:0/2987410260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e640ac120 0x7f5e640aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:24.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.829+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 >> 172.21.15.31:0/2987410260 conn(0x7f5e640c3980 msgr2=0x7f5e640c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:24.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.830+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 shutdown_connections 2024-03-30T01:43:24.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.831+0000 7f5e6a070700 1 -- 172.21.15.31:0/2987410260 wait complete. 2024-03-30T01:43:24.845 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs2_data' created 2024-03-30T01:43:24.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-30T01:43:25.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.998+0000 7ffa19897700 1 Processor -- start 2024-03-30T01:43:25.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.998+0000 7ffa19897700 1 -- start start 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.999+0000 7ffa19897700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa1414e650 0x7ffa141553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.999+0000 7ffa19897700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa140c4970 0x7ffa140c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.999+0000 7ffa19897700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa141584c0 con 0x7ffa140c7ef0 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.999+0000 7ffa19897700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa140c6430 con 0x7ffa140c4970 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.999+0000 7ffa19897700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1406cae0 con 0x7ffa1414e650 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.999+0000 7ffa12ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa1414e650 0x7ffa141553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.999+0000 7ffa137fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa140c4970 0x7ffa140c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.999+0000 7ffa12ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa1414e650 0x7ffa141553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:48946/0 (socket says 172.21.15.31:48946) 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:24.999+0000 7ffa12ffd700 1 -- 172.21.15.31:0/854445759 learned_addr learned my addr 172.21.15.31:0/854445759 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.000+0000 7ffa12ffd700 1 -- 172.21.15.31:0/854445759 >> v2:172.21.15.59:3300/0 conn(0x7ffa140c4970 msgr2=0x7ffa140c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.000+0000 7ffa12ffd700 1 --2- 172.21.15.31:0/854445759 >> v2:172.21.15.59:3300/0 conn(0x7ffa140c4970 0x7ffa140c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.000+0000 7ffa12ffd700 1 -- 172.21.15.31:0/854445759 >> v1:172.21.15.59:6789/0 conn(0x7ffa140c7ef0 legacy=0x7ffa14157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.000+0000 7ffa12ffd700 1 -- 172.21.15.31:0/854445759 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa140c8400 con 0x7ffa1414e650 2024-03-30T01:43:25.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.000+0000 7ffa137fe700 1 --2- 172.21.15.31:0/854445759 >> v2:172.21.15.59:3300/0 conn(0x7ffa140c4970 0x7ffa140c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:25.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.000+0000 7ffa12ffd700 1 --2- 172.21.15.31:0/854445759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa1414e650 0x7ffa141553b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa040098f0 tx=0x7ffa04009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdaa91dd1bf658dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:25.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.000+0000 7ffa127fc700 1 -- 172.21.15.31:0/854445759 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04011830 con 0x7ffa1414e650 2024-03-30T01:43:25.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.000+0000 7ffa127fc700 1 -- 172.21.15.31:0/854445759 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa04011990 con 0x7ffa1414e650 2024-03-30T01:43:25.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.000+0000 7ffa127fc700 1 -- 172.21.15.31:0/854445759 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04011d20 con 0x7ffa1414e650 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.001+0000 7ffa19897700 1 -- 172.21.15.31:0/854445759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa1414e650 msgr2=0x7ffa141553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.001+0000 7ffa19897700 1 --2- 172.21.15.31:0/854445759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa1414e650 0x7ffa141553b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa040098f0 tx=0x7ffa04009c00 comp rx=0 tx=0).stop 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.001+0000 7ffa19897700 1 -- 172.21.15.31:0/854445759 shutdown_connections 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.001+0000 7ffa19897700 1 --2- 172.21.15.31:0/854445759 >> v2:172.21.15.59:3300/0 conn(0x7ffa140c4970 0x7ffa140c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.001+0000 7ffa19897700 1 --2- 172.21.15.31:0/854445759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa1414e650 0x7ffa141553b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.001+0000 7ffa19897700 1 -- 172.21.15.31:0/854445759 >> 172.21.15.31:0/854445759 conn(0x7ffa14072010 msgr2=0x7ffa14072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.001+0000 7ffa19897700 1 -- 172.21.15.31:0/854445759 shutdown_connections 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.001+0000 7ffa19897700 1 -- 172.21.15.31:0/854445759 wait complete. 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.002+0000 7ffa19897700 1 Processor -- start 2024-03-30T01:43:25.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.002+0000 7ffa19897700 1 -- start start 2024-03-30T01:43:25.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa19897700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa140c4970 0x7ffa141ed260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa19897700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa140c7ef0 0x7ffa141ed7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa19897700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa1414e650 0x7ffa1414fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa19897700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140befe0 con 0x7ffa140c7ef0 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa19897700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140c0420 con 0x7ffa1414e650 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa19897700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141eddc0 con 0x7ffa140c4970 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa140c7ef0 0x7ffa141ed7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa0bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa140c7ef0 0x7ffa141ed7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54642/0 (socket says 172.21.15.31:54642) 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa0bfff700 1 -- 172.21.15.31:0/583620334 learned_addr learned my addr 172.21.15.31:0/583620334 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa137fe700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa1414e650 0x7ffa1414fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.003+0000 7ffa12ffd700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa140c4970 0x7ffa141ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa0bfff700 1 -- 172.21.15.31:0/583620334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa140c4970 msgr2=0x7ffa141ed260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa0bfff700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa140c4970 0x7ffa141ed260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa0bfff700 1 -- 172.21.15.31:0/583620334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa1414e650 msgr2=0x7ffa1414fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa0bfff700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa1414e650 0x7ffa1414fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa0bfff700 1 -- 172.21.15.31:0/583620334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c009a70 con 0x7ffa140c7ef0 2024-03-30T01:43:25.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa0bfff700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa140c7ef0 0x7ffa141ed7a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c4b0 tx=0x7ff9fc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa10ff9700 1 -- 172.21.15.31:0/583620334 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0052d0 con 0x7ffa140c7ef0 2024-03-30T01:43:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa10ff9700 1 -- 172.21.15.31:0/583620334 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc005430 con 0x7ffa140c7ef0 2024-03-30T01:43:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa04009550 con 0x7ffa140c7ef0 2024-03-30T01:43:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa10ff9700 1 -- 172.21.15.31:0/583620334 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0038c0 con 0x7ffa140c7ef0 2024-03-30T01:43:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.004+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa140c0140 con 0x7ffa140c7ef0 2024-03-30T01:43:25.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.005+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1406c4d0 con 0x7ffa140c7ef0 2024-03-30T01:43:25.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.006+0000 7ffa10ff9700 1 -- 172.21.15.31:0/583620334 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff9fc003a20 con 0x7ffa140c7ef0 2024-03-30T01:43:25.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.006+0000 7ffa10ff9700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff9f406ac90 0x7ff9f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.006+0000 7ffa10ff9700 1 -- 172.21.15.31:0/583620334 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6783+0+0 (secure 0 0 0) 0x7ff9fc088ab0 con 0x7ffa140c7ef0 2024-03-30T01:43:25.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.006+0000 7ffa12ffd700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff9f406ac90 0x7ff9f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.007+0000 7ffa12ffd700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff9f406ac90 0x7ff9f406d150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009c40 tx=0x7ffa0401d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:25.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.011+0000 7ffa10ff9700 1 -- 172.21.15.31:0/583620334 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff9fc053860 con 0x7ffa140c7ef0 2024-03-30T01:43:25.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.275+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7ffa1404ea80 con 0x7ffa140c7ef0 2024-03-30T01:43:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.281+0000 7ffa10ff9700 1 -- 172.21.15.31:0/583620334 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 11 and data pool 12 v32) v1 ==== 177+0+0 (secure 0 0 0) 0x7ff9fc056e80 con 0x7ffa140c7ef0 2024-03-30T01:43:25.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff9f406ac90 msgr2=0x7ff9f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff9f406ac90 0x7ff9f406d150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009c40 tx=0x7ffa0401d040 comp rx=0 tx=0).stop 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa140c7ef0 msgr2=0x7ffa141ed7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa140c7ef0 0x7ffa141ed7a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c4b0 tx=0x7ff9fc00c7c0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 shutdown_connections 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff9f406ac90 0x7ff9f406d150 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa1414e650 0x7ffa1414fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa140c4970 0x7ffa141ed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 --2- 172.21.15.31:0/583620334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa140c7ef0 0x7ffa141ed7a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 >> 172.21.15.31:0/583620334 conn(0x7ffa14072010 msgr2=0x7ffa140c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.286+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 shutdown_connections 2024-03-30T01:43:25.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.287+0000 7ffa19897700 1 -- 172.21.15.31:0/583620334 wait complete. 2024-03-30T01:43:25.306 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 11 and data pool 12 2024-03-30T01:43:25.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:25.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.457+0000 7fb1b27e4700 1 Processor -- start 2024-03-30T01:43:25.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.458+0000 7fb1b27e4700 1 -- start start 2024-03-30T01:43:25.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.458+0000 7fb1b27e4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.458+0000 7fb1b27e4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1ac0a74e0 0x7fb1ac0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.458+0000 7fb1b27e4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac153590 con 0x7fb1ac0ac520 2024-03-30T01:43:25.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.458+0000 7fb1b27e4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac0ab680 con 0x7fb1ac0a74e0 2024-03-30T01:43:25.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.458+0000 7fb1b27e4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac1500c0 con 0x7fb1ac0ad010 2024-03-30T01:43:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1ab7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1b0d81700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1ac0a74e0 0x7fb1ac0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1b0d81700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1ac0a74e0 0x7fb1ac0c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.31:47986/0 (socket says 172.21.15.31:47986) 2024-03-30T01:43:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1ab7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac0c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:36186/0 (socket says 172.21.15.31:36186) 2024-03-30T01:43:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1b0d81700 1 -- 172.21.15.31:0/1105098422 learned_addr learned my addr 172.21.15.31:0/1105098422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1ab7fe700 1 -- 172.21.15.31:0/1105098422 >> v2:172.21.15.59:3300/0 conn(0x7fb1ac0a74e0 msgr2=0x7fb1ac0c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1ab7fe700 1 --2- 172.21.15.31:0/1105098422 >> v2:172.21.15.59:3300/0 conn(0x7fb1ac0a74e0 0x7fb1ac0c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1ab7fe700 1 -- 172.21.15.31:0/1105098422 >> v1:172.21.15.59:6789/0 conn(0x7fb1ac0ac520 legacy=0x7fb1ac0ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1ab7fe700 1 -- 172.21.15.31:0/1105098422 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac0c7990 con 0x7fb1ac0ad010 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1b0d81700 1 --2- 172.21.15.31:0/1105098422 >> v2:172.21.15.59:3300/0 conn(0x7fb1ac0a74e0 0x7fb1ac0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.459+0000 7fb1ab7fe700 1 --2- 172.21.15.31:0/1105098422 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac0c6000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb194009870 tx=0x7fb194009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d987db39667d20b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1aaffd700 1 -- 172.21.15.31:0/1105098422 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194011830 con 0x7fb1ac0ad010 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1aaffd700 1 -- 172.21.15.31:0/1105098422 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb194011990 con 0x7fb1ac0ad010 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1aaffd700 1 -- 172.21.15.31:0/1105098422 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194011d20 con 0x7fb1ac0ad010 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1b27e4700 1 -- 172.21.15.31:0/1105098422 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0ad010 msgr2=0x7fb1ac0c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1b27e4700 1 --2- 172.21.15.31:0/1105098422 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac0c6000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb194009870 tx=0x7fb194009b80 comp rx=0 tx=0).stop 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1b27e4700 1 -- 172.21.15.31:0/1105098422 shutdown_connections 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1b27e4700 1 --2- 172.21.15.31:0/1105098422 >> v2:172.21.15.59:3300/0 conn(0x7fb1ac0a74e0 0x7fb1ac0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1b27e4700 1 --2- 172.21.15.31:0/1105098422 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0ad010 0x7fb1ac0c6000 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1b27e4700 1 -- 172.21.15.31:0/1105098422 >> 172.21.15.31:0/1105098422 conn(0x7fb1ac06ad30 msgr2=0x7fb1ac0a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1b27e4700 1 -- 172.21.15.31:0/1105098422 shutdown_connections 2024-03-30T01:43:25.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.460+0000 7fb1b27e4700 1 -- 172.21.15.31:0/1105098422 wait complete. 2024-03-30T01:43:25.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.462+0000 7fb1b27e4700 1 Processor -- start 2024-03-30T01:43:25.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.462+0000 7fb1b27e4700 1 -- start start 2024-03-30T01:43:25.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.462+0000 7fb1b27e4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0a74e0 0x7fb1ac0a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.462+0000 7fb1b27e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0ac520 0x7fb1ac14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.462+0000 7fb1b27e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.462+0000 7fb1b27e4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac14f080 con 0x7fb1ac0ac520 2024-03-30T01:43:25.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.462+0000 7fb1b27e4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac070350 con 0x7fb1ac0ad010 2024-03-30T01:43:25.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.462+0000 7fb1b27e4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac14f2d0 con 0x7fb1ac0a74e0 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0ac520 0x7fb1ac14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0ac520 0x7fb1ac14e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48618/0 (socket says 172.21.15.31:48618) 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1ab7fe700 1 -- 172.21.15.31:0/3557262800 learned_addr learned my addr 172.21.15.31:0/3557262800 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1abfff700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0a74e0 0x7fb1ac0a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1b0d81700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1ab7fe700 1 -- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0a74e0 msgr2=0x7fb1ac0a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1ab7fe700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0a74e0 0x7fb1ac0a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1ab7fe700 1 -- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ac0ad010 msgr2=0x7fb1ac153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1ab7fe700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1ab7fe700 1 -- 172.21.15.31:0/3557262800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4009b30 con 0x7fb1ac0ac520 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1b0d81700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1abfff700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0a74e0 0x7fb1ac0a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1ab7fe700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0ac520 0x7fb1ac14e610 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb194014040 tx=0x7fb194004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1a8ff9700 1 -- 172.21.15.31:0/3557262800 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940038a0 con 0x7fb1ac0ac520 2024-03-30T01:43:25.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1a8ff9700 1 -- 172.21.15.31:0/3557262800 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19401e070 con 0x7fb1ac0ac520 2024-03-30T01:43:25.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1940094a0 con 0x7fb1ac0ac520 2024-03-30T01:43:25.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.463+0000 7fb1a8ff9700 1 -- 172.21.15.31:0/3557262800 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194003c40 con 0x7fb1ac0ac520 2024-03-30T01:43:25.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.464+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac1ead20 con 0x7fb1ac0ac520 2024-03-30T01:43:25.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.465+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac04ea80 con 0x7fb1ac0ac520 2024-03-30T01:43:25.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.465+0000 7fb1a8ff9700 1 -- 172.21.15.31:0/3557262800 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb19402d450 con 0x7fb1ac0ac520 2024-03-30T01:43:25.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.466+0000 7fb1a8ff9700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb18c06ac90 0x7fb18c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.466+0000 7fb1a8ff9700 1 -- 172.21.15.31:0/3557262800 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fb19408a700 con 0x7fb1ac0ac520 2024-03-30T01:43:25.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.466+0000 7fb1abfff700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb18c06ac90 0x7fb18c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.466+0000 7fb1abfff700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb18c06ac90 0x7fb18c06d150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb19c000f60 tx=0x7fb19c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:25.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.471+0000 7fb1a8ff9700 1 -- 172.21.15.31:0/3557262800 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb1940554d0 con 0x7fb1ac0ac520 2024-03-30T01:43:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.689+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1ac0c6ff0 con 0x7fb1ac0ac520 2024-03-30T01:43:25.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.691+0000 7fb1a8ff9700 1 -- 172.21.15.31:0/3557262800 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+19931 (secure 0 0 0) 0x7fb194058af0 con 0x7fb1ac0ac520 2024-03-30T01:43:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:25.698 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":63,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:25.277506+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:43:09.911033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T01:43:11.216248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:43:23.783452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:43:24.238093+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb18c06ac90 msgr2=0x7fb18c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb18c06ac90 0x7fb18c06d150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb19c000f60 tx=0x7fb19c009040 comp rx=0 tx=0).stop 2024-03-30T01:43:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0ac520 msgr2=0x7fb1ac14e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0ac520 0x7fb1ac14e610 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb194014040 tx=0x7fb194004750 comp rx=0 tx=0).stop 2024-03-30T01:43:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 shutdown_connections 2024-03-30T01:43:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fb18c06ac90 0x7fb18c06d150 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ac0ad010 0x7fb1ac153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1ac0a74e0 0x7fb1ac0a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 --2- 172.21.15.31:0/3557262800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1ac0ac520 0x7fb1ac14e610 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 >> 172.21.15.31:0/3557262800 conn(0x7fb1ac06ad30 msgr2=0x7fb1ac06c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:25.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 shutdown_connections 2024-03-30T01:43:25.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.693+0000 7fb1b27e4700 1 -- 172.21.15.31:0/3557262800 wait complete. 2024-03-30T01:43:25.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:25.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.845+0000 7efc35def700 1 Processor -- start 2024-03-30T01:43:25.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.845+0000 7efc35def700 1 -- start start 2024-03-30T01:43:25.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.845+0000 7efc35def700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071240 0x7efc30154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.845+0000 7efc35def700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc30072160 con 0x7efc30071750 2024-03-30T01:43:25.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.845+0000 7efc35def700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300c2bd0 con 0x7efc3014e5f0 2024-03-30T01:43:25.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.845+0000 7efc35def700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc300c2e90 con 0x7efc30071240 2024-03-30T01:43:25.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2ffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efc30071750 0x7efc30154dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54804/0 (socket says 172.21.15.31:54804) 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2ffff700 1 -- 172.21.15.31:0/3602931207 learned_addr learned my addr 172.21.15.31:0/3602931207 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2effd700 1 --2- 172.21.15.31:0/3602931207 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071240 0x7efc30154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2e7fc700 1 -- 172.21.15.31:0/3602931207 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083045495 0 0) 0x7efc30072160 con 0x7efc30071750 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2e7fc700 1 -- 172.21.15.31:0/3602931207 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc180035b0 con 0x7efc30071750 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2e7fc700 1 -- 172.21.15.31:0/3602931207 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633333680 0 0) 0x7efc300c2bd0 con 0x7efc3014e5f0 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2e7fc700 1 -- 172.21.15.31:0/3602931207 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc30072160 con 0x7efc3014e5f0 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2effd700 1 -- 172.21.15.31:0/3602931207 >> v1:172.21.15.59:6789/0 conn(0x7efc3014e5f0 legacy=0x7efc300c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2effd700 1 -- 172.21.15.31:0/3602931207 >> v1:172.21.15.31:6789/0 conn(0x7efc30071750 legacy=0x7efc30154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2e7fc700 1 -- 172.21.15.31:0/3602931207 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3280989177 0 0) 0x7efc180035b0 con 0x7efc30071750 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.846+0000 7efc2effd700 1 -- 172.21.15.31:0/3602931207 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300723b0 con 0x7efc30071240 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc2effd700 1 --2- 172.21.15.31:0/3602931207 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071240 0x7efc30154890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc24009870 tx=0x7efc24009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea633f798a2f482e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc2e7fc700 1 -- 172.21.15.31:0/3602931207 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24011830 con 0x7efc30071240 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc2e7fc700 1 -- 172.21.15.31:0/3602931207 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc24011990 con 0x7efc30071240 2024-03-30T01:43:25.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc2e7fc700 1 -- 172.21.15.31:0/3602931207 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24011d20 con 0x7efc30071240 2024-03-30T01:43:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc35def700 1 -- 172.21.15.31:0/3602931207 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071240 msgr2=0x7efc30154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc35def700 1 --2- 172.21.15.31:0/3602931207 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071240 0x7efc30154890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efc24009870 tx=0x7efc24009b80 comp rx=0 tx=0).stop 2024-03-30T01:43:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc35def700 1 -- 172.21.15.31:0/3602931207 shutdown_connections 2024-03-30T01:43:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc35def700 1 --2- 172.21.15.31:0/3602931207 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071240 0x7efc30154890 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc35def700 1 -- 172.21.15.31:0/3602931207 >> 172.21.15.31:0/3602931207 conn(0x7efc3006ba20 msgr2=0x7efc3006be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.847+0000 7efc35def700 1 -- 172.21.15.31:0/3602931207 shutdown_connections 2024-03-30T01:43:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.848+0000 7efc35def700 1 -- 172.21.15.31:0/3602931207 wait complete. 2024-03-30T01:43:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.849+0000 7efc35def700 1 Processor -- start 2024-03-30T01:43:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.849+0000 7efc35def700 1 -- start start 2024-03-30T01:43:25.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc35def700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc30071240 0x7efc301ea410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc35def700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071750 0x7efc301467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc35def700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3014e5f0 0x7efc30153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc35def700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3014ec40 con 0x7efc3014e5f0 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc35def700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc301e9f80 con 0x7efc30071240 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc35def700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc301ed8d0 con 0x7efc30071750 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3014e5f0 0x7efc30153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3014e5f0 0x7efc30153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48624/0 (socket says 172.21.15.31:48624) 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2ffff700 1 -- 172.21.15.31:0/2673529084 learned_addr learned my addr 172.21.15.31:0/2673529084 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2f7fe700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc30071240 0x7efc301ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2effd700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071750 0x7efc301467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2ffff700 1 -- 172.21.15.31:0/2673529084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071750 msgr2=0x7efc301467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2ffff700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071750 0x7efc301467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2ffff700 1 -- 172.21.15.31:0/2673529084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc30071240 msgr2=0x7efc301ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2ffff700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc30071240 0x7efc301ea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2ffff700 1 -- 172.21.15.31:0/2673529084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc240094a0 con 0x7efc3014e5f0 2024-03-30T01:43:25.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.850+0000 7efc2ffff700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3014e5f0 0x7efc30153050 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cb70 tx=0x7efc2800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:25.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.851+0000 7efc17fff700 1 -- 172.21.15.31:0/2673529084 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800ab50 con 0x7efc3014e5f0 2024-03-30T01:43:25.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.851+0000 7efc17fff700 1 -- 172.21.15.31:0/2673529084 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2800acb0 con 0x7efc3014e5f0 2024-03-30T01:43:25.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.851+0000 7efc17fff700 1 -- 172.21.15.31:0/2673529084 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28012640 con 0x7efc3014e5f0 2024-03-30T01:43:25.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.851+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc301e9930 con 0x7efc3014e5f0 2024-03-30T01:43:25.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.851+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc301edbb0 con 0x7efc3014e5f0 2024-03-30T01:43:25.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.852+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc301ea110 con 0x7efc3014e5f0 2024-03-30T01:43:25.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.852+0000 7efc17fff700 1 -- 172.21.15.31:0/2673529084 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efc280127a0 con 0x7efc3014e5f0 2024-03-30T01:43:25.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.853+0000 7efc17fff700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc1c06ac90 0x7efc1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:25.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.853+0000 7efc2f7fe700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc1c06ac90 0x7efc1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:25.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.853+0000 7efc17fff700 1 -- 172.21.15.31:0/2673529084 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6885+0+0 (secure 0 0 0) 0x7efc28089b70 con 0x7efc3014e5f0 2024-03-30T01:43:25.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.853+0000 7efc2f7fe700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc1c06ac90 0x7efc1c06d150 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc20007fd0 tx=0x7efc20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:25.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:25.858+0000 7efc17fff700 1 -- 172.21.15.31:0/2673529084 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efc280583d0 con 0x7efc3014e5f0 2024-03-30T01:43:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.072+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc3004ea80 con 0x7efc3014e5f0 2024-03-30T01:43:26.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.073+0000 7efc17fff700 1 -- 172.21.15.31:0/2673529084 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+19931 (secure 0 0 0) 0x7efc28057f60 con 0x7efc3014e5f0 2024-03-30T01:43:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:26.080 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":63,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:25.277506+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:43:09.911033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T01:43:11.216248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:43:23.783452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:43:24.238093+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc1c06ac90 msgr2=0x7efc1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc1c06ac90 0x7efc1c06d150 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc20007fd0 tx=0x7efc20009040 comp rx=0 tx=0).stop 2024-03-30T01:43:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3014e5f0 msgr2=0x7efc30153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3014e5f0 0x7efc30153050 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cb70 tx=0x7efc2800cf30 comp rx=0 tx=0).stop 2024-03-30T01:43:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 shutdown_connections 2024-03-30T01:43:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc3014e5f0 0x7efc30153050 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7efc1c06ac90 0x7efc1c06d150 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc30071240 0x7efc301ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 --2- 172.21.15.31:0/2673529084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc30071750 0x7efc301467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 >> 172.21.15.31:0/2673529084 conn(0x7efc3006ba20 msgr2=0x7efc300c76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 shutdown_connections 2024-03-30T01:43:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.076+0000 7efc35def700 1 -- 172.21.15.31:0/2673529084 wait complete. 2024-03-30T01:43:26.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-30T01:43:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.231+0000 7f1b7fe3c700 1 Processor -- start 2024-03-30T01:43:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.232+0000 7f1b7fe3c700 1 -- start start 2024-03-30T01:43:26.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.232+0000 7f1b7fe3c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b780a7600 0x7f1b78152f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:26.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.232+0000 7f1b7fe3c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b780c4620 con 0x7f1b7814ff20 2024-03-30T01:43:26.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.232+0000 7f1b7fe3c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b78152ad0 con 0x7f1b7814fb40 2024-03-30T01:43:26.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.232+0000 7f1b7fe3c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b78152570 con 0x7f1b780a7600 2024-03-30T01:43:26.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7e3d9700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1b7814ff20 0x7f1b780c47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54818/0 (socket says 172.21.15.31:54818) 2024-03-30T01:43:26.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7e3d9700 1 -- 172.21.15.31:0/708527380 learned_addr learned my addr 172.21.15.31:0/708527380 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7d3d7700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b780a7600 0x7f1b78152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192263047 0 0) 0x7f1b70001490 con 0x7f1b7814ff20 2024-03-30T01:43:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b640035b0 con 0x7f1b7814ff20 2024-03-30T01:43:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777549235 0 0) 0x7f1b78152ad0 con 0x7f1b7814fb40 2024-03-30T01:43:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b70001490 con 0x7f1b7814fb40 2024-03-30T01:43:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1493048748 0 0) 0x7f1b640035b0 con 0x7f1b7814ff20 2024-03-30T01:43:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b780a7600 msgr2=0x7f1b78152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b780a7600 0x7f1b78152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 >> v1:172.21.15.59:6789/0 conn(0x7f1b7814fb40 legacy=0x7f1b780c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b781534a0 con 0x7f1b7814ff20 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f1b70001b80 con 0x7f1b7814ff20 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 >> v1:172.21.15.31:6789/0 conn(0x7f1b7814ff20 legacy=0x7f1b780c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b780a7600 0x7f1b64003ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b64004250 0x7f1b64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b64004d90 0x7f1b64005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6400bbd0 con 0x7f1b64004250 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6400bd40 con 0x7f1b64004d90 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6400bf00 con 0x7f1b780a7600 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.233+0000 7f1b7d3d7700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b64004250 0x7f1b64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7dbd8700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b780a7600 0x7f1b64003ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7e3d9700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b64004d90 0x7f1b64005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7d3d7700 1 -- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b780a7600 msgr2=0x7f1b64003ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7d3d7700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b780a7600 0x7f1b64003ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7d3d7700 1 -- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b64004d90 msgr2=0x7f1b64005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7d3d7700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b64004d90 0x7f1b64005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7d3d7700 1 -- 172.21.15.31:0/708527380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b640035b0 con 0x7f1b64004250 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7d3d7700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b64004250 0x7f1b64004850 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009b60 tx=0x7f1b64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0591d1c13aa802c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800e6c0 con 0x7f1b64004250 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6800e820 con 0x7f1b64004250 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.234+0000 7f1b7cbd6700 1 -- 172.21.15.31:0/708527380 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800eb90 con 0x7f1b64004250 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.235+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/708527380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b64004250 msgr2=0x7f1b64004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.235+0000 7f1b7fe3c700 1 --2- 172.21.15.31:0/708527380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b64004250 0x7f1b64004850 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009b60 tx=0x7f1b64005880 comp rx=0 tx=0).stop 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.235+0000 7f1b7e3d9700 1 -- 172.21.15.31:0/708527380 reap_dead start 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.235+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/708527380 shutdown_connections 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.235+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/708527380 >> 172.21.15.31:0/708527380 conn(0x7f1b780c7d60 msgr2=0x7f1b780c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.235+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/708527380 shutdown_connections 2024-03-30T01:43:26.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.235+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/708527380 wait complete. 2024-03-30T01:43:26.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.236+0000 7f1b7fe3c700 1 Processor -- start 2024-03-30T01:43:26.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.236+0000 7f1b7fe3c700 1 -- start start 2024-03-30T01:43:26.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7fe3c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b64004250 0x7f1b780c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:26.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7fe3c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7806b810 0x7f1b780c1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7fe3c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b781e9da0 0x7f1b781ea220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7fe3c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780c5d20 con 0x7f1b7806b810 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7fe3c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780c6d40 con 0x7f1b64004250 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7fe3c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7806c090 con 0x7f1b781e9da0 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7d3d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7806b810 0x7f1b780c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7d3d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7806b810 0x7f1b780c1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48650/0 (socket says 172.21.15.31:48650) 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7d3d7700 1 -- 172.21.15.31:0/3319601406 learned_addr learned my addr 172.21.15.31:0/3319601406 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7e3d9700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b781e9da0 0x7f1b781ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7dbd8700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b64004250 0x7f1b780c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7d3d7700 1 -- 172.21.15.31:0/3319601406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b781e9da0 msgr2=0x7f1b781ea220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7d3d7700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b781e9da0 0x7f1b781ea220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7d3d7700 1 -- 172.21.15.31:0/3319601406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b64004250 msgr2=0x7f1b780c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7d3d7700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b64004250 0x7f1b780c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:26.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7d3d7700 1 -- 172.21.15.31:0/3319601406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7806c650 con 0x7f1b7806b810 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.237+0000 7f1b7d3d7700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7806b810 0x7f1b780c1430 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1b680090b0 tx=0x7f1b680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.238+0000 7f1b6e7fc700 1 -- 172.21.15.31:0/3319601406 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800e6c0 con 0x7f1b7806b810 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.238+0000 7f1b6e7fc700 1 -- 172.21.15.31:0/3319601406 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68003ad0 con 0x7f1b7806b810 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.238+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b781edba0 con 0x7f1b7806b810 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.238+0000 7f1b6e7fc700 1 -- 172.21.15.31:0/3319601406 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b680045d0 con 0x7f1b7806b810 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.238+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7806c2e0 con 0x7f1b7806b810 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.239+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7806c4a0 con 0x7f1b7806b810 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.239+0000 7f1b6e7fc700 1 -- 172.21.15.31:0/3319601406 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1b68013470 con 0x7f1b7806b810 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.240+0000 7f1b6e7fc700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1b5c06ac90 0x7f1b5c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.240+0000 7f1b6e7fc700 1 -- 172.21.15.31:0/3319601406 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f1b6808aeb0 con 0x7f1b7806b810 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.240+0000 7f1b7dbd8700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1b5c06ac90 0x7f1b5c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.240+0000 7f1b7dbd8700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1b5c06ac90 0x7f1b5c06d150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b74000c00 tx=0x7f1b74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:26.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.244+0000 7f1b6e7fc700 1 -- 172.21.15.31:0/3319601406 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1b68055c80 con 0x7f1b7806b810 2024-03-30T01:43:26.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:26.512+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1b780ae130 con 0x7f1b7806b810 2024-03-30T01:43:27.007 INFO:tasks.ceph.mds.c.smithi031.stderr:2024-03-30T01:43:27.001+0000 7f3268f60700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T01:43:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.287+0000 7f1b6e7fc700 1 -- 172.21.15.31:0/3319601406 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 119+0+0 (secure 0 0 0) 0x7f1b680592a0 con 0x7f1b7806b810 2024-03-30T01:43:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1b5c06ac90 msgr2=0x7f1b5c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1b5c06ac90 0x7f1b5c06d150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b74000c00 tx=0x7f1b74009040 comp rx=0 tx=0).stop 2024-03-30T01:43:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7806b810 msgr2=0x7f1b780c1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7806b810 0x7f1b780c1430 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1b680090b0 tx=0x7f1b680047c0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 shutdown_connections 2024-03-30T01:43:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f1b5c06ac90 0x7f1b5c06d150 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b781e9da0 0x7f1b781ea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b64004250 0x7f1b780c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 --2- 172.21.15.31:0/3319601406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7806b810 0x7f1b780c1430 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 >> 172.21.15.31:0/3319601406 conn(0x7f1b780c7d60 msgr2=0x7f1b780c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 shutdown_connections 2024-03-30T01:43:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.290+0000 7f1b7fe3c700 1 -- 172.21.15.31:0/3319601406 wait complete. 2024-03-30T01:43:27.312 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T01:43:27.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.448+0000 7f06367f4700 1 Processor -- start 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.448+0000 7f06367f4700 1 -- start start 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.449+0000 7f06367f4700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06301549a0 0x7f0630072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.449+0000 7f06367f4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 0x7f063006dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.449+0000 7f06367f4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063014ee60 con 0x7f063006d040 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.449+0000 7f06367f4700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06300c2290 con 0x7f06301549a0 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.449+0000 7f06367f4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063006e2b0 con 0x7f063006d900 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.449+0000 7f062ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06301549a0 0x7f0630072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.449+0000 7f062ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06301549a0 0x7f0630072410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48666/0 (socket says 172.21.15.31:48666) 2024-03-30T01:43:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.449+0000 7f062ffff700 1 -- 172.21.15.31:0/2231574876 learned_addr learned my addr 172.21.15.31:0/2231574876 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.450+0000 7f0634d91700 1 --2- 172.21.15.31:0/2231574876 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 0x7f063006dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.450+0000 7f062ffff700 1 -- 172.21.15.31:0/2231574876 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 msgr2=0x7f063006dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.450+0000 7f062ffff700 1 --2- 172.21.15.31:0/2231574876 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 0x7f063006dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.450+0000 7f062ffff700 1 -- 172.21.15.31:0/2231574876 >> v1:172.21.15.59:6789/0 conn(0x7f063006d040 legacy=0x7f0630154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.450+0000 7f062ffff700 1 -- 172.21.15.31:0/2231574876 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630155490 con 0x7f06301549a0 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.450+0000 7f062ffff700 1 --2- 172.21.15.31:0/2231574876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301549a0 0x7f0630072410 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f062000bb90 tx=0x7f062000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70ee5ab49d89329 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.450+0000 7f062effd700 1 -- 172.21.15.31:0/2231574876 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062000dd10 con 0x7f06301549a0 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.450+0000 7f062effd700 1 -- 172.21.15.31:0/2231574876 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062000de70 con 0x7f06301549a0 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.450+0000 7f062effd700 1 -- 172.21.15.31:0/2231574876 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620004230 con 0x7f06301549a0 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.451+0000 7f06367f4700 1 -- 172.21.15.31:0/2231574876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301549a0 msgr2=0x7f0630072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.451+0000 7f06367f4700 1 --2- 172.21.15.31:0/2231574876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301549a0 0x7f0630072410 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f062000bb90 tx=0x7f062000bea0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.451+0000 7f06367f4700 1 -- 172.21.15.31:0/2231574876 shutdown_connections 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.451+0000 7f06367f4700 1 --2- 172.21.15.31:0/2231574876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06301549a0 0x7f0630072410 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.451+0000 7f06367f4700 1 --2- 172.21.15.31:0/2231574876 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 0x7f063006dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.451+0000 7f06367f4700 1 -- 172.21.15.31:0/2231574876 >> 172.21.15.31:0/2231574876 conn(0x7f06300c3980 msgr2=0x7f06300c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.451+0000 7f06367f4700 1 -- 172.21.15.31:0/2231574876 shutdown_connections 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.451+0000 7f06367f4700 1 -- 172.21.15.31:0/2231574876 wait complete. 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.452+0000 7f06367f4700 1 Processor -- start 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.452+0000 7f06367f4700 1 -- start start 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f06367f4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f063006d040 0x7f06300c0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f06367f4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 0x7f06301447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f06367f4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06301549a0 0x7f0630144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f06367f4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630152560 con 0x7f063006d040 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f06367f4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630146320 con 0x7f06301549a0 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f06367f4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0630146800 con 0x7f063006d900 2024-03-30T01:43:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f063006d040 0x7f06300c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f063006d040 0x7f06300c0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48668/0 (socket says 172.21.15.31:48668) 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062ffff700 1 -- 172.21.15.31:0/3444234802 learned_addr learned my addr 172.21.15.31:0/3444234802 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f0634d91700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06301549a0 0x7f0630144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062f7fe700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 0x7f06301447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062ffff700 1 -- 172.21.15.31:0/3444234802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 msgr2=0x7f06301447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062ffff700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 0x7f06301447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062ffff700 1 -- 172.21.15.31:0/3444234802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06301549a0 msgr2=0x7f0630144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062ffff700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06301549a0 0x7f0630144cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062ffff700 1 -- 172.21.15.31:0/3444234802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062000b7f0 con 0x7f063006d040 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062f7fe700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 0x7f06301447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.453+0000 7f062ffff700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f063006d040 0x7f06300c0500 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0620000c00 tx=0x7f0620004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.454+0000 7f062cff9700 1 -- 172.21.15.31:0/3444234802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620010670 con 0x7f063006d040 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.454+0000 7f062cff9700 1 -- 172.21.15.31:0/3444234802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06200107d0 con 0x7f063006d040 2024-03-30T01:43:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.454+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06300abd40 con 0x7f063006d040 2024-03-30T01:43:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.454+0000 7f062cff9700 1 -- 172.21.15.31:0/3444234802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620010b60 con 0x7f063006d040 2024-03-30T01:43:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.454+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06301f2830 con 0x7f063006d040 2024-03-30T01:43:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.455+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06300c4870 con 0x7f063006d040 2024-03-30T01:43:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.455+0000 7f062cff9700 1 -- 172.21.15.31:0/3444234802 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0620006bd0 con 0x7f063006d040 2024-03-30T01:43:27.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.456+0000 7f062cff9700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f061c06ac90 0x7f061c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:27.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.456+0000 7f062cff9700 1 -- 172.21.15.31:0/3444234802 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f062008b990 con 0x7f063006d040 2024-03-30T01:43:27.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.456+0000 7f062f7fe700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f061c06ac90 0x7f061c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:27.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.456+0000 7f062f7fe700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f061c06ac90 0x7f061c06d150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0630154df0 tx=0x7f0624009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:27.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.461+0000 7f062cff9700 1 -- 172.21.15.31:0/3444234802 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0620056760 con 0x7f063006d040 2024-03-30T01:43:27.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:27.726+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f06301f23b0 con 0x7f063006d040 2024-03-30T01:43:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.289+0000 7f062cff9700 1 -- 172.21.15.31:0/3444234802 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v36) v1 ==== 123+0+0 (secure 0 0 0) 0x7f06301f23b0 con 0x7f063006d040 2024-03-30T01:43:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f061c06ac90 msgr2=0x7f061c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f061c06ac90 0x7f061c06d150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0630154df0 tx=0x7f0624009040 comp rx=0 tx=0).stop 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f063006d040 msgr2=0x7f06300c0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f063006d040 0x7f06300c0500 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0620000c00 tx=0x7f0620004750 comp rx=0 tx=0).stop 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 shutdown_connections 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f063006d040 0x7f06300c0500 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f061c06ac90 0x7f061c06d150 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06301549a0 0x7f0630144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 --2- 172.21.15.31:0/3444234802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f063006d900 0x7f06301447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 >> 172.21.15.31:0/3444234802 conn(0x7f06300c3980 msgr2=0x7f063006af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 shutdown_connections 2024-03-30T01:43:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.292+0000 7f06367f4700 1 -- 172.21.15.31:0/3444234802 wait complete. 2024-03-30T01:43:28.314 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-30T01:43:28.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T01:43:28.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:43:28.438 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:43:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.448+0000 7f37b09c8700 1 Processor -- start 2024-03-30T01:43:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.448+0000 7f37b09c8700 1 -- start start 2024-03-30T01:43:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37b09c8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f37ac06a970 0x7f37ac14f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37b09c8700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f37ac14fdd0 0x7f37ac1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37b09c8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac06b630 con 0x7f37ac072360 2024-03-30T01:43:28.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37b09c8700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac155c40 con 0x7f37ac14fdd0 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37b09c8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac06d5e0 con 0x7f37ac06a970 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37a1d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f37ac14fdd0 0x7f37ac1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37a1d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f37ac14fdd0 0x7f37ac1560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48672/0 (socket says 172.21.15.31:48672) 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37a1d9b700 1 -- 172.21.15.31:0/311219205 learned_addr learned my addr 172.21.15.31:0/311219205 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37aa59c700 1 --2- 172.21.15.31:0/311219205 >> v2:172.21.15.59:3300/0 conn(0x7f37ac06a970 0x7f37ac14f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37a1d9b700 1 -- 172.21.15.31:0/311219205 >> v2:172.21.15.59:3300/0 conn(0x7f37ac06a970 msgr2=0x7f37ac14f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37a1d9b700 1 --2- 172.21.15.31:0/311219205 >> v2:172.21.15.59:3300/0 conn(0x7f37ac06a970 0x7f37ac14f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37a1d9b700 1 -- 172.21.15.31:0/311219205 >> v1:172.21.15.59:6789/0 conn(0x7f37ac072360 legacy=0x7f37ac156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.449+0000 7f37a1d9b700 1 -- 172.21.15.31:0/311219205 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0c1310 con 0x7f37ac14fdd0 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37a1d9b700 1 --2- 172.21.15.31:0/311219205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac14fdd0 0x7f37ac1560d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f37ac072b50 tx=0x7f3794009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c6f2199768c8c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37a9d9b700 1 -- 172.21.15.31:0/311219205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794011830 con 0x7f37ac14fdd0 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37a9d9b700 1 -- 172.21.15.31:0/311219205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3794011990 con 0x7f37ac14fdd0 2024-03-30T01:43:28.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37a9d9b700 1 -- 172.21.15.31:0/311219205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794011d20 con 0x7f37ac14fdd0 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37b09c8700 1 -- 172.21.15.31:0/311219205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac14fdd0 msgr2=0x7f37ac1560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37b09c8700 1 --2- 172.21.15.31:0/311219205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac14fdd0 0x7f37ac1560d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f37ac072b50 tx=0x7f3794009bd0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37b09c8700 1 -- 172.21.15.31:0/311219205 shutdown_connections 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37b09c8700 1 --2- 172.21.15.31:0/311219205 >> v2:172.21.15.59:3300/0 conn(0x7f37ac06a970 0x7f37ac14f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37b09c8700 1 --2- 172.21.15.31:0/311219205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac14fdd0 0x7f37ac1560d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.450+0000 7f37b09c8700 1 -- 172.21.15.31:0/311219205 >> 172.21.15.31:0/311219205 conn(0x7f37ac0c3a60 msgr2=0x7f37ac0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.451+0000 7f37b09c8700 1 -- 172.21.15.31:0/311219205 shutdown_connections 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.451+0000 7f37b09c8700 1 -- 172.21.15.31:0/311219205 wait complete. 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.452+0000 7f37b09c8700 1 Processor -- start 2024-03-30T01:43:28.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.452+0000 7f37b09c8700 1 -- start start 2024-03-30T01:43:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37b09c8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac06a970 0x7f37ac154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37b09c8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37ac072360 0x7f37ac144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37b09c8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37ac14fdd0 0x7f37ac150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37b09c8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac151950 con 0x7f37ac06a970 2024-03-30T01:43:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37b09c8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac1505c0 con 0x7f37ac14fdd0 2024-03-30T01:43:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37b09c8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac1552a0 con 0x7f37ac072360 2024-03-30T01:43:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37aa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac06a970 0x7f37ac154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37aa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac06a970 0x7f37ac154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48688/0 (socket says 172.21.15.31:48688) 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37aa59c700 1 -- 172.21.15.31:0/4105348312 learned_addr learned my addr 172.21.15.31:0/4105348312 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37aad9d700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37ac14fdd0 0x7f37ac150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.453+0000 7f37a1d9b700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37ac072360 0x7f37ac144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37aa59c700 1 -- 172.21.15.31:0/4105348312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37ac072360 msgr2=0x7f37ac144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37aa59c700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37ac072360 0x7f37ac144930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37aa59c700 1 -- 172.21.15.31:0/4105348312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37ac14fdd0 msgr2=0x7f37ac150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37aa59c700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37ac14fdd0 0x7f37ac150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37aa59c700 1 -- 172.21.15.31:0/4105348312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3794009550 con 0x7f37ac06a970 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37aa59c700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac06a970 0x7f37ac154190 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e9b0 tx=0x7f379c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37a37fe700 1 -- 172.21.15.31:0/4105348312 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c009710 con 0x7f37ac06a970 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37a37fe700 1 -- 172.21.15.31:0/4105348312 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00ee50 con 0x7f37ac06a970 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac154a10 con 0x7f37ac06a970 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37a37fe700 1 -- 172.21.15.31:0/4105348312 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00ca50 con 0x7f37ac06a970 2024-03-30T01:43:28.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.454+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac154ce0 con 0x7f37ac06a970 2024-03-30T01:43:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.455+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac1508a0 con 0x7f37ac06a970 2024-03-30T01:43:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.456+0000 7f37a37fe700 1 -- 172.21.15.31:0/4105348312 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f379c00cbb0 con 0x7f37ac06a970 2024-03-30T01:43:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.456+0000 7f37a37fe700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f378c06ac90 0x7f378c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.457+0000 7f37a37fe700 1 -- 172.21.15.31:0/4105348312 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f379c0896e0 con 0x7f37ac06a970 2024-03-30T01:43:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.457+0000 7f37a1d9b700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f378c06ac90 0x7f378c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.457+0000 7f37a1d9b700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f378c06ac90 0x7f378c06d150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3794014040 tx=0x7f3794008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:28.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.461+0000 7f37a37fe700 1 -- 172.21.15.31:0/4105348312 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f379c0544b0 con 0x7f37ac06a970 2024-03-30T01:43:28.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.729+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37ac04ea80 con 0x7f37ac06a970 2024-03-30T01:43:28.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.729+0000 7f37a37fe700 1 -- 172.21.15.31:0/4105348312 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5945 (secure 0 0 0) 0x7f379c00ce60 con 0x7f37ac06a970 2024-03-30T01:43:28.736 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:28.736 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5830,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6836/3969533250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6835","nonce":3969533250},{"type":"v1","addr":"172.21.15.31:6836","nonce":3969533250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3308862528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3308862528},{"type":"v1","addr":"172.21.15.59:6835","nonce":3308862528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:43:12.245458+0000","modified":"2024-03-30T01:43:15.261882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6000":{"gid":6000,"name":"d","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3498371628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3498371628},{"type":"v1","addr":"172.21.15.59:6834","nonce":3498371628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T01:43:25.277936+0000","modified":"2024-03-30T01:43:28.286200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":6,"addr":"172.21.15.31:6837/3281184299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3281184299},{"type":"v1","addr":"172.21.15.31:6837","nonce":3281184299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-30T01:43:28.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f378c06ac90 msgr2=0x7f378c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f378c06ac90 0x7f378c06d150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3794014040 tx=0x7f3794008030 comp rx=0 tx=0).stop 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac06a970 msgr2=0x7f37ac154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac06a970 0x7f37ac154190 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e9b0 tx=0x7f379c00ecc0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 shutdown_connections 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37ac06a970 0x7f37ac154190 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f378c06ac90 0x7f378c06d150 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37ac14fdd0 0x7f37ac150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 --2- 172.21.15.31:0/4105348312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37ac072360 0x7f37ac144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 >> 172.21.15.31:0/4105348312 conn(0x7f37ac0c3a60 msgr2=0x7f37ac14d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 shutdown_connections 2024-03-30T01:43:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.732+0000 7f37b09c8700 1 -- 172.21.15.31:0/4105348312 wait complete. 2024-03-30T01:43:28.740 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 36 2024-03-30T01:43:28.752 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:28.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.885+0000 7f196750e700 1 Processor -- start 2024-03-30T01:43:28.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.885+0000 7f196750e700 1 -- start start 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.886+0000 7f196750e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1960156290 0x7f196006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.886+0000 7f196750e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19600c2060 0x7f196006d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.886+0000 7f196750e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19600c7210 con 0x7f19600a7690 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.886+0000 7f196750e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19600c7a90 con 0x7f19600c2060 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.886+0000 7f196750e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1960159b50 con 0x7f1960156290 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.886+0000 7f1965aab700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19600c2060 0x7f196006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.886+0000 7f1964aa9700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f19600a7690 0x7f19600c6180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54824/0 (socket says 172.21.15.31:54824) 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.886+0000 7f1964aa9700 1 -- 172.21.15.31:0/436318660 learned_addr learned my addr 172.21.15.31:0/436318660 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f19652aa700 1 --2- 172.21.15.31:0/436318660 >> v2:172.21.15.59:3300/0 conn(0x7f1960156290 0x7f196006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f1965aab700 1 -- 172.21.15.31:0/436318660 >> v2:172.21.15.59:3300/0 conn(0x7f1960156290 msgr2=0x7f196006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f1965aab700 1 --2- 172.21.15.31:0/436318660 >> v2:172.21.15.59:3300/0 conn(0x7f1960156290 0x7f196006cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f1965aab700 1 -- 172.21.15.31:0/436318660 >> v1:172.21.15.31:6789/0 conn(0x7f19600a7690 legacy=0x7f19600c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f1965aab700 1 -- 172.21.15.31:0/436318660 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19600c74d0 con 0x7f19600c2060 2024-03-30T01:43:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f1957fff700 1 -- 172.21.15.31:0/436318660 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213701637 0 0) 0x7f19600c7210 con 0x7f19600a7690 2024-03-30T01:43:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f1965aab700 1 --2- 172.21.15.31:0/436318660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19600c2060 0x7f196006d270 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f19600c5660 tx=0x7f195800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d849dfc252ea2e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f1957fff700 1 -- 172.21.15.31:0/436318660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195800e070 con 0x7f19600c2060 2024-03-30T01:43:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f1957fff700 1 -- 172.21.15.31:0/436318660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195800bb20 con 0x7f19600c2060 2024-03-30T01:43:28.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.887+0000 7f1957fff700 1 -- 172.21.15.31:0/436318660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958012430 con 0x7f19600c2060 2024-03-30T01:43:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.888+0000 7f196750e700 1 -- 172.21.15.31:0/436318660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19600c2060 msgr2=0x7f196006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.888+0000 7f196750e700 1 --2- 172.21.15.31:0/436318660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19600c2060 0x7f196006d270 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f19600c5660 tx=0x7f195800b330 comp rx=0 tx=0).stop 2024-03-30T01:43:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.888+0000 7f196750e700 1 -- 172.21.15.31:0/436318660 shutdown_connections 2024-03-30T01:43:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.888+0000 7f196750e700 1 --2- 172.21.15.31:0/436318660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19600c2060 0x7f196006d270 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.888+0000 7f196750e700 1 --2- 172.21.15.31:0/436318660 >> v2:172.21.15.59:3300/0 conn(0x7f1960156290 0x7f196006cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.888+0000 7f196750e700 1 -- 172.21.15.31:0/436318660 >> 172.21.15.31:0/436318660 conn(0x7f196006ad30 msgr2=0x7f19600a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:28.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.888+0000 7f196750e700 1 -- 172.21.15.31:0/436318660 shutdown_connections 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.888+0000 7f196750e700 1 -- 172.21.15.31:0/436318660 wait complete. 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.889+0000 7f196750e700 1 Processor -- start 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.889+0000 7f196750e700 1 -- start start 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f196750e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19600a7690 0x7f1960153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f196750e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19600c2060 0x7f196006f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f196750e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1960156290 0x7f196006fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f196750e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19600ab920 con 0x7f1960156290 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f196750e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1960158340 con 0x7f19600a7690 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f196750e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19601ee290 con 0x7f19600c2060 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f1965aab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1960156290 0x7f196006fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f1965aab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1960156290 0x7f196006fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48712/0 (socket says 172.21.15.31:48712) 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f1965aab700 1 -- 172.21.15.31:0/4012696155 learned_addr learned my addr 172.21.15.31:0/4012696155 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f19652aa700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19600a7690 0x7f1960153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f1964aa9700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19600c2060 0x7f196006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f1965aab700 1 -- 172.21.15.31:0/4012696155 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19600c2060 msgr2=0x7f196006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.890+0000 7f1965aab700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19600c2060 0x7f196006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.891+0000 7f1965aab700 1 -- 172.21.15.31:0/4012696155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19600a7690 msgr2=0x7f1960153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.891+0000 7f1965aab700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19600a7690 0x7f1960153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.891+0000 7f1965aab700 1 -- 172.21.15.31:0/4012696155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958009a70 con 0x7f1960156290 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.891+0000 7f1965aab700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1960156290 0x7f196006fd90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f19600c5690 tx=0x7f1958004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.891+0000 7f1955ffb700 1 -- 172.21.15.31:0/4012696155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958012430 con 0x7f1960156290 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.891+0000 7f1955ffb700 1 -- 172.21.15.31:0/4012696155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1958020070 con 0x7f1960156290 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.891+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19601ef260 con 0x7f1960156290 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.891+0000 7f1955ffb700 1 -- 172.21.15.31:0/4012696155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958004280 con 0x7f1960156290 2024-03-30T01:43:28.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.891+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19601eea60 con 0x7f1960156290 2024-03-30T01:43:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.892+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196004ea80 con 0x7f1960156290 2024-03-30T01:43:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.893+0000 7f1955ffb700 1 -- 172.21.15.31:0/4012696155 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f19580043e0 con 0x7f1960156290 2024-03-30T01:43:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.893+0000 7f1955ffb700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f194406ac90 0x7f194406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.893+0000 7f1955ffb700 1 -- 172.21.15.31:0/4012696155 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f1958022070 con 0x7f1960156290 2024-03-30T01:43:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.893+0000 7f19652aa700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f194406ac90 0x7f194406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:28.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.894+0000 7f19652aa700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f194406ac90 0x7f194406d150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f195c009b40 tx=0x7f195c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:28.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:28.898+0000 7f1955ffb700 1 -- 172.21.15.31:0/4012696155 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1958056320 con 0x7f1960156290 2024-03-30T01:43:29.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.114+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19600c4f30 con 0x7f1960156290 2024-03-30T01:43:29.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.116+0000 7f1955ffb700 1 -- 172.21.15.31:0/4012696155 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+19918 (secure 0 0 0) 0x7f1958059940 con 0x7f1960156290 2024-03-30T01:43:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:29.123 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":64,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:26.280018+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:43:09.911033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T01:43:11.216248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:43:23.783452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:43:24.238093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f194406ac90 msgr2=0x7f194406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f194406ac90 0x7f194406d150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f195c009b40 tx=0x7f195c009210 comp rx=0 tx=0).stop 2024-03-30T01:43:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1960156290 msgr2=0x7f196006fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1960156290 0x7f196006fd90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f19600c5690 tx=0x7f1958004ad0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 shutdown_connections 2024-03-30T01:43:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1960156290 0x7f196006fd90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f194406ac90 0x7f194406d150 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19600a7690 0x7f1960153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 --2- 172.21.15.31:0/4012696155 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19600c2060 0x7f196006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 >> 172.21.15.31:0/4012696155 conn(0x7f196006ad30 msgr2=0x7f1960154850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 shutdown_connections 2024-03-30T01:43:29.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.118+0000 7f196750e700 1 -- 172.21.15.31:0/4012696155 wait complete. 2024-03-30T01:43:29.139 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T01:43:29.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:43:29.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.269+0000 7ff8f71b9700 1 Processor -- start 2024-03-30T01:43:29.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.269+0000 7ff8f71b9700 1 -- start start 2024-03-30T01:43:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.270+0000 7ff8f71b9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8f006dc50 0x7ff8f00c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.270+0000 7ff8f71b9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff8f0158590 0x7ff8f00c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f71b9700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f01524e0 con 0x7ff8f006a700 2024-03-30T01:43:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f71b9700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f01550e0 con 0x7ff8f006dc50 2024-03-30T01:43:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f71b9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f01553c0 con 0x7ff8f0158590 2024-03-30T01:43:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f4f55700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8f006dc50 0x7ff8f00c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8effff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff8f006a700 0x7ff8f0154820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54832/0 (socket says 172.21.15.31:54832) 2024-03-30T01:43:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f4f55700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8f006dc50 0x7ff8f00c10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48716/0 (socket says 172.21.15.31:48716) 2024-03-30T01:43:29.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8effff700 1 -- 172.21.15.31:0/926487673 learned_addr learned my addr 172.21.15.31:0/926487673 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f5756700 1 --2- 172.21.15.31:0/926487673 >> v2:172.21.15.59:3300/0 conn(0x7ff8f0158590 0x7ff8f00c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f4f55700 1 -- 172.21.15.31:0/926487673 >> v2:172.21.15.59:3300/0 conn(0x7ff8f0158590 msgr2=0x7ff8f00c1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f4f55700 1 --2- 172.21.15.31:0/926487673 >> v2:172.21.15.59:3300/0 conn(0x7ff8f0158590 0x7ff8f00c1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f4f55700 1 -- 172.21.15.31:0/926487673 >> v1:172.21.15.31:6789/0 conn(0x7ff8f006a700 legacy=0x7ff8f0154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f4f55700 1 -- 172.21.15.31:0/926487673 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f01556a0 con 0x7ff8f006dc50 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8ef7fe700 1 -- 172.21.15.31:0/926487673 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263993154 0 0) 0x7ff8f01524e0 con 0x7ff8f006a700 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8f4f55700 1 --2- 172.21.15.31:0/926487673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f006dc50 0x7ff8f00c10c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff8f006ab90 tx=0x7ff8e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b3da31aea6bb7f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8ef7fe700 1 -- 172.21.15.31:0/926487673 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0011830 con 0x7ff8f006dc50 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.271+0000 7ff8ef7fe700 1 -- 172.21.15.31:0/926487673 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e0011990 con 0x7ff8f006dc50 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.272+0000 7ff8ef7fe700 1 -- 172.21.15.31:0/926487673 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0011d20 con 0x7ff8f006dc50 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.272+0000 7ff8f71b9700 1 -- 172.21.15.31:0/926487673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f006dc50 msgr2=0x7ff8f00c10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.272+0000 7ff8f71b9700 1 --2- 172.21.15.31:0/926487673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f006dc50 0x7ff8f00c10c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff8f006ab90 tx=0x7ff8e0009b20 comp rx=0 tx=0).stop 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.272+0000 7ff8f71b9700 1 -- 172.21.15.31:0/926487673 shutdown_connections 2024-03-30T01:43:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.272+0000 7ff8f71b9700 1 --2- 172.21.15.31:0/926487673 >> v2:172.21.15.59:3300/0 conn(0x7ff8f0158590 0x7ff8f00c1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.272+0000 7ff8f71b9700 1 --2- 172.21.15.31:0/926487673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f006dc50 0x7ff8f00c10c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.272+0000 7ff8f71b9700 1 -- 172.21.15.31:0/926487673 >> 172.21.15.31:0/926487673 conn(0x7ff8f00c3190 msgr2=0x7ff8f00c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.272+0000 7ff8f71b9700 1 -- 172.21.15.31:0/926487673 shutdown_connections 2024-03-30T01:43:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.272+0000 7ff8f71b9700 1 -- 172.21.15.31:0/926487673 wait complete. 2024-03-30T01:43:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.273+0000 7ff8f71b9700 1 Processor -- start 2024-03-30T01:43:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.273+0000 7ff8f71b9700 1 -- start start 2024-03-30T01:43:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.274+0000 7ff8f71b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f006a700 0x7ff8f0153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.274+0000 7ff8f71b9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8f006dc50 0x7ff8f014d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.274+0000 7ff8f71b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f0158590 0x7ff8f0148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.274+0000 7ff8f71b9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00c4420 con 0x7ff8f0158590 2024-03-30T01:43:29.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.274+0000 7ff8f71b9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00ab4b0 con 0x7ff8f006a700 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.274+0000 7ff8f71b9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00ab020 con 0x7ff8f006dc50 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f5756700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f0158590 0x7ff8f0148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f5756700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f0158590 0x7ff8f0148f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48720/0 (socket says 172.21.15.31:48720) 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f5756700 1 -- 172.21.15.31:0/4056306654 learned_addr learned my addr 172.21.15.31:0/4056306654 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8effff700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8f006dc50 0x7ff8f014d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f4f55700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f006a700 0x7ff8f0153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f5756700 1 -- 172.21.15.31:0/4056306654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8f006dc50 msgr2=0x7ff8f014d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f5756700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8f006dc50 0x7ff8f014d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f5756700 1 -- 172.21.15.31:0/4056306654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f006a700 msgr2=0x7ff8f0153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f5756700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f006a700 0x7ff8f0153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f5756700 1 -- 172.21.15.31:0/4056306654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e00094a0 con 0x7ff8f0158590 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f5756700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f0158590 0x7ff8f0148f60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009dd0 tx=0x7ff8e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8ed7fa700 1 -- 172.21.15.31:0/4056306654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8009c40 con 0x7ff8f0158590 2024-03-30T01:43:29.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8ed7fa700 1 -- 172.21.15.31:0/4056306654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e800f040 con 0x7ff8f0158590 2024-03-30T01:43:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8ed7fa700 1 -- 172.21.15.31:0/4056306654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800ceb0 con 0x7ff8f0158590 2024-03-30T01:43:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f0153c50 con 0x7ff8f0158590 2024-03-30T01:43:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.275+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f0152830 con 0x7ff8f0158590 2024-03-30T01:43:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.277+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f0152690 con 0x7ff8f0158590 2024-03-30T01:43:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.277+0000 7ff8ed7fa700 1 -- 172.21.15.31:0/4056306654 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff8e8014480 con 0x7ff8f0158590 2024-03-30T01:43:29.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.277+0000 7ff8ed7fa700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8dc06ac90 0x7ff8dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.277+0000 7ff8ed7fa700 1 -- 172.21.15.31:0/4056306654 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7ff8e808a360 con 0x7ff8f0158590 2024-03-30T01:43:29.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.278+0000 7ff8f4f55700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8dc06ac90 0x7ff8dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.278+0000 7ff8f4f55700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8dc06ac90 0x7ff8dc06d150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00090b0 tx=0x7ff8e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:29.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.282+0000 7ff8ed7fa700 1 -- 172.21.15.31:0/4056306654 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff8f0152690 con 0x7ff8f0158590 2024-03-30T01:43:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.491+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8f0158070 con 0x7ff8dc06ac90 2024-03-30T01:43:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.505+0000 7ff8ed7fa700 1 -- 172.21.15.31:0/4056306654 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+512925 (secure 0 0 0) 0x7ff8e0011c00 con 0x7ff8dc06ac90 2024-03-30T01:43:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8dc06ac90 msgr2=0x7ff8dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8dc06ac90 0x7ff8dc06d150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00090b0 tx=0x7ff8e0008030 comp rx=0 tx=0).stop 2024-03-30T01:43:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f0158590 msgr2=0x7ff8f0148f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f0158590 0x7ff8f0148f60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009dd0 tx=0x7ff8e800c3a0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 shutdown_connections 2024-03-30T01:43:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f0158590 0x7ff8f0148f60 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ff8dc06ac90 0x7ff8dc06d150 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8f006a700 0x7ff8f0153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 --2- 172.21.15.31:0/4056306654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8f006dc50 0x7ff8f014d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 >> 172.21.15.31:0/4056306654 conn(0x7ff8f00c3190 msgr2=0x7ff8f00bf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 shutdown_connections 2024-03-30T01:43:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.509+0000 7ff8f71b9700 1 -- 172.21.15.31:0/4056306654 wait complete. 2024-03-30T01:43:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:43:29.535 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-03-30T01:43:27.841699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594241,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223052,"kb_used_data":6884,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751668,"statfs":{"total":773094113280,"available":772865708032,"internally_reserved":0,"allocated":7049216,"data_stored":3851817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.034051"},"pg_stats":[{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884605+0000","last_change":"2024-03-30T01:43:12.269722+0000","last_active":"2024-03-30T01:43:12.884605+0000","last_peered":"2024-03-30T01:43:12.884605+0000","last_clean":"2024-03-30T01:43:12.884605+0000","last_became_active":"2024-03-30T01:43:12.269445+0000","last_became_peered":"2024-03-30T01:43:12.269445+0000","last_unstale":"2024-03-30T01:43:12.884605+0000","last_undegraded":"2024-03-30T01:43:12.884605+0000","last_fullsized":"2024-03-30T01:43:12.884605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884588+0000","last_change":"2024-03-30T01:43:12.261636+0000","last_active":"2024-03-30T01:43:12.884588+0000","last_peered":"2024-03-30T01:43:12.884588+0000","last_clean":"2024-03-30T01:43:12.884588+0000","last_became_active":"2024-03-30T01:43:12.261289+0000","last_became_peered":"2024-03-30T01:43:12.261289+0000","last_unstale":"2024-03-30T01:43:12.884588+0000","last_undegraded":"2024-03-30T01:43:12.884588+0000","last_fullsized":"2024-03-30T01:43:12.884588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:53.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881963+0000","last_change":"2024-03-30T01:43:12.273399+0000","last_active":"2024-03-30T01:43:12.881963+0000","last_peered":"2024-03-30T01:43:12.881963+0000","last_clean":"2024-03-30T01:43:12.881963+0000","last_became_active":"2024-03-30T01:43:12.273099+0000","last_became_peered":"2024-03-30T01:43:12.273099+0000","last_unstale":"2024-03-30T01:43:12.881963+0000","last_undegraded":"2024-03-30T01:43:12.881963+0000","last_fullsized":"2024-03-30T01:43:12.881963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881572+0000","last_change":"2024-03-30T01:43:12.277385+0000","last_active":"2024-03-30T01:43:12.881572+0000","last_peered":"2024-03-30T01:43:12.881572+0000","last_clean":"2024-03-30T01:43:12.881572+0000","last_became_active":"2024-03-30T01:43:12.277091+0000","last_became_peered":"2024-03-30T01:43:12.277091+0000","last_unstale":"2024-03-30T01:43:12.881572+0000","last_undegraded":"2024-03-30T01:43:12.881572+0000","last_fullsized":"2024-03-30T01:43:12.881572+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:09:35.397960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888284+0000","last_change":"2024-03-30T01:43:12.276432+0000","last_active":"2024-03-30T01:43:12.888284+0000","last_peered":"2024-03-30T01:43:12.888284+0000","last_clean":"2024-03-30T01:43:12.888284+0000","last_became_active":"2024-03-30T01:43:12.276157+0000","last_became_peered":"2024-03-30T01:43:12.276157+0000","last_unstale":"2024-03-30T01:43:12.888284+0000","last_undegraded":"2024-03-30T01:43:12.888284+0000","last_fullsized":"2024-03-30T01:43:12.888284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:56.492934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885111+0000","last_change":"2024-03-30T01:43:12.273311+0000","last_active":"2024-03-30T01:43:12.885111+0000","last_peered":"2024-03-30T01:43:12.885111+0000","last_clean":"2024-03-30T01:43:12.885111+0000","last_became_active":"2024-03-30T01:43:12.272939+0000","last_became_peered":"2024-03-30T01:43:12.272939+0000","last_unstale":"2024-03-30T01:43:12.885111+0000","last_undegraded":"2024-03-30T01:43:12.885111+0000","last_fullsized":"2024-03-30T01:43:12.885111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:38.756747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881834+0000","last_change":"2024-03-30T01:43:12.279192+0000","last_active":"2024-03-30T01:43:12.881834+0000","last_peered":"2024-03-30T01:43:12.881834+0000","last_clean":"2024-03-30T01:43:12.881834+0000","last_became_active":"2024-03-30T01:43:12.278893+0000","last_became_peered":"2024-03-30T01:43:12.278893+0000","last_unstale":"2024-03-30T01:43:12.881834+0000","last_undegraded":"2024-03-30T01:43:12.881834+0000","last_fullsized":"2024-03-30T01:43:12.881834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:21:09.466088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885435+0000","last_change":"2024-03-30T01:43:12.274078+0000","last_active":"2024-03-30T01:43:12.885435+0000","last_peered":"2024-03-30T01:43:12.885435+0000","last_clean":"2024-03-30T01:43:12.885435+0000","last_became_active":"2024-03-30T01:43:12.273703+0000","last_became_peered":"2024-03-30T01:43:12.273703+0000","last_unstale":"2024-03-30T01:43:12.885435+0000","last_undegraded":"2024-03-30T01:43:12.885435+0000","last_fullsized":"2024-03-30T01:43:12.885435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.882021+0000","last_change":"2024-03-30T01:43:12.274182+0000","last_active":"2024-03-30T01:43:12.882021+0000","last_peered":"2024-03-30T01:43:12.882021+0000","last_clean":"2024-03-30T01:43:12.882021+0000","last_became_active":"2024-03-30T01:43:12.273863+0000","last_became_peered":"2024-03-30T01:43:12.273863+0000","last_unstale":"2024-03-30T01:43:12.882021+0000","last_undegraded":"2024-03-30T01:43:12.882021+0000","last_fullsized":"2024-03-30T01:43:12.882021+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881992+0000","last_change":"2024-03-30T01:43:12.277023+0000","last_active":"2024-03-30T01:43:12.881992+0000","last_peered":"2024-03-30T01:43:12.881992+0000","last_clean":"2024-03-30T01:43:12.881992+0000","last_became_active":"2024-03-30T01:43:12.276685+0000","last_became_peered":"2024-03-30T01:43:12.276685+0000","last_unstale":"2024-03-30T01:43:12.881992+0000","last_undegraded":"2024-03-30T01:43:12.881992+0000","last_fullsized":"2024-03-30T01:43:12.881992+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:53.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881586+0000","last_change":"2024-03-30T01:43:12.273450+0000","last_active":"2024-03-30T01:43:12.881586+0000","last_peered":"2024-03-30T01:43:12.881586+0000","last_clean":"2024-03-30T01:43:12.881586+0000","last_became_active":"2024-03-30T01:43:12.273185+0000","last_became_peered":"2024-03-30T01:43:12.273185+0000","last_unstale":"2024-03-30T01:43:12.881586+0000","last_undegraded":"2024-03-30T01:43:12.881586+0000","last_fullsized":"2024-03-30T01:43:12.881586+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885537+0000","last_change":"2024-03-30T01:43:12.273238+0000","last_active":"2024-03-30T01:43:12.885537+0000","last_peered":"2024-03-30T01:43:12.885537+0000","last_clean":"2024-03-30T01:43:12.885537+0000","last_became_active":"2024-03-30T01:43:12.272970+0000","last_became_peered":"2024-03-30T01:43:12.272970+0000","last_unstale":"2024-03-30T01:43:12.885537+0000","last_undegraded":"2024-03-30T01:43:12.885537+0000","last_fullsized":"2024-03-30T01:43:12.885537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:49.877712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885514+0000","last_change":"2024-03-30T01:43:12.276660+0000","last_active":"2024-03-30T01:43:12.885514+0000","last_peered":"2024-03-30T01:43:12.885514+0000","last_clean":"2024-03-30T01:43:12.885514+0000","last_became_active":"2024-03-30T01:43:12.276437+0000","last_became_peered":"2024-03-30T01:43:12.276437+0000","last_unstale":"2024-03-30T01:43:12.885514+0000","last_undegraded":"2024-03-30T01:43:12.885514+0000","last_fullsized":"2024-03-30T01:43:12.885514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881873+0000","last_change":"2024-03-30T01:43:12.276823+0000","last_active":"2024-03-30T01:43:12.881873+0000","last_peered":"2024-03-30T01:43:12.881873+0000","last_clean":"2024-03-30T01:43:12.881873+0000","last_became_active":"2024-03-30T01:43:12.272988+0000","last_became_peered":"2024-03-30T01:43:12.272988+0000","last_unstale":"2024-03-30T01:43:12.881873+0000","last_undegraded":"2024-03-30T01:43:12.881873+0000","last_fullsized":"2024-03-30T01:43:12.881873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881845+0000","last_change":"2024-03-30T01:43:12.277931+0000","last_active":"2024-03-30T01:43:12.881845+0000","last_peered":"2024-03-30T01:43:12.881845+0000","last_clean":"2024-03-30T01:43:12.881845+0000","last_became_active":"2024-03-30T01:43:12.277601+0000","last_became_peered":"2024-03-30T01:43:12.277601+0000","last_unstale":"2024-03-30T01:43:12.881845+0000","last_undegraded":"2024-03-30T01:43:12.881845+0000","last_fullsized":"2024-03-30T01:43:12.881845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881781+0000","last_change":"2024-03-30T01:43:12.275724+0000","last_active":"2024-03-30T01:43:12.881781+0000","last_peered":"2024-03-30T01:43:12.881781+0000","last_clean":"2024-03-30T01:43:12.881781+0000","last_became_active":"2024-03-30T01:43:12.275438+0000","last_became_peered":"2024-03-30T01:43:12.275438+0000","last_unstale":"2024-03-30T01:43:12.881781+0000","last_undegraded":"2024-03-30T01:43:12.881781+0000","last_fullsized":"2024-03-30T01:43:12.881781+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:18.506294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884617+0000","last_change":"2024-03-30T01:43:12.268431+0000","last_active":"2024-03-30T01:43:12.884617+0000","last_peered":"2024-03-30T01:43:12.884617+0000","last_clean":"2024-03-30T01:43:12.884617+0000","last_became_active":"2024-03-30T01:43:12.268148+0000","last_became_peered":"2024-03-30T01:43:12.268148+0000","last_unstale":"2024-03-30T01:43:12.884617+0000","last_undegraded":"2024-03-30T01:43:12.884617+0000","last_fullsized":"2024-03-30T01:43:12.884617+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:49.877712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881624+0000","last_change":"2024-03-30T01:43:12.275632+0000","last_active":"2024-03-30T01:43:12.881624+0000","last_peered":"2024-03-30T01:43:12.881624+0000","last_clean":"2024-03-30T01:43:12.881624+0000","last_became_active":"2024-03-30T01:43:12.275329+0000","last_became_peered":"2024-03-30T01:43:12.275329+0000","last_unstale":"2024-03-30T01:43:12.881624+0000","last_undegraded":"2024-03-30T01:43:12.881624+0000","last_fullsized":"2024-03-30T01:43:12.881624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:44.982486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888324+0000","last_change":"2024-03-30T01:43:12.265240+0000","last_active":"2024-03-30T01:43:12.888324+0000","last_peered":"2024-03-30T01:43:12.888324+0000","last_clean":"2024-03-30T01:43:12.888324+0000","last_became_active":"2024-03-30T01:43:12.264980+0000","last_became_peered":"2024-03-30T01:43:12.264980+0000","last_unstale":"2024-03-30T01:43:12.888324+0000","last_undegraded":"2024-03-30T01:43:12.888324+0000","last_fullsized":"2024-03-30T01:43:12.888324+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:15:38.037358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884251+0000","last_change":"2024-03-30T01:43:12.266614+0000","last_active":"2024-03-30T01:43:12.884251+0000","last_peered":"2024-03-30T01:43:12.884251+0000","last_clean":"2024-03-30T01:43:12.884251+0000","last_became_active":"2024-03-30T01:43:12.266325+0000","last_became_peered":"2024-03-30T01:43:12.266325+0000","last_unstale":"2024-03-30T01:43:12.884251+0000","last_undegraded":"2024-03-30T01:43:12.884251+0000","last_fullsized":"2024-03-30T01:43:12.884251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:45:23.286791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885526+0000","last_change":"2024-03-30T01:43:12.269577+0000","last_active":"2024-03-30T01:43:12.885526+0000","last_peered":"2024-03-30T01:43:12.885526+0000","last_clean":"2024-03-30T01:43:12.885526+0000","last_became_active":"2024-03-30T01:43:12.269263+0000","last_became_peered":"2024-03-30T01:43:12.269263+0000","last_unstale":"2024-03-30T01:43:12.885526+0000","last_undegraded":"2024-03-30T01:43:12.885526+0000","last_fullsized":"2024-03-30T01:43:12.885526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884162+0000","last_change":"2024-03-30T01:43:12.276626+0000","last_active":"2024-03-30T01:43:12.884162+0000","last_peered":"2024-03-30T01:43:12.884162+0000","last_clean":"2024-03-30T01:43:12.884162+0000","last_became_active":"2024-03-30T01:43:12.276302+0000","last_became_peered":"2024-03-30T01:43:12.276302+0000","last_unstale":"2024-03-30T01:43:12.884162+0000","last_undegraded":"2024-03-30T01:43:12.884162+0000","last_fullsized":"2024-03-30T01:43:12.884162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:11.027051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884074+0000","last_change":"2024-03-30T01:43:12.276658+0000","last_active":"2024-03-30T01:43:12.884074+0000","last_peered":"2024-03-30T01:43:12.884074+0000","last_clean":"2024-03-30T01:43:12.884074+0000","last_became_active":"2024-03-30T01:43:12.276363+0000","last_became_peered":"2024-03-30T01:43:12.276363+0000","last_unstale":"2024-03-30T01:43:12.884074+0000","last_undegraded":"2024-03-30T01:43:12.884074+0000","last_fullsized":"2024-03-30T01:43:12.884074+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:06.329557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881811+0000","last_change":"2024-03-30T01:43:12.273270+0000","last_active":"2024-03-30T01:43:12.881811+0000","last_peered":"2024-03-30T01:43:12.881811+0000","last_clean":"2024-03-30T01:43:12.881811+0000","last_became_active":"2024-03-30T01:43:12.272973+0000","last_became_peered":"2024-03-30T01:43:12.272973+0000","last_unstale":"2024-03-30T01:43:12.881811+0000","last_undegraded":"2024-03-30T01:43:12.881811+0000","last_fullsized":"2024-03-30T01:43:12.881811+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885590+0000","last_change":"2024-03-30T01:43:12.270679+0000","last_active":"2024-03-30T01:43:12.885590+0000","last_peered":"2024-03-30T01:43:12.885590+0000","last_clean":"2024-03-30T01:43:12.885590+0000","last_became_active":"2024-03-30T01:43:12.270457+0000","last_became_peered":"2024-03-30T01:43:12.270457+0000","last_unstale":"2024-03-30T01:43:12.885590+0000","last_undegraded":"2024-03-30T01:43:12.885590+0000","last_fullsized":"2024-03-30T01:43:12.885590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:45.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885551+0000","last_change":"2024-03-30T01:43:12.269460+0000","last_active":"2024-03-30T01:43:12.885551+0000","last_peered":"2024-03-30T01:43:12.885551+0000","last_clean":"2024-03-30T01:43:12.885551+0000","last_became_active":"2024-03-30T01:43:12.269159+0000","last_became_peered":"2024-03-30T01:43:12.269159+0000","last_unstale":"2024-03-30T01:43:12.885551+0000","last_undegraded":"2024-03-30T01:43:12.885551+0000","last_fullsized":"2024-03-30T01:43:12.885551+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881601+0000","last_change":"2024-03-30T01:43:12.276051+0000","last_active":"2024-03-30T01:43:12.881601+0000","last_peered":"2024-03-30T01:43:12.881601+0000","last_clean":"2024-03-30T01:43:12.881601+0000","last_became_active":"2024-03-30T01:43:12.275296+0000","last_became_peered":"2024-03-30T01:43:12.275296+0000","last_unstale":"2024-03-30T01:43:12.881601+0000","last_undegraded":"2024-03-30T01:43:12.881601+0000","last_fullsized":"2024-03-30T01:43:12.881601+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:17.176008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884534+0000","last_change":"2024-03-30T01:43:12.266882+0000","last_active":"2024-03-30T01:43:12.884534+0000","last_peered":"2024-03-30T01:43:12.884534+0000","last_clean":"2024-03-30T01:43:12.884534+0000","last_became_active":"2024-03-30T01:43:12.266450+0000","last_became_peered":"2024-03-30T01:43:12.266450+0000","last_unstale":"2024-03-30T01:43:12.884534+0000","last_undegraded":"2024-03-30T01:43:12.884534+0000","last_fullsized":"2024-03-30T01:43:12.884534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881894+0000","last_change":"2024-03-30T01:43:12.279605+0000","last_active":"2024-03-30T01:43:12.881894+0000","last_peered":"2024-03-30T01:43:12.881894+0000","last_clean":"2024-03-30T01:43:12.881894+0000","last_became_active":"2024-03-30T01:43:12.279323+0000","last_became_peered":"2024-03-30T01:43:12.279323+0000","last_unstale":"2024-03-30T01:43:12.881894+0000","last_undegraded":"2024-03-30T01:43:12.881894+0000","last_fullsized":"2024-03-30T01:43:12.881894+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:32:07.610941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885474+0000","last_change":"2024-03-30T01:43:12.277845+0000","last_active":"2024-03-30T01:43:12.885474+0000","last_peered":"2024-03-30T01:43:12.885474+0000","last_clean":"2024-03-30T01:43:12.885474+0000","last_became_active":"2024-03-30T01:43:12.277482+0000","last_became_peered":"2024-03-30T01:43:12.277482+0000","last_unstale":"2024-03-30T01:43:12.885474+0000","last_undegraded":"2024-03-30T01:43:12.885474+0000","last_fullsized":"2024-03-30T01:43:12.885474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881547+0000","last_change":"2024-03-30T01:43:12.276064+0000","last_active":"2024-03-30T01:43:12.881547+0000","last_peered":"2024-03-30T01:43:12.881547+0000","last_clean":"2024-03-30T01:43:12.881547+0000","last_became_active":"2024-03-30T01:43:12.275670+0000","last_became_peered":"2024-03-30T01:43:12.275670+0000","last_unstale":"2024-03-30T01:43:12.881547+0000","last_undegraded":"2024-03-30T01:43:12.881547+0000","last_fullsized":"2024-03-30T01:43:12.881547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:45.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885072+0000","last_change":"2024-03-30T01:43:12.260768+0000","last_active":"2024-03-30T01:43:12.885072+0000","last_peered":"2024-03-30T01:43:12.885072+0000","last_clean":"2024-03-30T01:43:12.885072+0000","last_became_active":"2024-03-30T01:43:12.260512+0000","last_became_peered":"2024-03-30T01:43:12.260512+0000","last_unstale":"2024-03-30T01:43:12.885072+0000","last_undegraded":"2024-03-30T01:43:12.885072+0000","last_fullsized":"2024-03-30T01:43:12.885072+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:59.579718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888309+0000","last_change":"2024-03-30T01:43:12.270602+0000","last_active":"2024-03-30T01:43:12.888309+0000","last_peered":"2024-03-30T01:43:12.888309+0000","last_clean":"2024-03-30T01:43:12.888309+0000","last_became_active":"2024-03-30T01:43:12.270328+0000","last_became_peered":"2024-03-30T01:43:12.270328+0000","last_unstale":"2024-03-30T01:43:12.888309+0000","last_undegraded":"2024-03-30T01:43:12.888309+0000","last_fullsized":"2024-03-30T01:43:12.888309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:24.040846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884627+0000","last_change":"2024-03-30T01:43:11.800815+0000","last_active":"2024-03-30T01:43:12.884627+0000","last_peered":"2024-03-30T01:43:12.884627+0000","last_clean":"2024-03-30T01:43:12.884627+0000","last_became_active":"2024-03-30T01:43:11.800527+0000","last_became_peered":"2024-03-30T01:43:11.800527+0000","last_unstale":"2024-03-30T01:43:12.884627+0000","last_undegraded":"2024-03-30T01:43:12.884627+0000","last_fullsized":"2024-03-30T01:43:12.884627+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:36.821233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881613+0000","last_change":"2024-03-30T01:43:12.268225+0000","last_active":"2024-03-30T01:43:12.881613+0000","last_peered":"2024-03-30T01:43:12.881613+0000","last_clean":"2024-03-30T01:43:12.881613+0000","last_became_active":"2024-03-30T01:43:12.267769+0000","last_became_peered":"2024-03-30T01:43:12.267769+0000","last_unstale":"2024-03-30T01:43:12.881613+0000","last_undegraded":"2024-03-30T01:43:12.881613+0000","last_fullsized":"2024-03-30T01:43:12.881613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885190+0000","last_change":"2024-03-30T01:43:11.806420+0000","last_active":"2024-03-30T01:43:12.885190+0000","last_peered":"2024-03-30T01:43:12.885190+0000","last_clean":"2024-03-30T01:43:12.885190+0000","last_became_active":"2024-03-30T01:43:11.806144+0000","last_became_peered":"2024-03-30T01:43:11.806144+0000","last_unstale":"2024-03-30T01:43:12.885190+0000","last_undegraded":"2024-03-30T01:43:12.885190+0000","last_fullsized":"2024-03-30T01:43:12.885190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:19.302319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885179+0000","last_change":"2024-03-30T01:43:12.259615+0000","last_active":"2024-03-30T01:43:12.885179+0000","last_peered":"2024-03-30T01:43:12.885179+0000","last_clean":"2024-03-30T01:43:12.885179+0000","last_became_active":"2024-03-30T01:43:12.259331+0000","last_became_peered":"2024-03-30T01:43:12.259331+0000","last_unstale":"2024-03-30T01:43:12.885179+0000","last_undegraded":"2024-03-30T01:43:12.885179+0000","last_fullsized":"2024-03-30T01:43:12.885179+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:10.944027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885158+0000","last_change":"2024-03-30T01:43:11.811391+0000","last_active":"2024-03-30T01:43:12.885158+0000","last_peered":"2024-03-30T01:43:12.885158+0000","last_clean":"2024-03-30T01:43:12.885158+0000","last_became_active":"2024-03-30T01:43:11.810873+0000","last_became_peered":"2024-03-30T01:43:11.810873+0000","last_unstale":"2024-03-30T01:43:12.885158+0000","last_undegraded":"2024-03-30T01:43:12.885158+0000","last_fullsized":"2024-03-30T01:43:12.885158+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:22.774118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.882035+0000","last_change":"2024-03-30T01:43:12.276578+0000","last_active":"2024-03-30T01:43:12.882035+0000","last_peered":"2024-03-30T01:43:12.882035+0000","last_clean":"2024-03-30T01:43:12.882035+0000","last_became_active":"2024-03-30T01:43:12.276293+0000","last_became_peered":"2024-03-30T01:43:12.276293+0000","last_unstale":"2024-03-30T01:43:12.882035+0000","last_undegraded":"2024-03-30T01:43:12.882035+0000","last_fullsized":"2024-03-30T01:43:12.882035+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885146+0000","last_change":"2024-03-30T01:43:11.803525+0000","last_active":"2024-03-30T01:43:12.885146+0000","last_peered":"2024-03-30T01:43:12.885146+0000","last_clean":"2024-03-30T01:43:12.885146+0000","last_became_active":"2024-03-30T01:43:11.803115+0000","last_became_peered":"2024-03-30T01:43:11.803115+0000","last_unstale":"2024-03-30T01:43:12.885146+0000","last_undegraded":"2024-03-30T01:43:12.885146+0000","last_fullsized":"2024-03-30T01:43:12.885146+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:10.022632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881813+0000","last_change":"2024-03-30T01:43:12.267451+0000","last_active":"2024-03-30T01:43:12.881813+0000","last_peered":"2024-03-30T01:43:12.881813+0000","last_clean":"2024-03-30T01:43:12.881813+0000","last_became_active":"2024-03-30T01:43:12.267249+0000","last_became_peered":"2024-03-30T01:43:12.267249+0000","last_unstale":"2024-03-30T01:43:12.881813+0000","last_undegraded":"2024-03-30T01:43:12.881813+0000","last_fullsized":"2024-03-30T01:43:12.881813+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:49.333290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884227+0000","last_change":"2024-03-30T01:43:11.804981+0000","last_active":"2024-03-30T01:43:12.884227+0000","last_peered":"2024-03-30T01:43:12.884227+0000","last_clean":"2024-03-30T01:43:12.884227+0000","last_became_active":"2024-03-30T01:43:11.804690+0000","last_became_peered":"2024-03-30T01:43:11.804690+0000","last_unstale":"2024-03-30T01:43:12.884227+0000","last_undegraded":"2024-03-30T01:43:12.884227+0000","last_fullsized":"2024-03-30T01:43:12.884227+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:25:59.487668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884491+0000","last_change":"2024-03-30T01:43:12.268426+0000","last_active":"2024-03-30T01:43:12.884491+0000","last_peered":"2024-03-30T01:43:12.884491+0000","last_clean":"2024-03-30T01:43:12.884491+0000","last_became_active":"2024-03-30T01:43:12.268106+0000","last_became_peered":"2024-03-30T01:43:12.268106+0000","last_unstale":"2024-03-30T01:43:12.884491+0000","last_undegraded":"2024-03-30T01:43:12.884491+0000","last_fullsized":"2024-03-30T01:43:12.884491+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:32:07.610941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884138+0000","last_change":"2024-03-30T01:43:11.798840+0000","last_active":"2024-03-30T01:43:12.884138+0000","last_peered":"2024-03-30T01:43:12.884138+0000","last_clean":"2024-03-30T01:43:12.884138+0000","last_became_active":"2024-03-30T01:43:11.798493+0000","last_became_peered":"2024-03-30T01:43:11.798493+0000","last_unstale":"2024-03-30T01:43:12.884138+0000","last_undegraded":"2024-03-30T01:43:12.884138+0000","last_fullsized":"2024-03-30T01:43:12.884138+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:35.032315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888247+0000","last_change":"2024-03-30T01:43:12.272426+0000","last_active":"2024-03-30T01:43:12.888247+0000","last_peered":"2024-03-30T01:43:12.888247+0000","last_clean":"2024-03-30T01:43:12.888247+0000","last_became_active":"2024-03-30T01:43:12.272160+0000","last_became_peered":"2024-03-30T01:43:12.272160+0000","last_unstale":"2024-03-30T01:43:12.888247+0000","last_undegraded":"2024-03-30T01:43:12.888247+0000","last_fullsized":"2024-03-30T01:43:12.888247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:37.825651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884095+0000","last_change":"2024-03-30T01:43:11.805468+0000","last_active":"2024-03-30T01:43:12.884095+0000","last_peered":"2024-03-30T01:43:12.884095+0000","last_clean":"2024-03-30T01:43:12.884095+0000","last_became_active":"2024-03-30T01:43:11.804984+0000","last_became_peered":"2024-03-30T01:43:11.804984+0000","last_unstale":"2024-03-30T01:43:12.884095+0000","last_undegraded":"2024-03-30T01:43:12.884095+0000","last_fullsized":"2024-03-30T01:43:12.884095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:16.171589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884205+0000","last_change":"2024-03-30T01:43:12.265005+0000","last_active":"2024-03-30T01:43:12.884205+0000","last_peered":"2024-03-30T01:43:12.884205+0000","last_clean":"2024-03-30T01:43:12.884205+0000","last_became_active":"2024-03-30T01:43:12.264726+0000","last_became_peered":"2024-03-30T01:43:12.264726+0000","last_unstale":"2024-03-30T01:43:12.884205+0000","last_undegraded":"2024-03-30T01:43:12.884205+0000","last_fullsized":"2024-03-30T01:43:12.884205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:41:19.989099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888234+0000","last_change":"2024-03-30T01:43:11.799020+0000","last_active":"2024-03-30T01:43:12.888234+0000","last_peered":"2024-03-30T01:43:12.888234+0000","last_clean":"2024-03-30T01:43:12.888234+0000","last_became_active":"2024-03-30T01:43:11.797840+0000","last_became_peered":"2024-03-30T01:43:11.797840+0000","last_unstale":"2024-03-30T01:43:12.888234+0000","last_undegraded":"2024-03-30T01:43:12.888234+0000","last_fullsized":"2024-03-30T01:43:12.888234+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:48:17.963383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884552+0000","last_change":"2024-03-30T01:43:12.265237+0000","last_active":"2024-03-30T01:43:12.884552+0000","last_peered":"2024-03-30T01:43:12.884552+0000","last_clean":"2024-03-30T01:43:12.884552+0000","last_became_active":"2024-03-30T01:43:12.264995+0000","last_became_peered":"2024-03-30T01:43:12.264995+0000","last_unstale":"2024-03-30T01:43:12.884552+0000","last_undegraded":"2024-03-30T01:43:12.884552+0000","last_fullsized":"2024-03-30T01:43:12.884552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.882047+0000","last_change":"2024-03-30T01:43:11.803287+0000","last_active":"2024-03-30T01:43:12.882047+0000","last_peered":"2024-03-30T01:43:12.882047+0000","last_clean":"2024-03-30T01:43:12.882047+0000","last_became_active":"2024-03-30T01:43:11.803067+0000","last_became_peered":"2024-03-30T01:43:11.803067+0000","last_unstale":"2024-03-30T01:43:12.882047+0000","last_undegraded":"2024-03-30T01:43:12.882047+0000","last_fullsized":"2024-03-30T01:43:12.882047+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:36.821233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884295+0000","last_change":"2024-03-30T01:43:12.275223+0000","last_active":"2024-03-30T01:43:12.884295+0000","last_peered":"2024-03-30T01:43:12.884295+0000","last_clean":"2024-03-30T01:43:12.884295+0000","last_became_active":"2024-03-30T01:43:12.274949+0000","last_became_peered":"2024-03-30T01:43:12.274949+0000","last_unstale":"2024-03-30T01:43:12.884295+0000","last_undegraded":"2024-03-30T01:43:12.884295+0000","last_fullsized":"2024-03-30T01:43:12.884295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:34.024584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881648+0000","last_change":"2024-03-30T01:43:11.802972+0000","last_active":"2024-03-30T01:43:12.881648+0000","last_peered":"2024-03-30T01:43:12.881648+0000","last_clean":"2024-03-30T01:43:12.881648+0000","last_became_active":"2024-03-30T01:43:11.802665+0000","last_became_peered":"2024-03-30T01:43:11.802665+0000","last_unstale":"2024-03-30T01:43:12.881648+0000","last_undegraded":"2024-03-30T01:43:12.881648+0000","last_fullsized":"2024-03-30T01:43:12.881648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:24.494514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888347+0000","last_change":"2024-03-30T01:43:12.271580+0000","last_active":"2024-03-30T01:43:12.888347+0000","last_peered":"2024-03-30T01:43:12.888347+0000","last_clean":"2024-03-30T01:43:12.888347+0000","last_became_active":"2024-03-30T01:43:12.271342+0000","last_became_peered":"2024-03-30T01:43:12.271342+0000","last_unstale":"2024-03-30T01:43:12.888347+0000","last_undegraded":"2024-03-30T01:43:12.888347+0000","last_fullsized":"2024-03-30T01:43:12.888347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:25.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881948+0000","last_change":"2024-03-30T01:43:11.802252+0000","last_active":"2024-03-30T01:43:12.881948+0000","last_peered":"2024-03-30T01:43:12.881948+0000","last_clean":"2024-03-30T01:43:12.881948+0000","last_became_active":"2024-03-30T01:43:11.801845+0000","last_became_peered":"2024-03-30T01:43:11.801845+0000","last_unstale":"2024-03-30T01:43:12.881948+0000","last_undegraded":"2024-03-30T01:43:12.881948+0000","last_fullsized":"2024-03-30T01:43:12.881948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881660+0000","last_change":"2024-03-30T01:43:12.268100+0000","last_active":"2024-03-30T01:43:12.881660+0000","last_peered":"2024-03-30T01:43:12.881660+0000","last_clean":"2024-03-30T01:43:12.881660+0000","last_became_active":"2024-03-30T01:43:12.267755+0000","last_became_peered":"2024-03-30T01:43:12.267755+0000","last_unstale":"2024-03-30T01:43:12.881660+0000","last_undegraded":"2024-03-30T01:43:12.881660+0000","last_fullsized":"2024-03-30T01:43:12.881660+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'8","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-03-30T01:43:19.533618+0000","last_change":"2024-03-30T01:43:11.806909+0000","last_active":"2024-03-30T01:43:19.533618+0000","last_peered":"2024-03-30T01:43:19.533618+0000","last_clean":"2024-03-30T01:43:19.533618+0000","last_became_active":"2024-03-30T01:43:11.806609+0000","last_became_peered":"2024-03-30T01:43:11.806609+0000","last_unstale":"2024-03-30T01:43:19.533618+0000","last_undegraded":"2024-03-30T01:43:19.533618+0000","last_fullsized":"2024-03-30T01:43:19.533618+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881507+0000","last_change":"2024-03-30T01:43:12.280292+0000","last_active":"2024-03-30T01:43:12.881507+0000","last_peered":"2024-03-30T01:43:12.881507+0000","last_clean":"2024-03-30T01:43:12.881507+0000","last_became_active":"2024-03-30T01:43:12.279966+0000","last_became_peered":"2024-03-30T01:43:12.279966+0000","last_unstale":"2024-03-30T01:43:12.881507+0000","last_undegraded":"2024-03-30T01:43:12.881507+0000","last_fullsized":"2024-03-30T01:43:12.881507+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881907+0000","last_change":"2024-03-30T01:43:11.804635+0000","last_active":"2024-03-30T01:43:12.881907+0000","last_peered":"2024-03-30T01:43:12.881907+0000","last_clean":"2024-03-30T01:43:12.881907+0000","last_became_active":"2024-03-30T01:43:11.804355+0000","last_became_peered":"2024-03-30T01:43:11.804355+0000","last_unstale":"2024-03-30T01:43:12.881907+0000","last_undegraded":"2024-03-30T01:43:12.881907+0000","last_fullsized":"2024-03-30T01:43:12.881907+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:57.117126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885452+0000","last_change":"2024-03-30T01:43:12.268978+0000","last_active":"2024-03-30T01:43:12.885452+0000","last_peered":"2024-03-30T01:43:12.885452+0000","last_clean":"2024-03-30T01:43:12.885452+0000","last_became_active":"2024-03-30T01:43:12.268633+0000","last_became_peered":"2024-03-30T01:43:12.268633+0000","last_unstale":"2024-03-30T01:43:12.885452+0000","last_undegraded":"2024-03-30T01:43:12.885452+0000","last_fullsized":"2024-03-30T01:43:12.885452+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885489+0000","last_change":"2024-03-30T01:43:11.811100+0000","last_active":"2024-03-30T01:43:12.885489+0000","last_peered":"2024-03-30T01:43:12.885489+0000","last_clean":"2024-03-30T01:43:12.885489+0000","last_became_active":"2024-03-30T01:43:11.810811+0000","last_became_peered":"2024-03-30T01:43:11.810811+0000","last_unstale":"2024-03-30T01:43:12.885489+0000","last_undegraded":"2024-03-30T01:43:12.885489+0000","last_fullsized":"2024-03-30T01:43:12.885489+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888261+0000","last_change":"2024-03-30T01:43:12.265729+0000","last_active":"2024-03-30T01:43:12.888261+0000","last_peered":"2024-03-30T01:43:12.888261+0000","last_clean":"2024-03-30T01:43:12.888261+0000","last_became_active":"2024-03-30T01:43:12.265427+0000","last_became_peered":"2024-03-30T01:43:12.265427+0000","last_unstale":"2024-03-30T01:43:12.888261+0000","last_undegraded":"2024-03-30T01:43:12.888261+0000","last_fullsized":"2024-03-30T01:43:12.888261+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:40:39.587585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881860+0000","last_change":"2024-03-30T01:43:11.802626+0000","last_active":"2024-03-30T01:43:12.881860+0000","last_peered":"2024-03-30T01:43:12.881860+0000","last_clean":"2024-03-30T01:43:12.881860+0000","last_became_active":"2024-03-30T01:43:11.802314+0000","last_became_peered":"2024-03-30T01:43:11.802314+0000","last_unstale":"2024-03-30T01:43:12.881860+0000","last_undegraded":"2024-03-30T01:43:12.881860+0000","last_fullsized":"2024-03-30T01:43:12.881860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884517+0000","last_change":"2024-03-30T01:43:12.274115+0000","last_active":"2024-03-30T01:43:12.884517+0000","last_peered":"2024-03-30T01:43:12.884517+0000","last_clean":"2024-03-30T01:43:12.884517+0000","last_became_active":"2024-03-30T01:43:12.273842+0000","last_became_peered":"2024-03-30T01:43:12.273842+0000","last_unstale":"2024-03-30T01:43:12.884517+0000","last_undegraded":"2024-03-30T01:43:12.884517+0000","last_fullsized":"2024-03-30T01:43:12.884517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:44:49.086267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881529+0000","last_change":"2024-03-30T01:43:11.802309+0000","last_active":"2024-03-30T01:43:12.881529+0000","last_peered":"2024-03-30T01:43:12.881529+0000","last_clean":"2024-03-30T01:43:12.881529+0000","last_became_active":"2024-03-30T01:43:11.802008+0000","last_became_peered":"2024-03-30T01:43:11.802008+0000","last_unstale":"2024-03-30T01:43:12.881529+0000","last_undegraded":"2024-03-30T01:43:12.881529+0000","last_fullsized":"2024-03-30T01:43:12.881529+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:11.268414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881935+0000","last_change":"2024-03-30T01:43:12.279554+0000","last_active":"2024-03-30T01:43:12.881935+0000","last_peered":"2024-03-30T01:43:12.881935+0000","last_clean":"2024-03-30T01:43:12.881935+0000","last_became_active":"2024-03-30T01:43:12.279266+0000","last_became_peered":"2024-03-30T01:43:12.279266+0000","last_unstale":"2024-03-30T01:43:12.881935+0000","last_undegraded":"2024-03-30T01:43:12.881935+0000","last_fullsized":"2024-03-30T01:43:12.881935+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885602+0000","last_change":"2024-03-30T01:43:11.809877+0000","last_active":"2024-03-30T01:43:12.885602+0000","last_peered":"2024-03-30T01:43:12.885602+0000","last_clean":"2024-03-30T01:43:12.885602+0000","last_became_active":"2024-03-30T01:43:11.808130+0000","last_became_peered":"2024-03-30T01:43:11.808130+0000","last_unstale":"2024-03-30T01:43:12.885602+0000","last_undegraded":"2024-03-30T01:43:12.885602+0000","last_fullsized":"2024-03-30T01:43:12.885602+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888336+0000","last_change":"2024-03-30T01:43:12.275996+0000","last_active":"2024-03-30T01:43:12.888336+0000","last_peered":"2024-03-30T01:43:12.888336+0000","last_clean":"2024-03-30T01:43:12.888336+0000","last_became_active":"2024-03-30T01:43:12.275705+0000","last_became_peered":"2024-03-30T01:43:12.275705+0000","last_unstale":"2024-03-30T01:43:12.888336+0000","last_undegraded":"2024-03-30T01:43:12.888336+0000","last_fullsized":"2024-03-30T01:43:12.888336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:47.117365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.882006+0000","last_change":"2024-03-30T01:43:11.806234+0000","last_active":"2024-03-30T01:43:12.882006+0000","last_peered":"2024-03-30T01:43:12.882006+0000","last_clean":"2024-03-30T01:43:12.882006+0000","last_became_active":"2024-03-30T01:43:11.806012+0000","last_became_peered":"2024-03-30T01:43:11.806012+0000","last_unstale":"2024-03-30T01:43:12.882006+0000","last_undegraded":"2024-03-30T01:43:12.882006+0000","last_fullsized":"2024-03-30T01:43:12.882006+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:24.494514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881636+0000","last_change":"2024-03-30T01:43:12.271940+0000","last_active":"2024-03-30T01:43:12.881636+0000","last_peered":"2024-03-30T01:43:12.881636+0000","last_clean":"2024-03-30T01:43:12.881636+0000","last_became_active":"2024-03-30T01:43:12.268234+0000","last_became_peered":"2024-03-30T01:43:12.268234+0000","last_unstale":"2024-03-30T01:43:12.881636+0000","last_undegraded":"2024-03-30T01:43:12.881636+0000","last_fullsized":"2024-03-30T01:43:12.881636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881978+0000","last_change":"2024-03-30T01:43:11.803111+0000","last_active":"2024-03-30T01:43:12.881978+0000","last_peered":"2024-03-30T01:43:12.881978+0000","last_clean":"2024-03-30T01:43:12.881978+0000","last_became_active":"2024-03-30T01:43:11.802781+0000","last_became_peered":"2024-03-30T01:43:11.802781+0000","last_unstale":"2024-03-30T01:43:12.881978+0000","last_undegraded":"2024-03-30T01:43:12.881978+0000","last_fullsized":"2024-03-30T01:43:12.881978+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:30.914337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888358+0000","last_change":"2024-03-30T01:43:12.272623+0000","last_active":"2024-03-30T01:43:12.888358+0000","last_peered":"2024-03-30T01:43:12.888358+0000","last_clean":"2024-03-30T01:43:12.888358+0000","last_became_active":"2024-03-30T01:43:12.272330+0000","last_became_peered":"2024-03-30T01:43:12.272330+0000","last_unstale":"2024-03-30T01:43:12.888358+0000","last_undegraded":"2024-03-30T01:43:12.888358+0000","last_fullsized":"2024-03-30T01:43:12.888358+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:11.229993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884274+0000","last_change":"2024-03-30T01:43:11.802228+0000","last_active":"2024-03-30T01:43:12.884274+0000","last_peered":"2024-03-30T01:43:12.884274+0000","last_clean":"2024-03-30T01:43:12.884274+0000","last_became_active":"2024-03-30T01:43:11.801742+0000","last_became_peered":"2024-03-30T01:43:11.801742+0000","last_unstale":"2024-03-30T01:43:12.884274+0000","last_undegraded":"2024-03-30T01:43:12.884274+0000","last_fullsized":"2024-03-30T01:43:12.884274+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:01.225672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885088+0000","last_change":"2024-03-30T01:43:12.259920+0000","last_active":"2024-03-30T01:43:12.885088+0000","last_peered":"2024-03-30T01:43:12.885088+0000","last_clean":"2024-03-30T01:43:12.885088+0000","last_became_active":"2024-03-30T01:43:12.259636+0000","last_became_peered":"2024-03-30T01:43:12.259636+0000","last_unstale":"2024-03-30T01:43:12.885088+0000","last_undegraded":"2024-03-30T01:43:12.885088+0000","last_fullsized":"2024-03-30T01:43:12.885088+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:45:42.926912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885123+0000","last_change":"2024-03-30T01:43:11.804854+0000","last_active":"2024-03-30T01:43:12.885123+0000","last_peered":"2024-03-30T01:43:12.885123+0000","last_clean":"2024-03-30T01:43:12.885123+0000","last_became_active":"2024-03-30T01:43:11.804593+0000","last_became_peered":"2024-03-30T01:43:11.804593+0000","last_unstale":"2024-03-30T01:43:12.885123+0000","last_undegraded":"2024-03-30T01:43:12.885123+0000","last_fullsized":"2024-03-30T01:43:12.885123+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:45:22.282372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885464+0000","last_change":"2024-03-30T01:43:12.269648+0000","last_active":"2024-03-30T01:43:12.885464+0000","last_peered":"2024-03-30T01:43:12.885464+0000","last_clean":"2024-03-30T01:43:12.885464+0000","last_became_active":"2024-03-30T01:43:12.269295+0000","last_became_peered":"2024-03-30T01:43:12.269295+0000","last_unstale":"2024-03-30T01:43:12.885464+0000","last_undegraded":"2024-03-30T01:43:12.885464+0000","last_fullsized":"2024-03-30T01:43:12.885464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885500+0000","last_change":"2024-03-30T01:43:11.811090+0000","last_active":"2024-03-30T01:43:12.885500+0000","last_peered":"2024-03-30T01:43:12.885500+0000","last_clean":"2024-03-30T01:43:12.885500+0000","last_became_active":"2024-03-30T01:43:11.810767+0000","last_became_peered":"2024-03-30T01:43:11.810767+0000","last_unstale":"2024-03-30T01:43:12.885500+0000","last_undegraded":"2024-03-30T01:43:12.885500+0000","last_fullsized":"2024-03-30T01:43:12.885500+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888273+0000","last_change":"2024-03-30T01:43:12.265208+0000","last_active":"2024-03-30T01:43:12.888273+0000","last_peered":"2024-03-30T01:43:12.888273+0000","last_clean":"2024-03-30T01:43:12.888273+0000","last_became_active":"2024-03-30T01:43:12.264919+0000","last_became_peered":"2024-03-30T01:43:12.264919+0000","last_unstale":"2024-03-30T01:43:12.888273+0000","last_undegraded":"2024-03-30T01:43:12.888273+0000","last_fullsized":"2024-03-30T01:43:12.888273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:31.918755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885100+0000","last_change":"2024-03-30T01:43:11.803507+0000","last_active":"2024-03-30T01:43:12.885100+0000","last_peered":"2024-03-30T01:43:12.885100+0000","last_clean":"2024-03-30T01:43:12.885100+0000","last_became_active":"2024-03-30T01:43:11.803197+0000","last_became_peered":"2024-03-30T01:43:11.803197+0000","last_unstale":"2024-03-30T01:43:12.885100+0000","last_undegraded":"2024-03-30T01:43:12.885100+0000","last_fullsized":"2024-03-30T01:43:12.885100+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:41:18.984681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885135+0000","last_change":"2024-03-30T01:43:12.275594+0000","last_active":"2024-03-30T01:43:12.885135+0000","last_peered":"2024-03-30T01:43:12.885135+0000","last_clean":"2024-03-30T01:43:12.885135+0000","last_became_active":"2024-03-30T01:43:12.275249+0000","last_became_peered":"2024-03-30T01:43:12.275249+0000","last_unstale":"2024-03-30T01:43:12.885135+0000","last_undegraded":"2024-03-30T01:43:12.885135+0000","last_fullsized":"2024-03-30T01:43:12.885135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:09.869640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885617+0000","last_change":"2024-03-30T01:43:11.810058+0000","last_active":"2024-03-30T01:43:12.885617+0000","last_peered":"2024-03-30T01:43:12.885617+0000","last_clean":"2024-03-30T01:43:12.885617+0000","last_became_active":"2024-03-30T01:43:11.808406+0000","last_became_peered":"2024-03-30T01:43:11.808406+0000","last_unstale":"2024-03-30T01:43:12.885617+0000","last_undegraded":"2024-03-30T01:43:12.885617+0000","last_fullsized":"2024-03-30T01:43:12.885617+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:57.117126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881876+0000","last_change":"2024-03-30T01:43:12.259473+0000","last_active":"2024-03-30T01:43:12.881876+0000","last_peered":"2024-03-30T01:43:12.881876+0000","last_clean":"2024-03-30T01:43:12.881876+0000","last_became_active":"2024-03-30T01:43:12.259194+0000","last_became_peered":"2024-03-30T01:43:12.259194+0000","last_unstale":"2024-03-30T01:43:12.881876+0000","last_undegraded":"2024-03-30T01:43:12.881876+0000","last_fullsized":"2024-03-30T01:43:12.881876+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:14.284517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"60'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-03-30T01:43:13.259443+0000","last_change":"2024-03-30T01:43:11.810112+0000","last_active":"2024-03-30T01:43:13.259443+0000","last_peered":"2024-03-30T01:43:13.259443+0000","last_clean":"2024-03-30T01:43:13.259443+0000","last_became_active":"2024-03-30T01:43:11.808349+0000","last_became_peered":"2024-03-30T01:43:11.808349+0000","last_unstale":"2024-03-30T01:43:13.259443+0000","last_undegraded":"2024-03-30T01:43:13.259443+0000","last_fullsized":"2024-03-30T01:43:13.259443+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:30.914337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885564+0000","last_change":"2024-03-30T01:43:12.265689+0000","last_active":"2024-03-30T01:43:12.885564+0000","last_peered":"2024-03-30T01:43:12.885564+0000","last_clean":"2024-03-30T01:43:12.885564+0000","last_became_active":"2024-03-30T01:43:12.265382+0000","last_became_peered":"2024-03-30T01:43:12.265382+0000","last_unstale":"2024-03-30T01:43:12.885564+0000","last_undegraded":"2024-03-30T01:43:12.885564+0000","last_fullsized":"2024-03-30T01:43:12.885564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:17.176008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.885168+0000","last_change":"2024-03-30T01:43:11.811154+0000","last_active":"2024-03-30T01:43:12.885168+0000","last_peered":"2024-03-30T01:43:12.885168+0000","last_clean":"2024-03-30T01:43:12.885168+0000","last_became_active":"2024-03-30T01:43:11.810455+0000","last_became_peered":"2024-03-30T01:43:11.810455+0000","last_unstale":"2024-03-30T01:43:12.885168+0000","last_undegraded":"2024-03-30T01:43:12.885168+0000","last_fullsized":"2024-03-30T01:43:12.885168+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:33.020166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888369+0000","last_change":"2024-03-30T01:43:12.265758+0000","last_active":"2024-03-30T01:43:12.888369+0000","last_peered":"2024-03-30T01:43:12.888369+0000","last_clean":"2024-03-30T01:43:12.888369+0000","last_became_active":"2024-03-30T01:43:12.265461+0000","last_became_peered":"2024-03-30T01:43:12.265461+0000","last_unstale":"2024-03-30T01:43:12.888369+0000","last_undegraded":"2024-03-30T01:43:12.888369+0000","last_fullsized":"2024-03-30T01:43:12.888369+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:36.552215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884577+0000","last_change":"2024-03-30T01:43:11.806860+0000","last_active":"2024-03-30T01:43:12.884577+0000","last_peered":"2024-03-30T01:43:12.884577+0000","last_clean":"2024-03-30T01:43:12.884577+0000","last_became_active":"2024-03-30T01:43:11.806563+0000","last_became_peered":"2024-03-30T01:43:11.806563+0000","last_unstale":"2024-03-30T01:43:12.884577+0000","last_undegraded":"2024-03-30T01:43:12.884577+0000","last_fullsized":"2024-03-30T01:43:12.884577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888218+0000","last_change":"2024-03-30T01:43:12.269888+0000","last_active":"2024-03-30T01:43:12.888218+0000","last_peered":"2024-03-30T01:43:12.888218+0000","last_clean":"2024-03-30T01:43:12.888218+0000","last_became_active":"2024-03-30T01:43:12.269582+0000","last_became_peered":"2024-03-30T01:43:12.269582+0000","last_unstale":"2024-03-30T01:43:12.888218+0000","last_undegraded":"2024-03-30T01:43:12.888218+0000","last_fullsized":"2024-03-30T01:43:12.888218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:58.121544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881923+0000","last_change":"2024-03-30T01:43:11.802541+0000","last_active":"2024-03-30T01:43:12.881923+0000","last_peered":"2024-03-30T01:43:12.881923+0000","last_clean":"2024-03-30T01:43:12.881923+0000","last_became_active":"2024-03-30T01:43:11.802154+0000","last_became_peered":"2024-03-30T01:43:11.802154+0000","last_unstale":"2024-03-30T01:43:12.881923+0000","last_undegraded":"2024-03-30T01:43:12.881923+0000","last_fullsized":"2024-03-30T01:43:12.881923+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881560+0000","last_change":"2024-03-30T01:43:12.279403+0000","last_active":"2024-03-30T01:43:12.881560+0000","last_peered":"2024-03-30T01:43:12.881560+0000","last_clean":"2024-03-30T01:43:12.881560+0000","last_became_active":"2024-03-30T01:43:12.279071+0000","last_became_peered":"2024-03-30T01:43:12.279071+0000","last_unstale":"2024-03-30T01:43:12.881560+0000","last_undegraded":"2024-03-30T01:43:12.881560+0000","last_fullsized":"2024-03-30T01:43:12.881560+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884183+0000","last_change":"2024-03-30T01:43:11.799428+0000","last_active":"2024-03-30T01:43:12.884183+0000","last_peered":"2024-03-30T01:43:12.884183+0000","last_clean":"2024-03-30T01:43:12.884183+0000","last_became_active":"2024-03-30T01:43:11.799147+0000","last_became_peered":"2024-03-30T01:43:11.799147+0000","last_unstale":"2024-03-30T01:43:12.884183+0000","last_undegraded":"2024-03-30T01:43:12.884183+0000","last_fullsized":"2024-03-30T01:43:12.884183+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:56.396431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884504+0000","last_change":"2024-03-30T01:43:12.273079+0000","last_active":"2024-03-30T01:43:12.884504+0000","last_peered":"2024-03-30T01:43:12.884504+0000","last_clean":"2024-03-30T01:43:12.884504+0000","last_became_active":"2024-03-30T01:43:12.272766+0000","last_became_peered":"2024-03-30T01:43:12.272766+0000","last_unstale":"2024-03-30T01:43:12.884504+0000","last_undegraded":"2024-03-30T01:43:12.884504+0000","last_fullsized":"2024-03-30T01:43:12.884504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:57.882115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884114+0000","last_change":"2024-03-30T01:43:11.805262+0000","last_active":"2024-03-30T01:43:12.884114+0000","last_peered":"2024-03-30T01:43:12.884114+0000","last_clean":"2024-03-30T01:43:12.884114+0000","last_became_active":"2024-03-30T01:43:11.804622+0000","last_became_peered":"2024-03-30T01:43:11.804622+0000","last_unstale":"2024-03-30T01:43:12.884114+0000","last_undegraded":"2024-03-30T01:43:12.884114+0000","last_fullsized":"2024-03-30T01:43:12.884114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:08.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.881856+0000","last_change":"2024-03-30T01:43:12.275816+0000","last_active":"2024-03-30T01:43:12.881856+0000","last_peered":"2024-03-30T01:43:12.881856+0000","last_clean":"2024-03-30T01:43:12.881856+0000","last_became_active":"2024-03-30T01:43:12.275547+0000","last_became_peered":"2024-03-30T01:43:12.275547+0000","last_unstale":"2024-03-30T01:43:12.881856+0000","last_undegraded":"2024-03-30T01:43:12.881856+0000","last_fullsized":"2024-03-30T01:43:12.881856+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:31.125780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884464+0000","last_change":"2024-03-30T01:43:11.804870+0000","last_active":"2024-03-30T01:43:12.884464+0000","last_peered":"2024-03-30T01:43:12.884464+0000","last_clean":"2024-03-30T01:43:12.884464+0000","last_became_active":"2024-03-30T01:43:11.804580+0000","last_became_peered":"2024-03-30T01:43:11.804580+0000","last_unstale":"2024-03-30T01:43:12.884464+0000","last_undegraded":"2024-03-30T01:43:12.884464+0000","last_fullsized":"2024-03-30T01:43:12.884464+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:24.807845+0000","last_change":"2024-03-30T01:43:24.807845+0000","last_active":"2024-03-30T01:43:24.807845+0000","last_peered":"2024-03-30T01:43:24.807845+0000","last_clean":"2024-03-30T01:43:24.807845+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:24.807845+0000","last_undegraded":"2024-03-30T01:43:24.807845+0000","last_fullsized":"2024-03-30T01:43:24.807845+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-30T01:43:23.797822+0000","last_change":"2024-03-30T01:43:23.797822+0000","last_active":"2024-03-30T01:43:23.797822+0000","last_peered":"2024-03-30T01:43:23.797822+0000","last_clean":"2024-03-30T01:43:23.797822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T01:43:23.797822+0000","last_undegraded":"2024-03-30T01:43:23.797822+0000","last_fullsized":"2024-03-30T01:43:23.797822+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.888297+0000","last_change":"2024-03-30T01:43:12.270228+0000","last_active":"2024-03-30T01:43:12.888297+0000","last_peered":"2024-03-30T01:43:12.888297+0000","last_clean":"2024-03-30T01:43:12.888297+0000","last_became_active":"2024-03-30T01:43:12.269910+0000","last_became_peered":"2024-03-30T01:43:12.269910+0000","last_unstale":"2024-03-30T01:43:12.888297+0000","last_undegraded":"2024-03-30T01:43:12.888297+0000","last_fullsized":"2024-03-30T01:43:12.888297+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:00:52.920381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.884048+0000","last_change":"2024-03-30T01:43:11.803125+0000","last_active":"2024-03-30T01:43:12.884048+0000","last_peered":"2024-03-30T01:43:12.884048+0000","last_clean":"2024-03-30T01:43:12.884048+0000","last_became_active":"2024-03-30T01:43:11.802822+0000","last_became_peered":"2024-03-30T01:43:11.802822+0000","last_unstale":"2024-03-30T01:43:12.884048+0000","last_undegraded":"2024-03-30T01:43:12.884048+0000","last_fullsized":"2024-03-30T01:43:12.884048+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:21.501027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-03-30T01:43:12.256691+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:43:12.256691+0000","last_peered":"2024-03-30T01:43:12.256691+0000","last_clean":"2024-03-30T01:43:12.256691+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:43:12.256691+0000","last_undegraded":"2024-03-30T01:43:12.256691+0000","last_fullsized":"2024-03-30T01:43:12.256691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28332,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343508,"statfs":{"total":96636764160,"available":96607752192,"internally_reserved":0,"allocated":1335296,"data_stored":924182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.628}]},{"osd":1,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72699999999999998,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.81200000000000006,"15min":0.85599999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72699999999999998,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.76200000000000001,"15min":0.79500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.74099999999999999}]},{"osd":3,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73199999999999998,"5min":1.264,"15min":1.264},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79300000000000004,"5min":1.075,"15min":1.075},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77200000000000002,"5min":1.833,"15min":1.833},"last":0.51000000000000001}]},{"osd":5,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76300000000000001,"5min":1.238,"15min":1.238},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90600000000000003,"5min":1.2,"15min":1.2},"last":0.72799999999999998}]},{"osd":6,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85099999999999998,"5min":1.288,"15min":1.288},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.68300000000000005,"15min":0.69799999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82499999999999996,"5min":1.901,"15min":1.901},"last":0.70399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":729088,"data_stored":333072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.94899999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":1.361,"15min":1.361},"last":0.38400000000000001}]},{"osd":1,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96799999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.71699999999999997,"15min":0.74399999999999999},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.69199999999999995}]},{"osd":2,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85599999999999998,"5min":1.27,"15min":1.27},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.409,"15min":1.409},"last":0.46200000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.379,"5min":0.371,"15min":0.371},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.42499999999999999}]},{"osd":4,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.628},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53200000000000003}]},{"osd":5,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.71399999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":753664,"data_stored":336877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.432,"15min":0.442},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.503,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":1.111,"15min":1.2270000000000001},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":15.038,"15min":15.038},"last":0.39600000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.91400000000000003,"15min":1.006},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55500000000000005,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68400000000000005,"5min":1.444,"15min":1.444},"last":0.45800000000000002}]},{"osd":3,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":1.474,"15min":1.474},"last":0.378},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.624,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64000000000000001,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.32100000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.111,"15min":1.1970000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94499999999999995,"5min":11.99,"15min":11.99},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89900000000000002,"5min":1.407,"15min":1.407},"last":0.66000000000000003}]},{"osd":5,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":1.335,"15min":1.4490000000000001},"min":{"1min":0.48799999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97699999999999998,"5min":15.006,"15min":15.006},"last":0.628},{"interface":"front","average":{"1min":0.64800000000000002,"5min":1.2130000000000001,"15min":1.3069999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83199999999999996,"5min":11.926,"15min":11.926},"last":0.64800000000000002}]},{"osd":6,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.3149999999999999,"15min":1.429},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93100000000000005,"5min":14.974,"15min":14.974},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.1950000000000001,"15min":1.2949999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78100000000000003,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.67700000000000005}]},{"osd":7,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.504,"15min":1.504},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":1.343,"15min":1.4570000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.57399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28312,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343528,"statfs":{"total":96636764160,"available":96607772672,"internally_reserved":0,"allocated":1314816,"data_stored":922437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.57999999999999996,"15min":0.61499999999999999},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.439,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.54500000000000004,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.316},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.95499999999999996,"15min":1.05},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.443,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.86299999999999999,"15min":0.94299999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.503,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.59799999999999998,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.53300000000000003,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.39300000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.84699999999999998,"15min":0.875},"min":{"1min":0.47099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":5.431,"15min":5.431},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88300000000000001,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.50600000000000001}]},{"osd":5,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.80100000000000005,"15min":0.82899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77700000000000002,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.83799999999999997,"15min":0.87},"min":{"1min":0.47099999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.51600000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76500000000000001,"5min":1.177,"15min":1.177},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.82399999999999995,"15min":0.85399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.60099999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.81899999999999995,"15min":0.84799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.88800000000000001,"15min":0.92700000000000005},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81899999999999995,"5min":5.468,"15min":5.468},"last":0.47499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":333038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.89000000000000001,"15min":0.96199999999999997},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.1890000000000001,"5min":12.478999999999999,"15min":12.478999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.753,"15min":0.79600000000000004},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1469999999999998,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.40400000000000003}]},{"osd":1,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53900000000000003,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.93899999999999995,"15min":1.0089999999999999},"min":{"1min":0.28999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.1309999999999998,"5min":12.425000000000001,"15min":12.425000000000001},"last":0.377}]},{"osd":3,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1459999999999999,"5min":1.55,"15min":1.55},"last":0.318},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.98299999999999998,"15min":1.0620000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.8759999999999999,"5min":13.217000000000001,"15min":13.217000000000001},"last":0.46400000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.93899999999999995,"15min":0.97999999999999998},"min":{"1min":0.46400000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.956,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.95499999999999996,"15min":0.997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.0630000000000002,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.45100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89300000000000002,"5min":1.462,"15min":1.462},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.97499999999999998,"15min":1.0229999999999999},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.9159999999999999,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":0.54700000000000004}]},{"osd":6,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.97899999999999998,"15min":1.0229999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1139999999999999,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.48599999999999999}]},{"osd":7,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.72099999999999997,"15min":0.73699999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.524,"15min":1.524},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.0660000000000001,"15min":1.105},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":3.105,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":0.64700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":333038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.47699999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.36599999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.372,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41799999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.438}]},{"osd":2,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45000000000000001,"15min":0.45400000000000001},"min":{"1min":0.33800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.53600000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45000000000000001,"15min":0.45400000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33200000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76200000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91500000000000004,"5min":1.093,"15min":1.093},"last":0.50900000000000001}]},{"osd":5,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.80800000000000005,"5min":1.042,"15min":1.042},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89200000000000002,"5min":1.012,"15min":1.012},"last":0.51600000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.68899999999999995,"15min":0.69999999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.78900000000000003,"5min":1.024,"15min":1.024},"last":0.53200000000000003}]},{"osd":7,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.78200000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.878,"5min":1.085,"15min":1.085},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":737280,"data_stored":336113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.67600000000000005,"15min":0.69499999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.87,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.437}]},{"osd":1,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":1.0289999999999999,"15min":1.113},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.747,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.52800000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76800000000000002,"5min":1.494,"15min":1.494},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.86599999999999999,"15min":0.91000000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":1.1279999999999999,"15min":1.218},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73499999999999999,"5min":9.452,"15min":9.452},"last":0.67800000000000005}]},{"osd":5,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.621},{"interface":"front","average":{"1min":0.627,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78900000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.81899999999999995}]},{"osd":6,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76400000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.68500000000000005,"15min":0.69899999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82499999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89200000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.60199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":729088,"data_stored":333060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.83399999999999996,"15min":0.876},"min":{"1min":0.39800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.432},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.88400000000000001,"15min":0.93200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88200000000000001,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.61199999999999999}]},{"osd":1,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.84999999999999998,"15min":0.89800000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81999999999999995,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.89400000000000002,"15min":0.93999999999999995},"min":{"1min":0.436,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94099999999999995,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.83999999999999997,"15min":0.88600000000000001},"min":{"1min":0.373,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80400000000000005,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77000000000000002,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.88800000000000001,"15min":0.94499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91500000000000004,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86499999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.76600000000000001}]},{"osd":4,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85299999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.71599999999999997}]},{"osd":6,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89000000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59199999999999997}]},{"osd":7,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.627,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.751}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:43:29.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T01:43:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.709+0000 7fa049aef700 1 Processor -- start 2024-03-30T01:43:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.709+0000 7fa049aef700 1 -- start start 2024-03-30T01:43:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.710+0000 7fa049aef700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0440c3450 0x7fa044153100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.710+0000 7fa049aef700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 0x7fa044154e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.710+0000 7fa049aef700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0440c22c0 con 0x7fa044147f10 2024-03-30T01:43:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.710+0000 7fa049aef700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0440c2f40 con 0x7fa0440c3450 2024-03-30T01:43:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.710+0000 7fa049aef700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440c25a0 con 0x7fa044154a10 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.710+0000 7fa042ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0440c3450 0x7fa044153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa042ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0440c3450 0x7fa044153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48728/0 (socket says 172.21.15.31:48728) 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa042ffd700 1 -- 172.21.15.31:0/1291393825 learned_addr learned my addr 172.21.15.31:0/1291393825 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa043fff700 1 --2- 172.21.15.31:0/1291393825 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 0x7fa044154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa042ffd700 1 -- 172.21.15.31:0/1291393825 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 msgr2=0x7fa044154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa042ffd700 1 --2- 172.21.15.31:0/1291393825 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 0x7fa044154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa042ffd700 1 -- 172.21.15.31:0/1291393825 >> v1:172.21.15.59:6789/0 conn(0x7fa044147f10 legacy=0x7fa0441529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa042ffd700 1 -- 172.21.15.31:0/1291393825 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0440c2880 con 0x7fa0440c3450 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa042ffd700 1 --2- 172.21.15.31:0/1291393825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0440c3450 0x7fa044153100 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa03800bb90 tx=0x7fa03800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2ab8063686f016 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa0427fc700 1 -- 172.21.15.31:0/1291393825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03800dd30 con 0x7fa0440c3450 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa0427fc700 1 -- 172.21.15.31:0/1291393825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03800de90 con 0x7fa0440c3450 2024-03-30T01:43:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.711+0000 7fa0427fc700 1 -- 172.21.15.31:0/1291393825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa038010690 con 0x7fa0440c3450 2024-03-30T01:43:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.712+0000 7fa049aef700 1 -- 172.21.15.31:0/1291393825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0440c3450 msgr2=0x7fa044153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.712+0000 7fa049aef700 1 --2- 172.21.15.31:0/1291393825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0440c3450 0x7fa044153100 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa03800bb90 tx=0x7fa03800bea0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.712+0000 7fa049aef700 1 -- 172.21.15.31:0/1291393825 shutdown_connections 2024-03-30T01:43:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.712+0000 7fa049aef700 1 --2- 172.21.15.31:0/1291393825 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 0x7fa044154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.712+0000 7fa049aef700 1 --2- 172.21.15.31:0/1291393825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0440c3450 0x7fa044153100 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.712+0000 7fa049aef700 1 -- 172.21.15.31:0/1291393825 >> 172.21.15.31:0/1291393825 conn(0x7fa0440c3910 msgr2=0x7fa0440c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.712+0000 7fa049aef700 1 -- 172.21.15.31:0/1291393825 shutdown_connections 2024-03-30T01:43:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.712+0000 7fa049aef700 1 -- 172.21.15.31:0/1291393825 wait complete. 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.713+0000 7fa049aef700 1 Processor -- start 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.713+0000 7fa049aef700 1 -- start start 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.714+0000 7fa049aef700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440c3450 0x7fa0440c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.714+0000 7fa049aef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044147f10 0x7fa0441edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.714+0000 7fa049aef700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 0x7fa0440c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.714+0000 7fa049aef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0441ee100 con 0x7fa044147f10 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.714+0000 7fa049aef700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04406e0d0 con 0x7fa0440c3450 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.714+0000 7fa049aef700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04406e3b0 con 0x7fa044154a10 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.714+0000 7fa042ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044147f10 0x7fa0441edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.714+0000 7fa042ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044147f10 0x7fa0441edbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48744/0 (socket says 172.21.15.31:48744) 2024-03-30T01:43:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.714+0000 7fa042ffd700 1 -- 172.21.15.31:0/3048038343 learned_addr learned my addr 172.21.15.31:0/3048038343 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa043fff700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 0x7fa0440c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa0437fe700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440c3450 0x7fa0440c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa042ffd700 1 -- 172.21.15.31:0/3048038343 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 msgr2=0x7fa0440c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa042ffd700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 0x7fa0440c04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa042ffd700 1 -- 172.21.15.31:0/3048038343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440c3450 msgr2=0x7fa0440c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa042ffd700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440c3450 0x7fa0440c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa042ffd700 1 -- 172.21.15.31:0/3048038343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03800b7f0 con 0x7fa044147f10 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa0437fe700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440c3450 0x7fa0440c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa042ffd700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044147f10 0x7fa0441edbc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa038000c00 tx=0x7fa038004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa02bfff700 1 -- 172.21.15.31:0/3048038343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa038003cf0 con 0x7fa044147f10 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa02bfff700 1 -- 172.21.15.31:0/3048038343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa038003e70 con 0x7fa044147f10 2024-03-30T01:43:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04406e690 con 0x7fa044147f10 2024-03-30T01:43:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa02bfff700 1 -- 172.21.15.31:0/3048038343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa038010690 con 0x7fa044147f10 2024-03-30T01:43:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.715+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0441ed6a0 con 0x7fa044147f10 2024-03-30T01:43:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.716+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04406e8d0 con 0x7fa044147f10 2024-03-30T01:43:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.717+0000 7fa02bfff700 1 -- 172.21.15.31:0/3048038343 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa038019400 con 0x7fa044147f10 2024-03-30T01:43:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.717+0000 7fa02bfff700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa03006acc0 0x7fa03006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.718+0000 7fa02bfff700 1 -- 172.21.15.31:0/3048038343 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fa03808b030 con 0x7fa044147f10 2024-03-30T01:43:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.718+0000 7fa0437fe700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa03006acc0 0x7fa03006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.718+0000 7fa0437fe700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa03006acc0 0x7fa03006d180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa034000f60 tx=0x7fa034009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.722+0000 7fa02bfff700 1 -- 172.21.15.31:0/3048038343 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa038055ee0 con 0x7fa044147f10 2024-03-30T01:43:29.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.936+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 --> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0441482f0 con 0x7fa03006acc0 2024-03-30T01:43:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.951+0000 7fa02bfff700 1 -- 172.21.15.31:0/3048038343 <== mgr.4104 v2:172.21.15.31:6832/47081 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+527747 (secure 0 0 0) 0x7fa0340123d0 con 0x7fa03006acc0 2024-03-30T01:43:29.964 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:29.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa03006acc0 msgr2=0x7fa03006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa03006acc0 0x7fa03006d180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa034000f60 tx=0x7fa034009040 comp rx=0 tx=0).stop 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044147f10 msgr2=0x7fa0441edbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044147f10 0x7fa0441edbc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa038000c00 tx=0x7fa038004750 comp rx=0 tx=0).stop 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 shutdown_connections 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa03006acc0 0x7fa03006d180 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa044154a10 0x7fa0440c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0440c3450 0x7fa0440c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 --2- 172.21.15.31:0/3048038343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa044147f10 0x7fa0441edbc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 >> 172.21.15.31:0/3048038343 conn(0x7fa0440c3910 msgr2=0x7fa044152b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 shutdown_connections 2024-03-30T01:43:29.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:29.956+0000 7fa049aef700 1 -- 172.21.15.31:0/3048038343 wait complete. 2024-03-30T01:43:29.970 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-03-30T01:43:29.976 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-03-30T01:43:29.842162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596603,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":250,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":334,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223564,"kb_used_data":7332,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751156,"statfs":{"total":773094113280,"available":772865183744,"internally_reserved":0,"allocated":7507968,"data_stored":4095509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001245"},"pg_stats":[{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.323079+0000","last_change":"2024-03-30T01:43:25.297540+0000","last_active":"2024-03-30T01:43:26.323079+0000","last_peered":"2024-03-30T01:43:26.323079+0000","last_clean":"2024-03-30T01:43:26.323079+0000","last_became_active":"2024-03-30T01:43:25.296343+0000","last_became_peered":"2024-03-30T01:43:25.296343+0000","last_unstale":"2024-03-30T01:43:26.323079+0000","last_undegraded":"2024-03-30T01:43:26.323079+0000","last_fullsized":"2024-03-30T01:43:26.323079+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:23.970786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.298735+0000","last_change":"2024-03-30T01:43:12.269722+0000","last_active":"2024-03-30T01:43:25.298735+0000","last_peered":"2024-03-30T01:43:25.298735+0000","last_clean":"2024-03-30T01:43:25.298735+0000","last_became_active":"2024-03-30T01:43:12.269445+0000","last_became_peered":"2024-03-30T01:43:12.269445+0000","last_unstale":"2024-03-30T01:43:25.298735+0000","last_undegraded":"2024-03-30T01:43:25.298735+0000","last_fullsized":"2024-03-30T01:43:25.298735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287839+0000","last_change":"2024-03-30T01:43:25.298582+0000","last_active":"2024-03-30T01:43:26.287839+0000","last_peered":"2024-03-30T01:43:26.287839+0000","last_clean":"2024-03-30T01:43:26.287839+0000","last_became_active":"2024-03-30T01:43:25.298305+0000","last_became_peered":"2024-03-30T01:43:25.298305+0000","last_unstale":"2024-03-30T01:43:26.287839+0000","last_undegraded":"2024-03-30T01:43:26.287839+0000","last_fullsized":"2024-03-30T01:43:26.287839+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:12:11.148303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.297013+0000","last_change":"2024-03-30T01:43:12.261636+0000","last_active":"2024-03-30T01:43:25.297013+0000","last_peered":"2024-03-30T01:43:25.297013+0000","last_clean":"2024-03-30T01:43:25.297013+0000","last_became_active":"2024-03-30T01:43:12.261289+0000","last_became_peered":"2024-03-30T01:43:12.261289+0000","last_unstale":"2024-03-30T01:43:25.297013+0000","last_undegraded":"2024-03-30T01:43:25.297013+0000","last_fullsized":"2024-03-30T01:43:25.297013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:53.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287839+0000","last_change":"2024-03-30T01:43:25.304924+0000","last_active":"2024-03-30T01:43:26.287839+0000","last_peered":"2024-03-30T01:43:26.287839+0000","last_clean":"2024-03-30T01:43:26.287839+0000","last_became_active":"2024-03-30T01:43:25.304629+0000","last_became_peered":"2024-03-30T01:43:25.304629+0000","last_unstale":"2024-03-30T01:43:26.287839+0000","last_undegraded":"2024-03-30T01:43:26.287839+0000","last_fullsized":"2024-03-30T01:43:26.287839+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:46:04.549827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.295824+0000","last_change":"2024-03-30T01:43:12.273399+0000","last_active":"2024-03-30T01:43:25.295824+0000","last_peered":"2024-03-30T01:43:25.295824+0000","last_clean":"2024-03-30T01:43:25.295824+0000","last_became_active":"2024-03-30T01:43:12.273099+0000","last_became_peered":"2024-03-30T01:43:12.273099+0000","last_unstale":"2024-03-30T01:43:25.295824+0000","last_undegraded":"2024-03-30T01:43:25.295824+0000","last_fullsized":"2024-03-30T01:43:25.295824+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319964+0000","last_change":"2024-03-30T01:43:25.311593+0000","last_active":"2024-03-30T01:43:26.319964+0000","last_peered":"2024-03-30T01:43:26.319964+0000","last_clean":"2024-03-30T01:43:26.319964+0000","last_became_active":"2024-03-30T01:43:25.311240+0000","last_became_peered":"2024-03-30T01:43:25.311240+0000","last_unstale":"2024-03-30T01:43:26.319964+0000","last_undegraded":"2024-03-30T01:43:26.319964+0000","last_fullsized":"2024-03-30T01:43:26.319964+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:41:15.810063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.290178+0000","last_change":"2024-03-30T01:43:12.277385+0000","last_active":"2024-03-30T01:43:25.290178+0000","last_peered":"2024-03-30T01:43:25.290178+0000","last_clean":"2024-03-30T01:43:25.290178+0000","last_became_active":"2024-03-30T01:43:12.277091+0000","last_became_peered":"2024-03-30T01:43:12.277091+0000","last_unstale":"2024-03-30T01:43:25.290178+0000","last_undegraded":"2024-03-30T01:43:25.290178+0000","last_fullsized":"2024-03-30T01:43:25.290178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:09:35.397960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.286718+0000","last_change":"2024-03-30T01:43:25.315710+0000","last_active":"2024-03-30T01:43:26.286718+0000","last_peered":"2024-03-30T01:43:26.286718+0000","last_clean":"2024-03-30T01:43:26.286718+0000","last_became_active":"2024-03-30T01:43:25.315432+0000","last_became_peered":"2024-03-30T01:43:25.315432+0000","last_unstale":"2024-03-30T01:43:26.286718+0000","last_undegraded":"2024-03-30T01:43:26.286718+0000","last_fullsized":"2024-03-30T01:43:26.286718+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:33.620738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.283385+0000","last_change":"2024-03-30T01:43:12.276432+0000","last_active":"2024-03-30T01:43:25.283385+0000","last_peered":"2024-03-30T01:43:25.283385+0000","last_clean":"2024-03-30T01:43:25.283385+0000","last_became_active":"2024-03-30T01:43:12.276157+0000","last_became_peered":"2024-03-30T01:43:12.276157+0000","last_unstale":"2024-03-30T01:43:25.283385+0000","last_undegraded":"2024-03-30T01:43:25.283385+0000","last_fullsized":"2024-03-30T01:43:25.283385+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:59:56.492934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288315+0000","last_change":"2024-03-30T01:43:25.309948+0000","last_active":"2024-03-30T01:43:26.288315+0000","last_peered":"2024-03-30T01:43:26.288315+0000","last_clean":"2024-03-30T01:43:26.288315+0000","last_became_active":"2024-03-30T01:43:25.309665+0000","last_became_peered":"2024-03-30T01:43:25.309665+0000","last_unstale":"2024-03-30T01:43:26.288315+0000","last_undegraded":"2024-03-30T01:43:26.288315+0000","last_fullsized":"2024-03-30T01:43:26.288315+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:12.721684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.302329+0000","last_change":"2024-03-30T01:43:12.273311+0000","last_active":"2024-03-30T01:43:25.302329+0000","last_peered":"2024-03-30T01:43:25.302329+0000","last_clean":"2024-03-30T01:43:25.302329+0000","last_became_active":"2024-03-30T01:43:12.272939+0000","last_became_peered":"2024-03-30T01:43:12.272939+0000","last_unstale":"2024-03-30T01:43:25.302329+0000","last_undegraded":"2024-03-30T01:43:25.302329+0000","last_fullsized":"2024-03-30T01:43:25.302329+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:38.756747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321949+0000","last_change":"2024-03-30T01:43:25.307813+0000","last_active":"2024-03-30T01:43:26.321949+0000","last_peered":"2024-03-30T01:43:26.321949+0000","last_clean":"2024-03-30T01:43:26.321949+0000","last_became_active":"2024-03-30T01:43:25.307520+0000","last_became_peered":"2024-03-30T01:43:25.307520+0000","last_unstale":"2024-03-30T01:43:26.321949+0000","last_undegraded":"2024-03-30T01:43:26.321949+0000","last_fullsized":"2024-03-30T01:43:26.321949+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:33:51.006865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.287761+0000","last_change":"2024-03-30T01:43:12.279192+0000","last_active":"2024-03-30T01:43:25.287761+0000","last_peered":"2024-03-30T01:43:25.287761+0000","last_clean":"2024-03-30T01:43:25.287761+0000","last_became_active":"2024-03-30T01:43:12.278893+0000","last_became_peered":"2024-03-30T01:43:12.278893+0000","last_unstale":"2024-03-30T01:43:25.287761+0000","last_undegraded":"2024-03-30T01:43:25.287761+0000","last_fullsized":"2024-03-30T01:43:25.287761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:21:09.466088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320127+0000","last_change":"2024-03-30T01:43:25.315351+0000","last_active":"2024-03-30T01:43:26.320127+0000","last_peered":"2024-03-30T01:43:26.320127+0000","last_clean":"2024-03-30T01:43:26.320127+0000","last_became_active":"2024-03-30T01:43:25.315001+0000","last_became_peered":"2024-03-30T01:43:25.315001+0000","last_unstale":"2024-03-30T01:43:26.320127+0000","last_undegraded":"2024-03-30T01:43:26.320127+0000","last_fullsized":"2024-03-30T01:43:26.320127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:24:12.064316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.293473+0000","last_change":"2024-03-30T01:43:12.274078+0000","last_active":"2024-03-30T01:43:25.293473+0000","last_peered":"2024-03-30T01:43:25.293473+0000","last_clean":"2024-03-30T01:43:25.293473+0000","last_became_active":"2024-03-30T01:43:12.273703+0000","last_became_peered":"2024-03-30T01:43:12.273703+0000","last_unstale":"2024-03-30T01:43:25.293473+0000","last_undegraded":"2024-03-30T01:43:25.293473+0000","last_fullsized":"2024-03-30T01:43:25.293473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319130+0000","last_change":"2024-03-30T01:43:25.315471+0000","last_active":"2024-03-30T01:43:26.319130+0000","last_peered":"2024-03-30T01:43:26.319130+0000","last_clean":"2024-03-30T01:43:26.319130+0000","last_became_active":"2024-03-30T01:43:25.314115+0000","last_became_peered":"2024-03-30T01:43:25.314115+0000","last_unstale":"2024-03-30T01:43:26.319130+0000","last_undegraded":"2024-03-30T01:43:26.319130+0000","last_fullsized":"2024-03-30T01:43:26.319130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:22:27.664497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.293508+0000","last_change":"2024-03-30T01:43:12.274182+0000","last_active":"2024-03-30T01:43:25.293508+0000","last_peered":"2024-03-30T01:43:25.293508+0000","last_clean":"2024-03-30T01:43:25.293508+0000","last_became_active":"2024-03-30T01:43:12.273863+0000","last_became_peered":"2024-03-30T01:43:12.273863+0000","last_unstale":"2024-03-30T01:43:25.293508+0000","last_undegraded":"2024-03-30T01:43:25.293508+0000","last_fullsized":"2024-03-30T01:43:25.293508+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320532+0000","last_change":"2024-03-30T01:43:25.305844+0000","last_active":"2024-03-30T01:43:26.320532+0000","last_peered":"2024-03-30T01:43:26.320532+0000","last_clean":"2024-03-30T01:43:26.320532+0000","last_became_active":"2024-03-30T01:43:25.305553+0000","last_became_peered":"2024-03-30T01:43:25.305553+0000","last_unstale":"2024-03-30T01:43:26.320532+0000","last_undegraded":"2024-03-30T01:43:26.320532+0000","last_fullsized":"2024-03-30T01:43:26.320532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:39.501161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292219+0000","last_change":"2024-03-30T01:43:12.277023+0000","last_active":"2024-03-30T01:43:25.292219+0000","last_peered":"2024-03-30T01:43:25.292219+0000","last_clean":"2024-03-30T01:43:25.292219+0000","last_became_active":"2024-03-30T01:43:12.276685+0000","last_became_peered":"2024-03-30T01:43:12.276685+0000","last_unstale":"2024-03-30T01:43:25.292219+0000","last_undegraded":"2024-03-30T01:43:25.292219+0000","last_fullsized":"2024-03-30T01:43:25.292219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:53.003493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287955+0000","last_change":"2024-03-30T01:43:25.296523+0000","last_active":"2024-03-30T01:43:26.287955+0000","last_peered":"2024-03-30T01:43:26.287955+0000","last_clean":"2024-03-30T01:43:26.287955+0000","last_became_active":"2024-03-30T01:43:25.296125+0000","last_became_peered":"2024-03-30T01:43:25.296125+0000","last_unstale":"2024-03-30T01:43:26.287955+0000","last_undegraded":"2024-03-30T01:43:26.287955+0000","last_fullsized":"2024-03-30T01:43:26.287955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:23:54.134728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289259+0000","last_change":"2024-03-30T01:43:12.273450+0000","last_active":"2024-03-30T01:43:25.289259+0000","last_peered":"2024-03-30T01:43:25.289259+0000","last_clean":"2024-03-30T01:43:25.289259+0000","last_became_active":"2024-03-30T01:43:12.273185+0000","last_became_peered":"2024-03-30T01:43:12.273185+0000","last_unstale":"2024-03-30T01:43:25.289259+0000","last_undegraded":"2024-03-30T01:43:25.289259+0000","last_fullsized":"2024-03-30T01:43:25.289259+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320072+0000","last_change":"2024-03-30T01:43:25.308156+0000","last_active":"2024-03-30T01:43:26.320072+0000","last_peered":"2024-03-30T01:43:26.320072+0000","last_clean":"2024-03-30T01:43:26.320072+0000","last_became_active":"2024-03-30T01:43:25.307892+0000","last_became_peered":"2024-03-30T01:43:25.307892+0000","last_unstale":"2024-03-30T01:43:26.320072+0000","last_undegraded":"2024-03-30T01:43:26.320072+0000","last_fullsized":"2024-03-30T01:43:26.320072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:54.411891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.293162+0000","last_change":"2024-03-30T01:43:12.273238+0000","last_active":"2024-03-30T01:43:25.293162+0000","last_peered":"2024-03-30T01:43:25.293162+0000","last_clean":"2024-03-30T01:43:25.293162+0000","last_became_active":"2024-03-30T01:43:12.272970+0000","last_became_peered":"2024-03-30T01:43:12.272970+0000","last_unstale":"2024-03-30T01:43:25.293162+0000","last_undegraded":"2024-03-30T01:43:25.293162+0000","last_fullsized":"2024-03-30T01:43:25.293162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:49.877712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320293+0000","last_change":"2024-03-30T01:43:25.317751+0000","last_active":"2024-03-30T01:43:26.320293+0000","last_peered":"2024-03-30T01:43:26.320293+0000","last_clean":"2024-03-30T01:43:26.320293+0000","last_became_active":"2024-03-30T01:43:25.317420+0000","last_became_peered":"2024-03-30T01:43:25.317420+0000","last_unstale":"2024-03-30T01:43:26.320293+0000","last_undegraded":"2024-03-30T01:43:26.320293+0000","last_fullsized":"2024-03-30T01:43:26.320293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:20:31.303744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.293749+0000","last_change":"2024-03-30T01:43:12.276660+0000","last_active":"2024-03-30T01:43:25.293749+0000","last_peered":"2024-03-30T01:43:25.293749+0000","last_clean":"2024-03-30T01:43:25.293749+0000","last_became_active":"2024-03-30T01:43:12.276437+0000","last_became_peered":"2024-03-30T01:43:12.276437+0000","last_unstale":"2024-03-30T01:43:25.293749+0000","last_undegraded":"2024-03-30T01:43:25.293749+0000","last_fullsized":"2024-03-30T01:43:25.293749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288032+0000","last_change":"2024-03-30T01:43:25.318577+0000","last_active":"2024-03-30T01:43:26.288032+0000","last_peered":"2024-03-30T01:43:26.288032+0000","last_clean":"2024-03-30T01:43:26.288032+0000","last_became_active":"2024-03-30T01:43:25.318348+0000","last_became_peered":"2024-03-30T01:43:25.318348+0000","last_unstale":"2024-03-30T01:43:26.288032+0000","last_undegraded":"2024-03-30T01:43:26.288032+0000","last_fullsized":"2024-03-30T01:43:26.288032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:40.255196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.291716+0000","last_change":"2024-03-30T01:43:12.276823+0000","last_active":"2024-03-30T01:43:25.291716+0000","last_peered":"2024-03-30T01:43:25.291716+0000","last_clean":"2024-03-30T01:43:25.291716+0000","last_became_active":"2024-03-30T01:43:12.272988+0000","last_became_peered":"2024-03-30T01:43:12.272988+0000","last_unstale":"2024-03-30T01:43:25.291716+0000","last_undegraded":"2024-03-30T01:43:25.291716+0000","last_fullsized":"2024-03-30T01:43:25.291716+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320344+0000","last_change":"2024-03-30T01:43:25.306450+0000","last_active":"2024-03-30T01:43:26.320344+0000","last_peered":"2024-03-30T01:43:26.320344+0000","last_clean":"2024-03-30T01:43:26.320344+0000","last_became_active":"2024-03-30T01:43:25.306156+0000","last_became_peered":"2024-03-30T01:43:25.306156+0000","last_unstale":"2024-03-30T01:43:26.320344+0000","last_undegraded":"2024-03-30T01:43:26.320344+0000","last_fullsized":"2024-03-30T01:43:26.320344+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:12.721684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289696+0000","last_change":"2024-03-30T01:43:12.277931+0000","last_active":"2024-03-30T01:43:25.289696+0000","last_peered":"2024-03-30T01:43:25.289696+0000","last_clean":"2024-03-30T01:43:25.289696+0000","last_became_active":"2024-03-30T01:43:12.277601+0000","last_became_peered":"2024-03-30T01:43:12.277601+0000","last_unstale":"2024-03-30T01:43:25.289696+0000","last_undegraded":"2024-03-30T01:43:25.289696+0000","last_fullsized":"2024-03-30T01:43:25.289696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288020+0000","last_change":"2024-03-30T01:43:25.302826+0000","last_active":"2024-03-30T01:43:26.288020+0000","last_peered":"2024-03-30T01:43:26.288020+0000","last_clean":"2024-03-30T01:43:26.288020+0000","last_became_active":"2024-03-30T01:43:25.302487+0000","last_became_peered":"2024-03-30T01:43:25.302487+0000","last_unstale":"2024-03-30T01:43:26.288020+0000","last_undegraded":"2024-03-30T01:43:26.288020+0000","last_fullsized":"2024-03-30T01:43:26.288020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:18:04.513691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.287762+0000","last_change":"2024-03-30T01:43:12.275724+0000","last_active":"2024-03-30T01:43:25.287762+0000","last_peered":"2024-03-30T01:43:25.287762+0000","last_clean":"2024-03-30T01:43:25.287762+0000","last_became_active":"2024-03-30T01:43:12.275438+0000","last_became_peered":"2024-03-30T01:43:12.275438+0000","last_unstale":"2024-03-30T01:43:25.287762+0000","last_undegraded":"2024-03-30T01:43:25.287762+0000","last_fullsized":"2024-03-30T01:43:25.287762+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:18.506294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320704+0000","last_change":"2024-03-30T01:43:25.306934+0000","last_active":"2024-03-30T01:43:26.320704+0000","last_peered":"2024-03-30T01:43:26.320704+0000","last_clean":"2024-03-30T01:43:26.320704+0000","last_became_active":"2024-03-30T01:43:25.306378+0000","last_became_peered":"2024-03-30T01:43:25.306378+0000","last_unstale":"2024-03-30T01:43:26.320704+0000","last_undegraded":"2024-03-30T01:43:26.320704+0000","last_fullsized":"2024-03-30T01:43:26.320704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:39.501161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.297096+0000","last_change":"2024-03-30T01:43:12.268431+0000","last_active":"2024-03-30T01:43:25.297096+0000","last_peered":"2024-03-30T01:43:25.297096+0000","last_clean":"2024-03-30T01:43:25.297096+0000","last_became_active":"2024-03-30T01:43:12.268148+0000","last_became_peered":"2024-03-30T01:43:12.268148+0000","last_unstale":"2024-03-30T01:43:25.297096+0000","last_undegraded":"2024-03-30T01:43:25.297096+0000","last_fullsized":"2024-03-30T01:43:25.297096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:49.877712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319456+0000","last_change":"2024-03-30T01:43:25.313220+0000","last_active":"2024-03-30T01:43:26.319456+0000","last_peered":"2024-03-30T01:43:26.319456+0000","last_clean":"2024-03-30T01:43:26.319456+0000","last_became_active":"2024-03-30T01:43:25.312952+0000","last_became_peered":"2024-03-30T01:43:25.312952+0000","last_unstale":"2024-03-30T01:43:26.319456+0000","last_undegraded":"2024-03-30T01:43:26.319456+0000","last_fullsized":"2024-03-30T01:43:26.319456+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:31:57.200761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.290401+0000","last_change":"2024-03-30T01:43:12.275632+0000","last_active":"2024-03-30T01:43:25.290401+0000","last_peered":"2024-03-30T01:43:25.290401+0000","last_clean":"2024-03-30T01:43:25.290401+0000","last_became_active":"2024-03-30T01:43:12.275329+0000","last_became_peered":"2024-03-30T01:43:12.275329+0000","last_unstale":"2024-03-30T01:43:25.290401+0000","last_undegraded":"2024-03-30T01:43:25.290401+0000","last_fullsized":"2024-03-30T01:43:25.290401+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:44.982486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319948+0000","last_change":"2024-03-30T01:43:25.308488+0000","last_active":"2024-03-30T01:43:26.319948+0000","last_peered":"2024-03-30T01:43:26.319948+0000","last_clean":"2024-03-30T01:43:26.319948+0000","last_became_active":"2024-03-30T01:43:25.308188+0000","last_became_peered":"2024-03-30T01:43:25.308188+0000","last_unstale":"2024-03-30T01:43:26.319948+0000","last_undegraded":"2024-03-30T01:43:26.319948+0000","last_fullsized":"2024-03-30T01:43:26.319948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:18:04.513691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.284202+0000","last_change":"2024-03-30T01:43:12.265240+0000","last_active":"2024-03-30T01:43:25.284202+0000","last_peered":"2024-03-30T01:43:25.284202+0000","last_clean":"2024-03-30T01:43:25.284202+0000","last_became_active":"2024-03-30T01:43:12.264980+0000","last_became_peered":"2024-03-30T01:43:12.264980+0000","last_unstale":"2024-03-30T01:43:25.284202+0000","last_undegraded":"2024-03-30T01:43:25.284202+0000","last_fullsized":"2024-03-30T01:43:25.284202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:15:38.037358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288118+0000","last_change":"2024-03-30T01:43:25.297748+0000","last_active":"2024-03-30T01:43:26.288118+0000","last_peered":"2024-03-30T01:43:26.288118+0000","last_clean":"2024-03-30T01:43:26.288118+0000","last_became_active":"2024-03-30T01:43:25.297514+0000","last_became_peered":"2024-03-30T01:43:25.297514+0000","last_unstale":"2024-03-30T01:43:26.288118+0000","last_undegraded":"2024-03-30T01:43:26.288118+0000","last_fullsized":"2024-03-30T01:43:26.288118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:38.525691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.291937+0000","last_change":"2024-03-30T01:43:12.266614+0000","last_active":"2024-03-30T01:43:25.291937+0000","last_peered":"2024-03-30T01:43:25.291937+0000","last_clean":"2024-03-30T01:43:25.291937+0000","last_became_active":"2024-03-30T01:43:12.266325+0000","last_became_peered":"2024-03-30T01:43:12.266325+0000","last_unstale":"2024-03-30T01:43:25.291937+0000","last_undegraded":"2024-03-30T01:43:25.291937+0000","last_fullsized":"2024-03-30T01:43:25.291937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:45:23.286791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319837+0000","last_change":"2024-03-30T01:43:25.311921+0000","last_active":"2024-03-30T01:43:26.319837+0000","last_peered":"2024-03-30T01:43:26.319837+0000","last_clean":"2024-03-30T01:43:26.319837+0000","last_became_active":"2024-03-30T01:43:25.311629+0000","last_became_peered":"2024-03-30T01:43:25.311629+0000","last_unstale":"2024-03-30T01:43:26.319837+0000","last_undegraded":"2024-03-30T01:43:26.319837+0000","last_fullsized":"2024-03-30T01:43:26.319837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:45:09.207684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.293868+0000","last_change":"2024-03-30T01:43:12.269577+0000","last_active":"2024-03-30T01:43:25.293868+0000","last_peered":"2024-03-30T01:43:25.293868+0000","last_clean":"2024-03-30T01:43:25.293868+0000","last_became_active":"2024-03-30T01:43:12.269263+0000","last_became_peered":"2024-03-30T01:43:12.269263+0000","last_unstale":"2024-03-30T01:43:25.293868+0000","last_undegraded":"2024-03-30T01:43:25.293868+0000","last_fullsized":"2024-03-30T01:43:25.293868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288420+0000","last_change":"2024-03-30T01:43:25.300105+0000","last_active":"2024-03-30T01:43:26.288420+0000","last_peered":"2024-03-30T01:43:26.288420+0000","last_clean":"2024-03-30T01:43:26.288420+0000","last_became_active":"2024-03-30T01:43:25.299840+0000","last_became_peered":"2024-03-30T01:43:25.299840+0000","last_unstale":"2024-03-30T01:43:26.288420+0000","last_undegraded":"2024-03-30T01:43:26.288420+0000","last_fullsized":"2024-03-30T01:43:26.288420+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:56:10.908873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.291588+0000","last_change":"2024-03-30T01:43:12.276626+0000","last_active":"2024-03-30T01:43:25.291588+0000","last_peered":"2024-03-30T01:43:25.291588+0000","last_clean":"2024-03-30T01:43:25.291588+0000","last_became_active":"2024-03-30T01:43:12.276302+0000","last_became_peered":"2024-03-30T01:43:12.276302+0000","last_unstale":"2024-03-30T01:43:25.291588+0000","last_undegraded":"2024-03-30T01:43:25.291588+0000","last_fullsized":"2024-03-30T01:43:25.291588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:11.027051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.289730+0000","last_change":"2024-03-30T01:43:25.303917+0000","last_active":"2024-03-30T01:43:26.289730+0000","last_peered":"2024-03-30T01:43:26.289730+0000","last_clean":"2024-03-30T01:43:26.289730+0000","last_became_active":"2024-03-30T01:43:25.303607+0000","last_became_peered":"2024-03-30T01:43:25.303607+0000","last_unstale":"2024-03-30T01:43:26.289730+0000","last_undegraded":"2024-03-30T01:43:26.289730+0000","last_fullsized":"2024-03-30T01:43:26.289730+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:25.299591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.290614+0000","last_change":"2024-03-30T01:43:12.276658+0000","last_active":"2024-03-30T01:43:25.290614+0000","last_peered":"2024-03-30T01:43:25.290614+0000","last_clean":"2024-03-30T01:43:25.290614+0000","last_became_active":"2024-03-30T01:43:12.276363+0000","last_became_peered":"2024-03-30T01:43:12.276363+0000","last_unstale":"2024-03-30T01:43:25.290614+0000","last_undegraded":"2024-03-30T01:43:25.290614+0000","last_fullsized":"2024-03-30T01:43:25.290614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:10:06.329557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.286972+0000","last_change":"2024-03-30T01:43:25.308785+0000","last_active":"2024-03-30T01:43:26.286972+0000","last_peered":"2024-03-30T01:43:26.286972+0000","last_clean":"2024-03-30T01:43:26.286972+0000","last_became_active":"2024-03-30T01:43:25.308528+0000","last_became_peered":"2024-03-30T01:43:25.308528+0000","last_unstale":"2024-03-30T01:43:26.286972+0000","last_undegraded":"2024-03-30T01:43:26.286972+0000","last_fullsized":"2024-03-30T01:43:26.286972+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:20:01.658875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.294941+0000","last_change":"2024-03-30T01:43:12.273270+0000","last_active":"2024-03-30T01:43:25.294941+0000","last_peered":"2024-03-30T01:43:25.294941+0000","last_clean":"2024-03-30T01:43:25.294941+0000","last_became_active":"2024-03-30T01:43:12.272973+0000","last_became_peered":"2024-03-30T01:43:12.272973+0000","last_unstale":"2024-03-30T01:43:25.294941+0000","last_undegraded":"2024-03-30T01:43:25.294941+0000","last_fullsized":"2024-03-30T01:43:25.294941+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.286453+0000","last_change":"2024-03-30T01:43:25.299719+0000","last_active":"2024-03-30T01:43:26.286453+0000","last_peered":"2024-03-30T01:43:26.286453+0000","last_clean":"2024-03-30T01:43:26.286453+0000","last_became_active":"2024-03-30T01:43:25.299414+0000","last_became_peered":"2024-03-30T01:43:25.299414+0000","last_unstale":"2024-03-30T01:43:26.286453+0000","last_undegraded":"2024-03-30T01:43:26.286453+0000","last_fullsized":"2024-03-30T01:43:26.286453+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:38:22.331738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.294379+0000","last_change":"2024-03-30T01:43:12.270679+0000","last_active":"2024-03-30T01:43:25.294379+0000","last_peered":"2024-03-30T01:43:25.294379+0000","last_clean":"2024-03-30T01:43:25.294379+0000","last_became_active":"2024-03-30T01:43:12.270457+0000","last_became_peered":"2024-03-30T01:43:12.270457+0000","last_unstale":"2024-03-30T01:43:25.294379+0000","last_undegraded":"2024-03-30T01:43:25.294379+0000","last_fullsized":"2024-03-30T01:43:25.294379+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:45.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320426+0000","last_change":"2024-03-30T01:43:25.307651+0000","last_active":"2024-03-30T01:43:26.320426+0000","last_peered":"2024-03-30T01:43:26.320426+0000","last_clean":"2024-03-30T01:43:26.320426+0000","last_became_active":"2024-03-30T01:43:25.307356+0000","last_became_peered":"2024-03-30T01:43:25.307356+0000","last_unstale":"2024-03-30T01:43:26.320426+0000","last_undegraded":"2024-03-30T01:43:26.320426+0000","last_fullsized":"2024-03-30T01:43:26.320426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:43:07.270134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292733+0000","last_change":"2024-03-30T01:43:12.269460+0000","last_active":"2024-03-30T01:43:25.292733+0000","last_peered":"2024-03-30T01:43:25.292733+0000","last_clean":"2024-03-30T01:43:25.292733+0000","last_became_active":"2024-03-30T01:43:12.269159+0000","last_became_peered":"2024-03-30T01:43:12.269159+0000","last_unstale":"2024-03-30T01:43:25.292733+0000","last_undegraded":"2024-03-30T01:43:25.292733+0000","last_fullsized":"2024-03-30T01:43:25.292733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319574+0000","last_change":"2024-03-30T01:43:25.314487+0000","last_active":"2024-03-30T01:43:26.319574+0000","last_peered":"2024-03-30T01:43:26.319574+0000","last_clean":"2024-03-30T01:43:26.319574+0000","last_became_active":"2024-03-30T01:43:25.314208+0000","last_became_peered":"2024-03-30T01:43:25.314208+0000","last_unstale":"2024-03-30T01:43:26.319574+0000","last_undegraded":"2024-03-30T01:43:26.319574+0000","last_fullsized":"2024-03-30T01:43:26.319574+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:41:54.387635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289320+0000","last_change":"2024-03-30T01:43:12.276051+0000","last_active":"2024-03-30T01:43:25.289320+0000","last_peered":"2024-03-30T01:43:25.289320+0000","last_clean":"2024-03-30T01:43:25.289320+0000","last_became_active":"2024-03-30T01:43:12.275296+0000","last_became_peered":"2024-03-30T01:43:12.275296+0000","last_unstale":"2024-03-30T01:43:25.289320+0000","last_undegraded":"2024-03-30T01:43:25.289320+0000","last_fullsized":"2024-03-30T01:43:25.289320+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:17.176008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320694+0000","last_change":"2024-03-30T01:43:25.311973+0000","last_active":"2024-03-30T01:43:26.320694+0000","last_peered":"2024-03-30T01:43:26.320694+0000","last_clean":"2024-03-30T01:43:26.320694+0000","last_became_active":"2024-03-30T01:43:25.311393+0000","last_became_peered":"2024-03-30T01:43:25.311393+0000","last_unstale":"2024-03-30T01:43:26.320694+0000","last_undegraded":"2024-03-30T01:43:26.320694+0000","last_fullsized":"2024-03-30T01:43:26.320694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:21:00.374174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289398+0000","last_change":"2024-03-30T01:43:12.266882+0000","last_active":"2024-03-30T01:43:25.289398+0000","last_peered":"2024-03-30T01:43:25.289398+0000","last_clean":"2024-03-30T01:43:25.289398+0000","last_became_active":"2024-03-30T01:43:12.266450+0000","last_became_peered":"2024-03-30T01:43:12.266450+0000","last_unstale":"2024-03-30T01:43:25.289398+0000","last_undegraded":"2024-03-30T01:43:25.289398+0000","last_fullsized":"2024-03-30T01:43:25.289398+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321638+0000","last_change":"2024-03-30T01:43:25.306231+0000","last_active":"2024-03-30T01:43:26.321638+0000","last_peered":"2024-03-30T01:43:26.321638+0000","last_clean":"2024-03-30T01:43:26.321638+0000","last_became_active":"2024-03-30T01:43:25.305966+0000","last_became_peered":"2024-03-30T01:43:25.305966+0000","last_unstale":"2024-03-30T01:43:26.321638+0000","last_undegraded":"2024-03-30T01:43:26.321638+0000","last_fullsized":"2024-03-30T01:43:26.321638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:12.721684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.294828+0000","last_change":"2024-03-30T01:43:12.279605+0000","last_active":"2024-03-30T01:43:25.294828+0000","last_peered":"2024-03-30T01:43:25.294828+0000","last_clean":"2024-03-30T01:43:25.294828+0000","last_became_active":"2024-03-30T01:43:12.279323+0000","last_became_peered":"2024-03-30T01:43:12.279323+0000","last_unstale":"2024-03-30T01:43:25.294828+0000","last_undegraded":"2024-03-30T01:43:25.294828+0000","last_fullsized":"2024-03-30T01:43:25.294828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:32:07.610941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288512+0000","last_change":"2024-03-30T01:43:25.302847+0000","last_active":"2024-03-30T01:43:26.288512+0000","last_peered":"2024-03-30T01:43:26.288512+0000","last_clean":"2024-03-30T01:43:26.288512+0000","last_became_active":"2024-03-30T01:43:25.302569+0000","last_became_peered":"2024-03-30T01:43:25.302569+0000","last_unstale":"2024-03-30T01:43:26.288512+0000","last_undegraded":"2024-03-30T01:43:26.288512+0000","last_fullsized":"2024-03-30T01:43:26.288512+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:15.339464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292093+0000","last_change":"2024-03-30T01:43:12.277845+0000","last_active":"2024-03-30T01:43:25.292093+0000","last_peered":"2024-03-30T01:43:25.292093+0000","last_clean":"2024-03-30T01:43:25.292093+0000","last_became_active":"2024-03-30T01:43:12.277482+0000","last_became_peered":"2024-03-30T01:43:12.277482+0000","last_unstale":"2024-03-30T01:43:25.292093+0000","last_undegraded":"2024-03-30T01:43:25.292093+0000","last_fullsized":"2024-03-30T01:43:25.292093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319530+0000","last_change":"2024-03-30T01:43:25.315639+0000","last_active":"2024-03-30T01:43:26.319530+0000","last_peered":"2024-03-30T01:43:26.319530+0000","last_clean":"2024-03-30T01:43:26.319530+0000","last_became_active":"2024-03-30T01:43:25.314566+0000","last_became_peered":"2024-03-30T01:43:25.314566+0000","last_unstale":"2024-03-30T01:43:26.319530+0000","last_undegraded":"2024-03-30T01:43:26.319530+0000","last_fullsized":"2024-03-30T01:43:26.319530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:45.202228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289156+0000","last_change":"2024-03-30T01:43:12.276064+0000","last_active":"2024-03-30T01:43:25.289156+0000","last_peered":"2024-03-30T01:43:25.289156+0000","last_clean":"2024-03-30T01:43:25.289156+0000","last_became_active":"2024-03-30T01:43:12.275670+0000","last_became_peered":"2024-03-30T01:43:12.275670+0000","last_unstale":"2024-03-30T01:43:25.289156+0000","last_undegraded":"2024-03-30T01:43:25.289156+0000","last_fullsized":"2024-03-30T01:43:25.289156+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:45.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288025+0000","last_change":"2024-03-30T01:43:25.315581+0000","last_active":"2024-03-30T01:43:26.288025+0000","last_peered":"2024-03-30T01:43:26.288025+0000","last_clean":"2024-03-30T01:43:26.288025+0000","last_became_active":"2024-03-30T01:43:25.315278+0000","last_became_peered":"2024-03-30T01:43:25.315278+0000","last_unstale":"2024-03-30T01:43:26.288025+0000","last_undegraded":"2024-03-30T01:43:26.288025+0000","last_fullsized":"2024-03-30T01:43:26.288025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:21:00.374174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.302660+0000","last_change":"2024-03-30T01:43:12.260768+0000","last_active":"2024-03-30T01:43:25.302660+0000","last_peered":"2024-03-30T01:43:25.302660+0000","last_clean":"2024-03-30T01:43:25.302660+0000","last_became_active":"2024-03-30T01:43:12.260512+0000","last_became_peered":"2024-03-30T01:43:12.260512+0000","last_unstale":"2024-03-30T01:43:25.302660+0000","last_undegraded":"2024-03-30T01:43:25.302660+0000","last_fullsized":"2024-03-30T01:43:25.302660+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:59.579718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319939+0000","last_change":"2024-03-30T01:43:25.306524+0000","last_active":"2024-03-30T01:43:26.319939+0000","last_peered":"2024-03-30T01:43:26.319939+0000","last_clean":"2024-03-30T01:43:26.319939+0000","last_became_active":"2024-03-30T01:43:25.306230+0000","last_became_peered":"2024-03-30T01:43:25.306230+0000","last_unstale":"2024-03-30T01:43:26.319939+0000","last_undegraded":"2024-03-30T01:43:26.319939+0000","last_fullsized":"2024-03-30T01:43:26.319939+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:43:07.270134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"63'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321789+0000","last_change":"2024-03-30T01:43:24.837144+0000","last_active":"2024-03-30T01:43:26.321789+0000","last_peered":"2024-03-30T01:43:26.321789+0000","last_clean":"2024-03-30T01:43:26.321789+0000","last_became_active":"2024-03-30T01:43:24.836814+0000","last_became_peered":"2024-03-30T01:43:24.836814+0000","last_unstale":"2024-03-30T01:43:26.321789+0000","last_undegraded":"2024-03-30T01:43:26.321789+0000","last_fullsized":"2024-03-30T01:43:26.321789+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:20:00.648852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.284075+0000","last_change":"2024-03-30T01:43:12.270602+0000","last_active":"2024-03-30T01:43:25.284075+0000","last_peered":"2024-03-30T01:43:25.284075+0000","last_clean":"2024-03-30T01:43:25.284075+0000","last_became_active":"2024-03-30T01:43:12.270328+0000","last_became_peered":"2024-03-30T01:43:12.270328+0000","last_unstale":"2024-03-30T01:43:25.284075+0000","last_undegraded":"2024-03-30T01:43:25.284075+0000","last_fullsized":"2024-03-30T01:43:25.284075+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:24.040846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.296977+0000","last_change":"2024-03-30T01:43:11.800815+0000","last_active":"2024-03-30T01:43:25.296977+0000","last_peered":"2024-03-30T01:43:25.296977+0000","last_clean":"2024-03-30T01:43:25.296977+0000","last_became_active":"2024-03-30T01:43:11.800527+0000","last_became_peered":"2024-03-30T01:43:11.800527+0000","last_unstale":"2024-03-30T01:43:25.296977+0000","last_undegraded":"2024-03-30T01:43:25.296977+0000","last_fullsized":"2024-03-30T01:43:25.296977+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:36.821233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321415+0000","last_change":"2024-03-30T01:43:25.304957+0000","last_active":"2024-03-30T01:43:26.321415+0000","last_peered":"2024-03-30T01:43:26.321415+0000","last_clean":"2024-03-30T01:43:26.321415+0000","last_became_active":"2024-03-30T01:43:25.304685+0000","last_became_peered":"2024-03-30T01:43:25.304685+0000","last_unstale":"2024-03-30T01:43:26.321415+0000","last_undegraded":"2024-03-30T01:43:26.321415+0000","last_fullsized":"2024-03-30T01:43:26.321415+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:40.255196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"63'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.323539+0000","last_change":"2024-03-30T01:43:24.833874+0000","last_active":"2024-03-30T01:43:26.323539+0000","last_peered":"2024-03-30T01:43:26.323539+0000","last_clean":"2024-03-30T01:43:26.323539+0000","last_became_active":"2024-03-30T01:43:24.833586+0000","last_became_peered":"2024-03-30T01:43:24.833586+0000","last_unstale":"2024-03-30T01:43:26.323539+0000","last_undegraded":"2024-03-30T01:43:26.323539+0000","last_fullsized":"2024-03-30T01:43:26.323539+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:32.556948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289381+0000","last_change":"2024-03-30T01:43:12.268225+0000","last_active":"2024-03-30T01:43:25.289381+0000","last_peered":"2024-03-30T01:43:25.289381+0000","last_clean":"2024-03-30T01:43:25.289381+0000","last_became_active":"2024-03-30T01:43:12.267769+0000","last_became_peered":"2024-03-30T01:43:12.267769+0000","last_unstale":"2024-03-30T01:43:25.289381+0000","last_undegraded":"2024-03-30T01:43:25.289381+0000","last_fullsized":"2024-03-30T01:43:25.289381+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.302929+0000","last_change":"2024-03-30T01:43:11.806420+0000","last_active":"2024-03-30T01:43:25.302929+0000","last_peered":"2024-03-30T01:43:25.302929+0000","last_clean":"2024-03-30T01:43:25.302929+0000","last_became_active":"2024-03-30T01:43:11.806144+0000","last_became_peered":"2024-03-30T01:43:11.806144+0000","last_unstale":"2024-03-30T01:43:25.302929+0000","last_undegraded":"2024-03-30T01:43:25.302929+0000","last_fullsized":"2024-03-30T01:43:25.302929+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:42:19.302319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.286326+0000","last_change":"2024-03-30T01:43:25.308450+0000","last_active":"2024-03-30T01:43:26.286326+0000","last_peered":"2024-03-30T01:43:26.286326+0000","last_clean":"2024-03-30T01:43:26.286326+0000","last_became_active":"2024-03-30T01:43:25.308102+0000","last_became_peered":"2024-03-30T01:43:25.308102+0000","last_unstale":"2024-03-30T01:43:26.286326+0000","last_undegraded":"2024-03-30T01:43:26.286326+0000","last_fullsized":"2024-03-30T01:43:26.286326+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:19:55.642196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"63'4","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.289777+0000","last_change":"2024-03-30T01:43:24.828657+0000","last_active":"2024-03-30T01:43:26.289777+0000","last_peered":"2024-03-30T01:43:26.289777+0000","last_clean":"2024-03-30T01:43:26.289777+0000","last_became_active":"2024-03-30T01:43:24.828346+0000","last_became_peered":"2024-03-30T01:43:24.828346+0000","last_unstale":"2024-03-30T01:43:26.289777+0000","last_undegraded":"2024-03-30T01:43:26.289777+0000","last_fullsized":"2024-03-30T01:43:26.289777+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:24.232259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.305800+0000","last_change":"2024-03-30T01:43:12.259615+0000","last_active":"2024-03-30T01:43:25.305800+0000","last_peered":"2024-03-30T01:43:25.305800+0000","last_clean":"2024-03-30T01:43:25.305800+0000","last_became_active":"2024-03-30T01:43:12.259331+0000","last_became_peered":"2024-03-30T01:43:12.259331+0000","last_unstale":"2024-03-30T01:43:25.305800+0000","last_undegraded":"2024-03-30T01:43:25.305800+0000","last_fullsized":"2024-03-30T01:43:25.305800+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:10.944027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.302914+0000","last_change":"2024-03-30T01:43:11.811391+0000","last_active":"2024-03-30T01:43:25.302914+0000","last_peered":"2024-03-30T01:43:25.302914+0000","last_clean":"2024-03-30T01:43:25.302914+0000","last_became_active":"2024-03-30T01:43:11.810873+0000","last_became_peered":"2024-03-30T01:43:11.810873+0000","last_unstale":"2024-03-30T01:43:25.302914+0000","last_undegraded":"2024-03-30T01:43:25.302914+0000","last_fullsized":"2024-03-30T01:43:25.302914+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:10:22.774118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319419+0000","last_change":"2024-03-30T01:43:25.310343+0000","last_active":"2024-03-30T01:43:26.319419+0000","last_peered":"2024-03-30T01:43:26.319419+0000","last_clean":"2024-03-30T01:43:26.319419+0000","last_became_active":"2024-03-30T01:43:25.310050+0000","last_became_peered":"2024-03-30T01:43:25.310050+0000","last_unstale":"2024-03-30T01:43:26.319419+0000","last_undegraded":"2024-03-30T01:43:26.319419+0000","last_fullsized":"2024-03-30T01:43:26.319419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:53:05.857691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"63'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321826+0000","last_change":"2024-03-30T01:43:24.837551+0000","last_active":"2024-03-30T01:43:26.321826+0000","last_peered":"2024-03-30T01:43:26.321826+0000","last_clean":"2024-03-30T01:43:26.321826+0000","last_became_active":"2024-03-30T01:43:24.837273+0000","last_became_peered":"2024-03-30T01:43:24.837273+0000","last_unstale":"2024-03-30T01:43:26.321826+0000","last_undegraded":"2024-03-30T01:43:26.321826+0000","last_fullsized":"2024-03-30T01:43:26.321826+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:32.610715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.293745+0000","last_change":"2024-03-30T01:43:12.276578+0000","last_active":"2024-03-30T01:43:25.293745+0000","last_peered":"2024-03-30T01:43:25.293745+0000","last_clean":"2024-03-30T01:43:25.293745+0000","last_became_active":"2024-03-30T01:43:12.276293+0000","last_became_peered":"2024-03-30T01:43:12.276293+0000","last_unstale":"2024-03-30T01:43:25.293745+0000","last_undegraded":"2024-03-30T01:43:25.293745+0000","last_fullsized":"2024-03-30T01:43:25.293745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.306170+0000","last_change":"2024-03-30T01:43:11.803525+0000","last_active":"2024-03-30T01:43:25.306170+0000","last_peered":"2024-03-30T01:43:25.306170+0000","last_clean":"2024-03-30T01:43:25.306170+0000","last_became_active":"2024-03-30T01:43:11.803115+0000","last_became_peered":"2024-03-30T01:43:11.803115+0000","last_unstale":"2024-03-30T01:43:25.306170+0000","last_undegraded":"2024-03-30T01:43:25.306170+0000","last_fullsized":"2024-03-30T01:43:25.306170+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:10.022632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321199+0000","last_change":"2024-03-30T01:43:25.306566+0000","last_active":"2024-03-30T01:43:26.321199+0000","last_peered":"2024-03-30T01:43:26.321199+0000","last_clean":"2024-03-30T01:43:26.321199+0000","last_became_active":"2024-03-30T01:43:25.305725+0000","last_became_peered":"2024-03-30T01:43:25.305725+0000","last_unstale":"2024-03-30T01:43:26.321199+0000","last_undegraded":"2024-03-30T01:43:26.321199+0000","last_fullsized":"2024-03-30T01:43:26.321199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:08.899825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319234+0000","last_change":"2024-03-30T01:43:24.834239+0000","last_active":"2024-03-30T01:43:26.319234+0000","last_peered":"2024-03-30T01:43:26.319234+0000","last_clean":"2024-03-30T01:43:26.319234+0000","last_became_active":"2024-03-30T01:43:24.832934+0000","last_became_peered":"2024-03-30T01:43:24.832934+0000","last_unstale":"2024-03-30T01:43:26.319234+0000","last_undegraded":"2024-03-30T01:43:26.319234+0000","last_fullsized":"2024-03-30T01:43:26.319234+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:41:14.800040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.287875+0000","last_change":"2024-03-30T01:43:12.267451+0000","last_active":"2024-03-30T01:43:25.287875+0000","last_peered":"2024-03-30T01:43:25.287875+0000","last_clean":"2024-03-30T01:43:25.287875+0000","last_became_active":"2024-03-30T01:43:12.267249+0000","last_became_peered":"2024-03-30T01:43:12.267249+0000","last_unstale":"2024-03-30T01:43:25.287875+0000","last_undegraded":"2024-03-30T01:43:25.287875+0000","last_fullsized":"2024-03-30T01:43:25.287875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:28:49.333290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.291245+0000","last_change":"2024-03-30T01:43:11.804981+0000","last_active":"2024-03-30T01:43:25.291245+0000","last_peered":"2024-03-30T01:43:25.291245+0000","last_clean":"2024-03-30T01:43:25.291245+0000","last_became_active":"2024-03-30T01:43:11.804690+0000","last_became_peered":"2024-03-30T01:43:11.804690+0000","last_unstale":"2024-03-30T01:43:25.291245+0000","last_undegraded":"2024-03-30T01:43:25.291245+0000","last_fullsized":"2024-03-30T01:43:25.291245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:25:59.487668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.324625+0000","last_change":"2024-03-30T01:43:25.296856+0000","last_active":"2024-03-30T01:43:26.324625+0000","last_peered":"2024-03-30T01:43:26.324625+0000","last_clean":"2024-03-30T01:43:26.324625+0000","last_became_active":"2024-03-30T01:43:25.296581+0000","last_became_peered":"2024-03-30T01:43:25.296581+0000","last_unstale":"2024-03-30T01:43:26.324625+0000","last_undegraded":"2024-03-30T01:43:26.324625+0000","last_fullsized":"2024-03-30T01:43:26.324625+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:58:12.846403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287580+0000","last_change":"2024-03-30T01:43:24.838640+0000","last_active":"2024-03-30T01:43:26.287580+0000","last_peered":"2024-03-30T01:43:26.287580+0000","last_clean":"2024-03-30T01:43:26.287580+0000","last_became_active":"2024-03-30T01:43:24.838409+0000","last_became_peered":"2024-03-30T01:43:24.838409+0000","last_unstale":"2024-03-30T01:43:26.287580+0000","last_undegraded":"2024-03-30T01:43:26.287580+0000","last_fullsized":"2024-03-30T01:43:26.287580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:31:02.433685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.288464+0000","last_change":"2024-03-30T01:43:12.268426+0000","last_active":"2024-03-30T01:43:25.288464+0000","last_peered":"2024-03-30T01:43:25.288464+0000","last_clean":"2024-03-30T01:43:25.288464+0000","last_became_active":"2024-03-30T01:43:12.268106+0000","last_became_peered":"2024-03-30T01:43:12.268106+0000","last_unstale":"2024-03-30T01:43:25.288464+0000","last_undegraded":"2024-03-30T01:43:25.288464+0000","last_fullsized":"2024-03-30T01:43:25.288464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:32:07.610941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.291683+0000","last_change":"2024-03-30T01:43:11.798840+0000","last_active":"2024-03-30T01:43:25.291683+0000","last_peered":"2024-03-30T01:43:25.291683+0000","last_clean":"2024-03-30T01:43:25.291683+0000","last_became_active":"2024-03-30T01:43:11.798493+0000","last_became_peered":"2024-03-30T01:43:11.798493+0000","last_unstale":"2024-03-30T01:43:25.291683+0000","last_undegraded":"2024-03-30T01:43:25.291683+0000","last_fullsized":"2024-03-30T01:43:25.291683+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:35.032315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319704+0000","last_change":"2024-03-30T01:43:25.316246+0000","last_active":"2024-03-30T01:43:26.319704+0000","last_peered":"2024-03-30T01:43:26.319704+0000","last_clean":"2024-03-30T01:43:26.319704+0000","last_became_active":"2024-03-30T01:43:25.315949+0000","last_became_peered":"2024-03-30T01:43:25.315949+0000","last_unstale":"2024-03-30T01:43:26.319704+0000","last_undegraded":"2024-03-30T01:43:26.319704+0000","last_fullsized":"2024-03-30T01:43:26.319704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:40:01.596232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"63'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320071+0000","last_change":"2024-03-30T01:43:24.836482+0000","last_active":"2024-03-30T01:43:26.320071+0000","last_peered":"2024-03-30T01:43:26.320071+0000","last_clean":"2024-03-30T01:43:26.320071+0000","last_became_active":"2024-03-30T01:43:24.836146+0000","last_became_peered":"2024-03-30T01:43:24.836146+0000","last_unstale":"2024-03-30T01:43:26.320071+0000","last_undegraded":"2024-03-30T01:43:26.320071+0000","last_fullsized":"2024-03-30T01:43:26.320071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:04.183169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.282778+0000","last_change":"2024-03-30T01:43:12.272426+0000","last_active":"2024-03-30T01:43:25.282778+0000","last_peered":"2024-03-30T01:43:25.282778+0000","last_clean":"2024-03-30T01:43:25.282778+0000","last_became_active":"2024-03-30T01:43:12.272160+0000","last_became_peered":"2024-03-30T01:43:12.272160+0000","last_unstale":"2024-03-30T01:43:25.282778+0000","last_undegraded":"2024-03-30T01:43:25.282778+0000","last_fullsized":"2024-03-30T01:43:25.282778+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:37.825651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.290701+0000","last_change":"2024-03-30T01:43:11.805468+0000","last_active":"2024-03-30T01:43:25.290701+0000","last_peered":"2024-03-30T01:43:25.290701+0000","last_clean":"2024-03-30T01:43:25.290701+0000","last_became_active":"2024-03-30T01:43:11.804984+0000","last_became_peered":"2024-03-30T01:43:11.804984+0000","last_unstale":"2024-03-30T01:43:25.290701+0000","last_undegraded":"2024-03-30T01:43:25.290701+0000","last_fullsized":"2024-03-30T01:43:25.290701+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:16.171589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320773+0000","last_change":"2024-03-30T01:43:25.314544+0000","last_active":"2024-03-30T01:43:26.320773+0000","last_peered":"2024-03-30T01:43:26.320773+0000","last_clean":"2024-03-30T01:43:26.320773+0000","last_became_active":"2024-03-30T01:43:25.314235+0000","last_became_peered":"2024-03-30T01:43:25.314235+0000","last_unstale":"2024-03-30T01:43:26.320773+0000","last_undegraded":"2024-03-30T01:43:26.320773+0000","last_fullsized":"2024-03-30T01:43:26.320773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:23.399761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"63'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287603+0000","last_change":"2024-03-30T01:43:24.832027+0000","last_active":"2024-03-30T01:43:26.287603+0000","last_peered":"2024-03-30T01:43:26.287603+0000","last_clean":"2024-03-30T01:43:26.287603+0000","last_became_active":"2024-03-30T01:43:24.831691+0000","last_became_peered":"2024-03-30T01:43:24.831691+0000","last_unstale":"2024-03-30T01:43:26.287603+0000","last_undegraded":"2024-03-30T01:43:26.287603+0000","last_fullsized":"2024-03-30T01:43:26.287603+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:50.773482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.291149+0000","last_change":"2024-03-30T01:43:12.265005+0000","last_active":"2024-03-30T01:43:25.291149+0000","last_peered":"2024-03-30T01:43:25.291149+0000","last_clean":"2024-03-30T01:43:25.291149+0000","last_became_active":"2024-03-30T01:43:12.264726+0000","last_became_peered":"2024-03-30T01:43:12.264726+0000","last_unstale":"2024-03-30T01:43:25.291149+0000","last_undegraded":"2024-03-30T01:43:25.291149+0000","last_fullsized":"2024-03-30T01:43:25.291149+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:41:19.989099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.282945+0000","last_change":"2024-03-30T01:43:11.799020+0000","last_active":"2024-03-30T01:43:25.282945+0000","last_peered":"2024-03-30T01:43:25.282945+0000","last_clean":"2024-03-30T01:43:25.282945+0000","last_became_active":"2024-03-30T01:43:11.797840+0000","last_became_peered":"2024-03-30T01:43:11.797840+0000","last_unstale":"2024-03-30T01:43:25.282945+0000","last_undegraded":"2024-03-30T01:43:25.282945+0000","last_fullsized":"2024-03-30T01:43:25.282945+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:48:17.963383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319992+0000","last_change":"2024-03-30T01:43:25.300562+0000","last_active":"2024-03-30T01:43:26.319992+0000","last_peered":"2024-03-30T01:43:26.319992+0000","last_clean":"2024-03-30T01:43:26.319992+0000","last_became_active":"2024-03-30T01:43:25.300287+0000","last_became_peered":"2024-03-30T01:43:25.300287+0000","last_unstale":"2024-03-30T01:43:26.319992+0000","last_undegraded":"2024-03-30T01:43:26.319992+0000","last_fullsized":"2024-03-30T01:43:26.319992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:57.246223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"63'3","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.289282+0000","last_change":"2024-03-30T01:43:24.835287+0000","last_active":"2024-03-30T01:43:26.289282+0000","last_peered":"2024-03-30T01:43:26.289282+0000","last_clean":"2024-03-30T01:43:26.289282+0000","last_became_active":"2024-03-30T01:43:24.835000+0000","last_became_peered":"2024-03-30T01:43:24.835000+0000","last_unstale":"2024-03-30T01:43:26.289282+0000","last_undegraded":"2024-03-30T01:43:26.289282+0000","last_fullsized":"2024-03-30T01:43:26.289282+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:26.923623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289219+0000","last_change":"2024-03-30T01:43:12.265237+0000","last_active":"2024-03-30T01:43:25.289219+0000","last_peered":"2024-03-30T01:43:25.289219+0000","last_clean":"2024-03-30T01:43:25.289219+0000","last_became_active":"2024-03-30T01:43:12.264995+0000","last_became_peered":"2024-03-30T01:43:12.264995+0000","last_unstale":"2024-03-30T01:43:25.289219+0000","last_undegraded":"2024-03-30T01:43:25.289219+0000","last_fullsized":"2024-03-30T01:43:25.289219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:22.071009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.293839+0000","last_change":"2024-03-30T01:43:11.803287+0000","last_active":"2024-03-30T01:43:25.293839+0000","last_peered":"2024-03-30T01:43:25.293839+0000","last_clean":"2024-03-30T01:43:25.293839+0000","last_became_active":"2024-03-30T01:43:11.803067+0000","last_became_peered":"2024-03-30T01:43:11.803067+0000","last_unstale":"2024-03-30T01:43:25.293839+0000","last_undegraded":"2024-03-30T01:43:25.293839+0000","last_fullsized":"2024-03-30T01:43:25.293839+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:17:36.821233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.286960+0000","last_change":"2024-03-30T01:43:25.317087+0000","last_active":"2024-03-30T01:43:26.286960+0000","last_peered":"2024-03-30T01:43:26.286960+0000","last_clean":"2024-03-30T01:43:26.286960+0000","last_became_active":"2024-03-30T01:43:25.316816+0000","last_became_peered":"2024-03-30T01:43:25.316816+0000","last_unstale":"2024-03-30T01:43:26.286960+0000","last_undegraded":"2024-03-30T01:43:26.286960+0000","last_fullsized":"2024-03-30T01:43:26.286960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:37.403248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"63'6","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.323640+0000","last_change":"2024-03-30T01:43:24.838350+0000","last_active":"2024-03-30T01:43:26.323640+0000","last_peered":"2024-03-30T01:43:26.323640+0000","last_clean":"2024-03-30T01:43:26.323640+0000","last_became_active":"2024-03-30T01:43:24.837988+0000","last_became_peered":"2024-03-30T01:43:24.837988+0000","last_unstale":"2024-03-30T01:43:26.323640+0000","last_undegraded":"2024-03-30T01:43:26.323640+0000","last_fullsized":"2024-03-30T01:43:26.323640+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:56.999221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292552+0000","last_change":"2024-03-30T01:43:12.275223+0000","last_active":"2024-03-30T01:43:25.292552+0000","last_peered":"2024-03-30T01:43:25.292552+0000","last_clean":"2024-03-30T01:43:25.292552+0000","last_became_active":"2024-03-30T01:43:12.274949+0000","last_became_peered":"2024-03-30T01:43:12.274949+0000","last_unstale":"2024-03-30T01:43:25.292552+0000","last_undegraded":"2024-03-30T01:43:25.292552+0000","last_fullsized":"2024-03-30T01:43:25.292552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:34.024584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.290569+0000","last_change":"2024-03-30T01:43:11.802972+0000","last_active":"2024-03-30T01:43:25.290569+0000","last_peered":"2024-03-30T01:43:25.290569+0000","last_clean":"2024-03-30T01:43:25.290569+0000","last_became_active":"2024-03-30T01:43:11.802665+0000","last_became_peered":"2024-03-30T01:43:11.802665+0000","last_unstale":"2024-03-30T01:43:25.290569+0000","last_undegraded":"2024-03-30T01:43:25.290569+0000","last_fullsized":"2024-03-30T01:43:25.290569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:24.494514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319033+0000","last_change":"2024-03-30T01:43:25.314742+0000","last_active":"2024-03-30T01:43:26.319033+0000","last_peered":"2024-03-30T01:43:26.319033+0000","last_clean":"2024-03-30T01:43:26.319033+0000","last_became_active":"2024-03-30T01:43:25.314422+0000","last_became_peered":"2024-03-30T01:43:25.314422+0000","last_unstale":"2024-03-30T01:43:26.319033+0000","last_undegraded":"2024-03-30T01:43:26.319033+0000","last_fullsized":"2024-03-30T01:43:26.319033+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:54:12.193149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.322070+0000","last_change":"2024-03-30T01:43:24.835932+0000","last_active":"2024-03-30T01:43:26.322070+0000","last_peered":"2024-03-30T01:43:26.322070+0000","last_clean":"2024-03-30T01:43:26.322070+0000","last_became_active":"2024-03-30T01:43:24.835647+0000","last_became_peered":"2024-03-30T01:43:24.835647+0000","last_unstale":"2024-03-30T01:43:26.322070+0000","last_undegraded":"2024-03-30T01:43:26.322070+0000","last_fullsized":"2024-03-30T01:43:26.322070+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:04.183169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.284814+0000","last_change":"2024-03-30T01:43:12.271580+0000","last_active":"2024-03-30T01:43:25.284814+0000","last_peered":"2024-03-30T01:43:25.284814+0000","last_clean":"2024-03-30T01:43:25.284814+0000","last_became_active":"2024-03-30T01:43:12.271342+0000","last_became_peered":"2024-03-30T01:43:12.271342+0000","last_unstale":"2024-03-30T01:43:25.284814+0000","last_undegraded":"2024-03-30T01:43:25.284814+0000","last_fullsized":"2024-03-30T01:43:25.284814+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:25.498933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.295742+0000","last_change":"2024-03-30T01:43:11.802252+0000","last_active":"2024-03-30T01:43:25.295742+0000","last_peered":"2024-03-30T01:43:25.295742+0000","last_clean":"2024-03-30T01:43:25.295742+0000","last_became_active":"2024-03-30T01:43:11.801845+0000","last_became_peered":"2024-03-30T01:43:11.801845+0000","last_unstale":"2024-03-30T01:43:25.295742+0000","last_undegraded":"2024-03-30T01:43:25.295742+0000","last_fullsized":"2024-03-30T01:43:25.295742+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287371+0000","last_change":"2024-03-30T01:43:25.308307+0000","last_active":"2024-03-30T01:43:26.287371+0000","last_peered":"2024-03-30T01:43:26.287371+0000","last_clean":"2024-03-30T01:43:26.287371+0000","last_became_active":"2024-03-30T01:43:25.307948+0000","last_became_peered":"2024-03-30T01:43:25.307948+0000","last_unstale":"2024-03-30T01:43:26.287371+0000","last_undegraded":"2024-03-30T01:43:26.287371+0000","last_fullsized":"2024-03-30T01:43:26.287371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:31:39.501161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288276+0000","last_change":"2024-03-30T01:43:24.831307+0000","last_active":"2024-03-30T01:43:26.288276+0000","last_peered":"2024-03-30T01:43:26.288276+0000","last_clean":"2024-03-30T01:43:26.288276+0000","last_became_active":"2024-03-30T01:43:24.831010+0000","last_became_peered":"2024-03-30T01:43:24.831010+0000","last_unstale":"2024-03-30T01:43:26.288276+0000","last_undegraded":"2024-03-30T01:43:26.288276+0000","last_fullsized":"2024-03-30T01:43:26.288276+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:36.393225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.290698+0000","last_change":"2024-03-30T01:43:12.268100+0000","last_active":"2024-03-30T01:43:25.290698+0000","last_peered":"2024-03-30T01:43:25.290698+0000","last_clean":"2024-03-30T01:43:25.290698+0000","last_became_active":"2024-03-30T01:43:12.267755+0000","last_became_peered":"2024-03-30T01:43:12.267755+0000","last_unstale":"2024-03-30T01:43:25.290698+0000","last_undegraded":"2024-03-30T01:43:25.290698+0000","last_fullsized":"2024-03-30T01:43:25.290698+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:09.141688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'8","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289307+0000","last_change":"2024-03-30T01:43:11.806909+0000","last_active":"2024-03-30T01:43:25.289307+0000","last_peered":"2024-03-30T01:43:25.289307+0000","last_clean":"2024-03-30T01:43:25.289307+0000","last_became_active":"2024-03-30T01:43:11.806609+0000","last_became_peered":"2024-03-30T01:43:11.806609+0000","last_unstale":"2024-03-30T01:43:25.289307+0000","last_undegraded":"2024-03-30T01:43:25.289307+0000","last_fullsized":"2024-03-30T01:43:25.289307+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.323438+0000","last_change":"2024-03-30T01:43:25.304703+0000","last_active":"2024-03-30T01:43:26.323438+0000","last_peered":"2024-03-30T01:43:26.323438+0000","last_clean":"2024-03-30T01:43:26.323438+0000","last_became_active":"2024-03-30T01:43:25.304403+0000","last_became_peered":"2024-03-30T01:43:25.304403+0000","last_unstale":"2024-03-30T01:43:26.323438+0000","last_undegraded":"2024-03-30T01:43:26.323438+0000","last_fullsized":"2024-03-30T01:43:26.323438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:53:45.874161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287649+0000","last_change":"2024-03-30T01:43:24.837414+0000","last_active":"2024-03-30T01:43:26.287649+0000","last_peered":"2024-03-30T01:43:26.287649+0000","last_clean":"2024-03-30T01:43:26.287649+0000","last_became_active":"2024-03-30T01:43:24.837129+0000","last_became_peered":"2024-03-30T01:43:24.837129+0000","last_unstale":"2024-03-30T01:43:26.287649+0000","last_undegraded":"2024-03-30T01:43:26.287649+0000","last_fullsized":"2024-03-30T01:43:26.287649+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:05:33.024418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289003+0000","last_change":"2024-03-30T01:43:12.280292+0000","last_active":"2024-03-30T01:43:25.289003+0000","last_peered":"2024-03-30T01:43:25.289003+0000","last_clean":"2024-03-30T01:43:25.289003+0000","last_became_active":"2024-03-30T01:43:12.279966+0000","last_became_peered":"2024-03-30T01:43:12.279966+0000","last_unstale":"2024-03-30T01:43:25.289003+0000","last_undegraded":"2024-03-30T01:43:25.289003+0000","last_fullsized":"2024-03-30T01:43:25.289003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.294955+0000","last_change":"2024-03-30T01:43:11.804635+0000","last_active":"2024-03-30T01:43:25.294955+0000","last_peered":"2024-03-30T01:43:25.294955+0000","last_clean":"2024-03-30T01:43:25.294955+0000","last_became_active":"2024-03-30T01:43:11.804355+0000","last_became_peered":"2024-03-30T01:43:11.804355+0000","last_unstale":"2024-03-30T01:43:25.294955+0000","last_undegraded":"2024-03-30T01:43:25.294955+0000","last_fullsized":"2024-03-30T01:43:25.294955+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:57.117126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.324523+0000","last_change":"2024-03-30T01:43:25.308738+0000","last_active":"2024-03-30T01:43:26.324523+0000","last_peered":"2024-03-30T01:43:26.324523+0000","last_clean":"2024-03-30T01:43:26.324523+0000","last_became_active":"2024-03-30T01:43:25.308492+0000","last_became_peered":"2024-03-30T01:43:25.308492+0000","last_unstale":"2024-03-30T01:43:26.324523+0000","last_undegraded":"2024-03-30T01:43:26.324523+0000","last_fullsized":"2024-03-30T01:43:26.324523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:22.896399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"63'8","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319147+0000","last_change":"2024-03-30T01:43:24.832896+0000","last_active":"2024-03-30T01:43:26.319147+0000","last_peered":"2024-03-30T01:43:26.319147+0000","last_clean":"2024-03-30T01:43:26.319147+0000","last_became_active":"2024-03-30T01:43:24.832583+0000","last_became_peered":"2024-03-30T01:43:24.832583+0000","last_unstale":"2024-03-30T01:43:26.319147+0000","last_undegraded":"2024-03-30T01:43:26.319147+0000","last_fullsized":"2024-03-30T01:43:26.319147+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:02:27.382005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.294100+0000","last_change":"2024-03-30T01:43:12.268978+0000","last_active":"2024-03-30T01:43:25.294100+0000","last_peered":"2024-03-30T01:43:25.294100+0000","last_clean":"2024-03-30T01:43:25.294100+0000","last_became_active":"2024-03-30T01:43:12.268633+0000","last_became_peered":"2024-03-30T01:43:12.268633+0000","last_unstale":"2024-03-30T01:43:25.294100+0000","last_undegraded":"2024-03-30T01:43:25.294100+0000","last_fullsized":"2024-03-30T01:43:25.294100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:26:00.492086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292163+0000","last_change":"2024-03-30T01:43:11.811100+0000","last_active":"2024-03-30T01:43:25.292163+0000","last_peered":"2024-03-30T01:43:25.292163+0000","last_clean":"2024-03-30T01:43:25.292163+0000","last_became_active":"2024-03-30T01:43:11.810811+0000","last_became_peered":"2024-03-30T01:43:11.810811+0000","last_unstale":"2024-03-30T01:43:25.292163+0000","last_undegraded":"2024-03-30T01:43:25.292163+0000","last_fullsized":"2024-03-30T01:43:25.292163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288185+0000","last_change":"2024-03-30T01:43:25.306036+0000","last_active":"2024-03-30T01:43:26.288185+0000","last_peered":"2024-03-30T01:43:26.288185+0000","last_clean":"2024-03-30T01:43:26.288185+0000","last_became_active":"2024-03-30T01:43:25.305708+0000","last_became_peered":"2024-03-30T01:43:25.305708+0000","last_unstale":"2024-03-30T01:43:26.288185+0000","last_undegraded":"2024-03-30T01:43:26.288185+0000","last_fullsized":"2024-03-30T01:43:26.288185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:16.619651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"63'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287745+0000","last_change":"2024-03-30T01:43:24.838780+0000","last_active":"2024-03-30T01:43:26.287745+0000","last_peered":"2024-03-30T01:43:26.287745+0000","last_clean":"2024-03-30T01:43:26.287745+0000","last_became_active":"2024-03-30T01:43:24.838431+0000","last_became_peered":"2024-03-30T01:43:24.838431+0000","last_unstale":"2024-03-30T01:43:26.287745+0000","last_undegraded":"2024-03-30T01:43:26.287745+0000","last_fullsized":"2024-03-30T01:43:26.287745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:04.183169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.282951+0000","last_change":"2024-03-30T01:43:12.265729+0000","last_active":"2024-03-30T01:43:25.282951+0000","last_peered":"2024-03-30T01:43:25.282951+0000","last_clean":"2024-03-30T01:43:25.282951+0000","last_became_active":"2024-03-30T01:43:12.265427+0000","last_became_peered":"2024-03-30T01:43:12.265427+0000","last_unstale":"2024-03-30T01:43:25.282951+0000","last_undegraded":"2024-03-30T01:43:25.282951+0000","last_fullsized":"2024-03-30T01:43:25.282951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:40:39.587585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289788+0000","last_change":"2024-03-30T01:43:11.802626+0000","last_active":"2024-03-30T01:43:25.289788+0000","last_peered":"2024-03-30T01:43:25.289788+0000","last_clean":"2024-03-30T01:43:25.289788+0000","last_became_active":"2024-03-30T01:43:11.802314+0000","last_became_peered":"2024-03-30T01:43:11.802314+0000","last_unstale":"2024-03-30T01:43:25.289788+0000","last_undegraded":"2024-03-30T01:43:25.289788+0000","last_fullsized":"2024-03-30T01:43:25.289788+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:35.547797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.324207+0000","last_change":"2024-03-30T01:43:25.300299+0000","last_active":"2024-03-30T01:43:26.324207+0000","last_peered":"2024-03-30T01:43:26.324207+0000","last_clean":"2024-03-30T01:43:26.324207+0000","last_became_active":"2024-03-30T01:43:25.299967+0000","last_became_peered":"2024-03-30T01:43:25.299967+0000","last_unstale":"2024-03-30T01:43:26.324207+0000","last_undegraded":"2024-03-30T01:43:26.324207+0000","last_fullsized":"2024-03-30T01:43:26.324207+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:12.606476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321136+0000","last_change":"2024-03-30T01:43:24.835652+0000","last_active":"2024-03-30T01:43:26.321136+0000","last_peered":"2024-03-30T01:43:26.321136+0000","last_clean":"2024-03-30T01:43:26.321136+0000","last_became_active":"2024-03-30T01:43:24.835373+0000","last_became_peered":"2024-03-30T01:43:24.835373+0000","last_unstale":"2024-03-30T01:43:26.321136+0000","last_undegraded":"2024-03-30T01:43:26.321136+0000","last_fullsized":"2024-03-30T01:43:26.321136+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:36.393225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.288342+0000","last_change":"2024-03-30T01:43:12.274115+0000","last_active":"2024-03-30T01:43:25.288342+0000","last_peered":"2024-03-30T01:43:25.288342+0000","last_clean":"2024-03-30T01:43:25.288342+0000","last_became_active":"2024-03-30T01:43:12.273842+0000","last_became_peered":"2024-03-30T01:43:12.273842+0000","last_unstale":"2024-03-30T01:43:25.288342+0000","last_undegraded":"2024-03-30T01:43:25.288342+0000","last_fullsized":"2024-03-30T01:43:25.288342+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:44:49.086267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.288528+0000","last_change":"2024-03-30T01:43:11.802309+0000","last_active":"2024-03-30T01:43:25.288528+0000","last_peered":"2024-03-30T01:43:25.288528+0000","last_clean":"2024-03-30T01:43:25.288528+0000","last_became_active":"2024-03-30T01:43:11.802008+0000","last_became_peered":"2024-03-30T01:43:11.802008+0000","last_unstale":"2024-03-30T01:43:25.288528+0000","last_undegraded":"2024-03-30T01:43:25.288528+0000","last_fullsized":"2024-03-30T01:43:25.288528+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:11.268414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287936+0000","last_change":"2024-03-30T01:43:25.318335+0000","last_active":"2024-03-30T01:43:26.287936+0000","last_peered":"2024-03-30T01:43:26.287936+0000","last_clean":"2024-03-30T01:43:26.287936+0000","last_became_active":"2024-03-30T01:43:25.318056+0000","last_became_peered":"2024-03-30T01:43:25.318056+0000","last_unstale":"2024-03-30T01:43:26.287936+0000","last_undegraded":"2024-03-30T01:43:26.287936+0000","last_fullsized":"2024-03-30T01:43:26.287936+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:49:22.526179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"63'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.289137+0000","last_change":"2024-03-30T01:43:24.838600+0000","last_active":"2024-03-30T01:43:26.289137+0000","last_peered":"2024-03-30T01:43:26.289137+0000","last_clean":"2024-03-30T01:43:26.289137+0000","last_became_active":"2024-03-30T01:43:24.838297+0000","last_became_peered":"2024-03-30T01:43:24.838297+0000","last_unstale":"2024-03-30T01:43:26.289137+0000","last_undegraded":"2024-03-30T01:43:26.289137+0000","last_fullsized":"2024-03-30T01:43:26.289137+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:01.784462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292311+0000","last_change":"2024-03-30T01:43:12.279554+0000","last_active":"2024-03-30T01:43:25.292311+0000","last_peered":"2024-03-30T01:43:25.292311+0000","last_clean":"2024-03-30T01:43:25.292311+0000","last_became_active":"2024-03-30T01:43:12.279266+0000","last_became_peered":"2024-03-30T01:43:12.279266+0000","last_unstale":"2024-03-30T01:43:25.292311+0000","last_undegraded":"2024-03-30T01:43:25.292311+0000","last_fullsized":"2024-03-30T01:43:25.292311+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:57.400850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.294486+0000","last_change":"2024-03-30T01:43:11.809877+0000","last_active":"2024-03-30T01:43:25.294486+0000","last_peered":"2024-03-30T01:43:25.294486+0000","last_clean":"2024-03-30T01:43:25.294486+0000","last_became_active":"2024-03-30T01:43:11.808130+0000","last_became_peered":"2024-03-30T01:43:11.808130+0000","last_unstale":"2024-03-30T01:43:25.294486+0000","last_undegraded":"2024-03-30T01:43:25.294486+0000","last_fullsized":"2024-03-30T01:43:25.294486+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319738+0000","last_change":"2024-03-30T01:43:25.294359+0000","last_active":"2024-03-30T01:43:26.319738+0000","last_peered":"2024-03-30T01:43:26.319738+0000","last_clean":"2024-03-30T01:43:26.319738+0000","last_became_active":"2024-03-30T01:43:25.294005+0000","last_became_peered":"2024-03-30T01:43:25.294005+0000","last_unstale":"2024-03-30T01:43:26.319738+0000","last_undegraded":"2024-03-30T01:43:26.319738+0000","last_fullsized":"2024-03-30T01:43:26.319738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:09:43.918420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287155+0000","last_change":"2024-03-30T01:43:24.834200+0000","last_active":"2024-03-30T01:43:26.287155+0000","last_peered":"2024-03-30T01:43:26.287155+0000","last_clean":"2024-03-30T01:43:26.287155+0000","last_became_active":"2024-03-30T01:43:24.832654+0000","last_became_peered":"2024-03-30T01:43:24.832654+0000","last_unstale":"2024-03-30T01:43:26.287155+0000","last_undegraded":"2024-03-30T01:43:26.287155+0000","last_fullsized":"2024-03-30T01:43:26.287155+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:25.735352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.284310+0000","last_change":"2024-03-30T01:43:12.275996+0000","last_active":"2024-03-30T01:43:25.284310+0000","last_peered":"2024-03-30T01:43:25.284310+0000","last_clean":"2024-03-30T01:43:25.284310+0000","last_became_active":"2024-03-30T01:43:12.275705+0000","last_became_peered":"2024-03-30T01:43:12.275705+0000","last_unstale":"2024-03-30T01:43:25.284310+0000","last_undegraded":"2024-03-30T01:43:25.284310+0000","last_fullsized":"2024-03-30T01:43:25.284310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:23:47.117365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292314+0000","last_change":"2024-03-30T01:43:11.806234+0000","last_active":"2024-03-30T01:43:25.292314+0000","last_peered":"2024-03-30T01:43:25.292314+0000","last_clean":"2024-03-30T01:43:25.292314+0000","last_became_active":"2024-03-30T01:43:11.806012+0000","last_became_peered":"2024-03-30T01:43:11.806012+0000","last_unstale":"2024-03-30T01:43:25.292314+0000","last_undegraded":"2024-03-30T01:43:25.292314+0000","last_fullsized":"2024-03-30T01:43:25.292314+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:08:24.494514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319247+0000","last_change":"2024-03-30T01:43:25.311864+0000","last_active":"2024-03-30T01:43:26.319247+0000","last_peered":"2024-03-30T01:43:26.319247+0000","last_clean":"2024-03-30T01:43:26.319247+0000","last_became_active":"2024-03-30T01:43:25.311564+0000","last_became_peered":"2024-03-30T01:43:25.311564+0000","last_unstale":"2024-03-30T01:43:26.319247+0000","last_undegraded":"2024-03-30T01:43:26.319247+0000","last_fullsized":"2024-03-30T01:43:26.319247+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:30:54.743748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"63'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.322239+0000","last_change":"2024-03-30T01:43:24.835363+0000","last_active":"2024-03-30T01:43:26.322239+0000","last_peered":"2024-03-30T01:43:26.322239+0000","last_clean":"2024-03-30T01:43:26.322239+0000","last_became_active":"2024-03-30T01:43:24.835086+0000","last_became_peered":"2024-03-30T01:43:24.835086+0000","last_unstale":"2024-03-30T01:43:26.322239+0000","last_undegraded":"2024-03-30T01:43:26.322239+0000","last_fullsized":"2024-03-30T01:43:26.322239+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:21.886375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.290435+0000","last_change":"2024-03-30T01:43:12.271940+0000","last_active":"2024-03-30T01:43:25.290435+0000","last_peered":"2024-03-30T01:43:25.290435+0000","last_clean":"2024-03-30T01:43:25.290435+0000","last_became_active":"2024-03-30T01:43:12.268234+0000","last_became_peered":"2024-03-30T01:43:12.268234+0000","last_unstale":"2024-03-30T01:43:25.290435+0000","last_undegraded":"2024-03-30T01:43:25.290435+0000","last_fullsized":"2024-03-30T01:43:25.290435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:59:36.036734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.295892+0000","last_change":"2024-03-30T01:43:11.803111+0000","last_active":"2024-03-30T01:43:25.295892+0000","last_peered":"2024-03-30T01:43:25.295892+0000","last_clean":"2024-03-30T01:43:25.295892+0000","last_became_active":"2024-03-30T01:43:11.802781+0000","last_became_peered":"2024-03-30T01:43:11.802781+0000","last_unstale":"2024-03-30T01:43:25.295892+0000","last_undegraded":"2024-03-30T01:43:25.295892+0000","last_fullsized":"2024-03-30T01:43:25.295892+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:30.914337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287607+0000","last_change":"2024-03-30T01:43:25.303821+0000","last_active":"2024-03-30T01:43:26.287607+0000","last_peered":"2024-03-30T01:43:26.287607+0000","last_clean":"2024-03-30T01:43:26.287607+0000","last_became_active":"2024-03-30T01:43:25.303536+0000","last_became_peered":"2024-03-30T01:43:25.303536+0000","last_unstale":"2024-03-30T01:43:26.287607+0000","last_undegraded":"2024-03-30T01:43:26.287607+0000","last_fullsized":"2024-03-30T01:43:26.287607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:18:32.196925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"63'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320535+0000","last_change":"2024-03-30T01:43:24.839401+0000","last_active":"2024-03-30T01:43:26.320535+0000","last_peered":"2024-03-30T01:43:26.320535+0000","last_clean":"2024-03-30T01:43:26.320535+0000","last_became_active":"2024-03-30T01:43:24.836388+0000","last_became_peered":"2024-03-30T01:43:24.836388+0000","last_unstale":"2024-03-30T01:43:26.320535+0000","last_undegraded":"2024-03-30T01:43:26.320535+0000","last_fullsized":"2024-03-30T01:43:26.320535+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:21.886375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.285003+0000","last_change":"2024-03-30T01:43:12.272623+0000","last_active":"2024-03-30T01:43:25.285003+0000","last_peered":"2024-03-30T01:43:25.285003+0000","last_clean":"2024-03-30T01:43:25.285003+0000","last_became_active":"2024-03-30T01:43:12.272330+0000","last_became_peered":"2024-03-30T01:43:12.272330+0000","last_unstale":"2024-03-30T01:43:25.285003+0000","last_undegraded":"2024-03-30T01:43:25.285003+0000","last_fullsized":"2024-03-30T01:43:25.285003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:11.229993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292039+0000","last_change":"2024-03-30T01:43:11.802228+0000","last_active":"2024-03-30T01:43:25.292039+0000","last_peered":"2024-03-30T01:43:25.292039+0000","last_clean":"2024-03-30T01:43:25.292039+0000","last_became_active":"2024-03-30T01:43:11.801742+0000","last_became_peered":"2024-03-30T01:43:11.801742+0000","last_unstale":"2024-03-30T01:43:25.292039+0000","last_undegraded":"2024-03-30T01:43:25.292039+0000","last_fullsized":"2024-03-30T01:43:25.292039+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:01.225672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320527+0000","last_change":"2024-03-30T01:43:25.314830+0000","last_active":"2024-03-30T01:43:26.320527+0000","last_peered":"2024-03-30T01:43:26.320527+0000","last_clean":"2024-03-30T01:43:26.320527+0000","last_became_active":"2024-03-30T01:43:25.314175+0000","last_became_peered":"2024-03-30T01:43:25.314175+0000","last_unstale":"2024-03-30T01:43:26.320527+0000","last_undegraded":"2024-03-30T01:43:26.320527+0000","last_fullsized":"2024-03-30T01:43:26.320527+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:27:31.532134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320641+0000","last_change":"2024-03-30T01:43:24.837088+0000","last_active":"2024-03-30T01:43:26.320641+0000","last_peered":"2024-03-30T01:43:26.320641+0000","last_clean":"2024-03-30T01:43:26.320641+0000","last_became_active":"2024-03-30T01:43:24.836800+0000","last_became_peered":"2024-03-30T01:43:24.836800+0000","last_unstale":"2024-03-30T01:43:26.320641+0000","last_undegraded":"2024-03-30T01:43:26.320641+0000","last_fullsized":"2024-03-30T01:43:26.320641+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:20:59.364151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.302794+0000","last_change":"2024-03-30T01:43:12.259920+0000","last_active":"2024-03-30T01:43:25.302794+0000","last_peered":"2024-03-30T01:43:25.302794+0000","last_clean":"2024-03-30T01:43:25.302794+0000","last_became_active":"2024-03-30T01:43:12.259636+0000","last_became_peered":"2024-03-30T01:43:12.259636+0000","last_unstale":"2024-03-30T01:43:25.302794+0000","last_undegraded":"2024-03-30T01:43:25.302794+0000","last_fullsized":"2024-03-30T01:43:25.302794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:45:42.926912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.302487+0000","last_change":"2024-03-30T01:43:11.804854+0000","last_active":"2024-03-30T01:43:25.302487+0000","last_peered":"2024-03-30T01:43:25.302487+0000","last_clean":"2024-03-30T01:43:25.302487+0000","last_became_active":"2024-03-30T01:43:11.804593+0000","last_became_peered":"2024-03-30T01:43:11.804593+0000","last_unstale":"2024-03-30T01:43:25.302487+0000","last_undegraded":"2024-03-30T01:43:25.302487+0000","last_fullsized":"2024-03-30T01:43:25.302487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:45:22.282372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.324682+0000","last_change":"2024-03-30T01:43:25.306544+0000","last_active":"2024-03-30T01:43:26.324682+0000","last_peered":"2024-03-30T01:43:26.324682+0000","last_clean":"2024-03-30T01:43:26.324682+0000","last_became_active":"2024-03-30T01:43:25.306276+0000","last_became_peered":"2024-03-30T01:43:25.306276+0000","last_unstale":"2024-03-30T01:43:26.324682+0000","last_undegraded":"2024-03-30T01:43:26.324682+0000","last_fullsized":"2024-03-30T01:43:26.324682+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:16.547277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319071+0000","last_change":"2024-03-30T01:43:24.836475+0000","last_active":"2024-03-30T01:43:26.319071+0000","last_peered":"2024-03-30T01:43:26.319071+0000","last_clean":"2024-03-30T01:43:26.319071+0000","last_became_active":"2024-03-30T01:43:24.835907+0000","last_became_peered":"2024-03-30T01:43:24.835907+0000","last_unstale":"2024-03-30T01:43:26.319071+0000","last_undegraded":"2024-03-30T01:43:26.319071+0000","last_fullsized":"2024-03-30T01:43:26.319071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:25:39.245173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.294141+0000","last_change":"2024-03-30T01:43:12.269648+0000","last_active":"2024-03-30T01:43:25.294141+0000","last_peered":"2024-03-30T01:43:25.294141+0000","last_clean":"2024-03-30T01:43:25.294141+0000","last_became_active":"2024-03-30T01:43:12.269295+0000","last_became_peered":"2024-03-30T01:43:12.269295+0000","last_unstale":"2024-03-30T01:43:25.294141+0000","last_undegraded":"2024-03-30T01:43:25.294141+0000","last_fullsized":"2024-03-30T01:43:25.294141+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:22.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292217+0000","last_change":"2024-03-30T01:43:11.811090+0000","last_active":"2024-03-30T01:43:25.292217+0000","last_peered":"2024-03-30T01:43:25.292217+0000","last_clean":"2024-03-30T01:43:25.292217+0000","last_became_active":"2024-03-30T01:43:11.810767+0000","last_became_peered":"2024-03-30T01:43:11.810767+0000","last_unstale":"2024-03-30T01:43:25.292217+0000","last_undegraded":"2024-03-30T01:43:25.292217+0000","last_fullsized":"2024-03-30T01:43:25.292217+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319928+0000","last_change":"2024-03-30T01:43:25.310519+0000","last_active":"2024-03-30T01:43:26.319928+0000","last_peered":"2024-03-30T01:43:26.319928+0000","last_clean":"2024-03-30T01:43:26.319928+0000","last_became_active":"2024-03-30T01:43:25.310208+0000","last_became_peered":"2024-03-30T01:43:25.310208+0000","last_unstale":"2024-03-30T01:43:26.319928+0000","last_undegraded":"2024-03-30T01:43:26.319928+0000","last_fullsized":"2024-03-30T01:43:26.319928+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:21.765731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"63'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319840+0000","last_change":"2024-03-30T01:43:24.835431+0000","last_active":"2024-03-30T01:43:26.319840+0000","last_peered":"2024-03-30T01:43:26.319840+0000","last_clean":"2024-03-30T01:43:26.319840+0000","last_became_active":"2024-03-30T01:43:24.835126+0000","last_became_peered":"2024-03-30T01:43:24.835126+0000","last_unstale":"2024-03-30T01:43:26.319840+0000","last_undegraded":"2024-03-30T01:43:26.319840+0000","last_fullsized":"2024-03-30T01:43:26.319840+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:40:15.609628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.283048+0000","last_change":"2024-03-30T01:43:12.265208+0000","last_active":"2024-03-30T01:43:25.283048+0000","last_peered":"2024-03-30T01:43:25.283048+0000","last_clean":"2024-03-30T01:43:25.283048+0000","last_became_active":"2024-03-30T01:43:12.264919+0000","last_became_peered":"2024-03-30T01:43:12.264919+0000","last_unstale":"2024-03-30T01:43:25.283048+0000","last_undegraded":"2024-03-30T01:43:25.283048+0000","last_fullsized":"2024-03-30T01:43:25.283048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:31.918755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.302615+0000","last_change":"2024-03-30T01:43:11.803507+0000","last_active":"2024-03-30T01:43:25.302615+0000","last_peered":"2024-03-30T01:43:25.302615+0000","last_clean":"2024-03-30T01:43:25.302615+0000","last_became_active":"2024-03-30T01:43:11.803197+0000","last_became_peered":"2024-03-30T01:43:11.803197+0000","last_unstale":"2024-03-30T01:43:25.302615+0000","last_undegraded":"2024-03-30T01:43:25.302615+0000","last_fullsized":"2024-03-30T01:43:25.302615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:41:18.984681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320081+0000","last_change":"2024-03-30T01:43:25.307138+0000","last_active":"2024-03-30T01:43:26.320081+0000","last_peered":"2024-03-30T01:43:26.320081+0000","last_clean":"2024-03-30T01:43:26.320081+0000","last_became_active":"2024-03-30T01:43:25.306821+0000","last_became_peered":"2024-03-30T01:43:25.306821+0000","last_unstale":"2024-03-30T01:43:26.320081+0000","last_undegraded":"2024-03-30T01:43:26.320081+0000","last_fullsized":"2024-03-30T01:43:26.320081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:33:51.006865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321838+0000","last_change":"2024-03-30T01:43:24.834527+0000","last_active":"2024-03-30T01:43:26.321838+0000","last_peered":"2024-03-30T01:43:26.321838+0000","last_clean":"2024-03-30T01:43:26.321838+0000","last_became_active":"2024-03-30T01:43:24.833944+0000","last_became_peered":"2024-03-30T01:43:24.833944+0000","last_unstale":"2024-03-30T01:43:26.321838+0000","last_undegraded":"2024-03-30T01:43:26.321838+0000","last_fullsized":"2024-03-30T01:43:26.321838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:16:11.596453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.306087+0000","last_change":"2024-03-30T01:43:12.275594+0000","last_active":"2024-03-30T01:43:25.306087+0000","last_peered":"2024-03-30T01:43:25.306087+0000","last_clean":"2024-03-30T01:43:25.306087+0000","last_became_active":"2024-03-30T01:43:12.275249+0000","last_became_peered":"2024-03-30T01:43:12.275249+0000","last_unstale":"2024-03-30T01:43:25.306087+0000","last_undegraded":"2024-03-30T01:43:25.306087+0000","last_fullsized":"2024-03-30T01:43:25.306087+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:09.869640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.294591+0000","last_change":"2024-03-30T01:43:11.810058+0000","last_active":"2024-03-30T01:43:25.294591+0000","last_peered":"2024-03-30T01:43:25.294591+0000","last_clean":"2024-03-30T01:43:25.294591+0000","last_became_active":"2024-03-30T01:43:11.808406+0000","last_became_peered":"2024-03-30T01:43:11.808406+0000","last_unstale":"2024-03-30T01:43:25.294591+0000","last_undegraded":"2024-03-30T01:43:25.294591+0000","last_fullsized":"2024-03-30T01:43:25.294591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:57.117126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319511+0000","last_change":"2024-03-30T01:43:25.315370+0000","last_active":"2024-03-30T01:43:26.319511+0000","last_peered":"2024-03-30T01:43:26.319511+0000","last_clean":"2024-03-30T01:43:26.319511+0000","last_became_active":"2024-03-30T01:43:25.314887+0000","last_became_peered":"2024-03-30T01:43:25.314887+0000","last_unstale":"2024-03-30T01:43:26.319511+0000","last_undegraded":"2024-03-30T01:43:26.319511+0000","last_fullsized":"2024-03-30T01:43:26.319511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:38:46.199422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321324+0000","last_change":"2024-03-30T01:43:24.837124+0000","last_active":"2024-03-30T01:43:26.321324+0000","last_peered":"2024-03-30T01:43:26.321324+0000","last_clean":"2024-03-30T01:43:26.321324+0000","last_became_active":"2024-03-30T01:43:24.836850+0000","last_became_peered":"2024-03-30T01:43:24.836850+0000","last_unstale":"2024-03-30T01:43:26.321324+0000","last_undegraded":"2024-03-30T01:43:26.321324+0000","last_fullsized":"2024-03-30T01:43:26.321324+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:19:54.632173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.288696+0000","last_change":"2024-03-30T01:43:12.259473+0000","last_active":"2024-03-30T01:43:25.288696+0000","last_peered":"2024-03-30T01:43:25.288696+0000","last_clean":"2024-03-30T01:43:25.288696+0000","last_became_active":"2024-03-30T01:43:12.259194+0000","last_became_peered":"2024-03-30T01:43:12.259194+0000","last_unstale":"2024-03-30T01:43:25.288696+0000","last_undegraded":"2024-03-30T01:43:25.288696+0000","last_fullsized":"2024-03-30T01:43:25.288696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:14.284517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"60'1","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.293095+0000","last_change":"2024-03-30T01:43:11.810112+0000","last_active":"2024-03-30T01:43:25.293095+0000","last_peered":"2024-03-30T01:43:25.293095+0000","last_clean":"2024-03-30T01:43:25.293095+0000","last_became_active":"2024-03-30T01:43:11.808349+0000","last_became_peered":"2024-03-30T01:43:11.808349+0000","last_unstale":"2024-03-30T01:43:25.293095+0000","last_undegraded":"2024-03-30T01:43:25.293095+0000","last_fullsized":"2024-03-30T01:43:25.293095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:57:30.914337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.319786+0000","last_change":"2024-03-30T01:43:25.299230+0000","last_active":"2024-03-30T01:43:26.319786+0000","last_peered":"2024-03-30T01:43:26.319786+0000","last_clean":"2024-03-30T01:43:26.319786+0000","last_became_active":"2024-03-30T01:43:25.298933+0000","last_became_peered":"2024-03-30T01:43:25.298933+0000","last_unstale":"2024-03-30T01:43:26.319786+0000","last_undegraded":"2024-03-30T01:43:26.319786+0000","last_fullsized":"2024-03-30T01:43:26.319786+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:21:00.374174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"63'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288153+0000","last_change":"2024-03-30T01:43:24.835180+0000","last_active":"2024-03-30T01:43:26.288153+0000","last_peered":"2024-03-30T01:43:26.288153+0000","last_clean":"2024-03-30T01:43:26.288153+0000","last_became_active":"2024-03-30T01:43:24.834898+0000","last_became_peered":"2024-03-30T01:43:24.834898+0000","last_unstale":"2024-03-30T01:43:26.288153+0000","last_undegraded":"2024-03-30T01:43:26.288153+0000","last_fullsized":"2024-03-30T01:43:26.288153+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:21.886375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.292981+0000","last_change":"2024-03-30T01:43:12.265689+0000","last_active":"2024-03-30T01:43:25.292981+0000","last_peered":"2024-03-30T01:43:25.292981+0000","last_clean":"2024-03-30T01:43:25.292981+0000","last_became_active":"2024-03-30T01:43:12.265382+0000","last_became_peered":"2024-03-30T01:43:12.265382+0000","last_unstale":"2024-03-30T01:43:25.292981+0000","last_undegraded":"2024-03-30T01:43:25.292981+0000","last_fullsized":"2024-03-30T01:43:25.292981+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:17:17.176008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.302719+0000","last_change":"2024-03-30T01:43:11.811154+0000","last_active":"2024-03-30T01:43:25.302719+0000","last_peered":"2024-03-30T01:43:25.302719+0000","last_clean":"2024-03-30T01:43:25.302719+0000","last_became_active":"2024-03-30T01:43:11.810455+0000","last_became_peered":"2024-03-30T01:43:11.810455+0000","last_unstale":"2024-03-30T01:43:25.302719+0000","last_undegraded":"2024-03-30T01:43:25.302719+0000","last_fullsized":"2024-03-30T01:43:25.302719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:07:33.020166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.322983+0000","last_change":"2024-03-30T01:43:25.298017+0000","last_active":"2024-03-30T01:43:26.322983+0000","last_peered":"2024-03-30T01:43:26.322983+0000","last_clean":"2024-03-30T01:43:26.322983+0000","last_became_active":"2024-03-30T01:43:25.296759+0000","last_became_peered":"2024-03-30T01:43:25.296759+0000","last_unstale":"2024-03-30T01:43:26.322983+0000","last_undegraded":"2024-03-30T01:43:26.322983+0000","last_fullsized":"2024-03-30T01:43:26.322983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:34:51.783505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320404+0000","last_change":"2024-03-30T01:43:24.835280+0000","last_active":"2024-03-30T01:43:26.320404+0000","last_peered":"2024-03-30T01:43:26.320404+0000","last_clean":"2024-03-30T01:43:26.320404+0000","last_became_active":"2024-03-30T01:43:24.835031+0000","last_became_peered":"2024-03-30T01:43:24.835031+0000","last_unstale":"2024-03-30T01:43:26.320404+0000","last_undegraded":"2024-03-30T01:43:26.320404+0000","last_fullsized":"2024-03-30T01:43:26.320404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:40:53.401868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.284573+0000","last_change":"2024-03-30T01:43:12.265758+0000","last_active":"2024-03-30T01:43:25.284573+0000","last_peered":"2024-03-30T01:43:25.284573+0000","last_clean":"2024-03-30T01:43:25.284573+0000","last_became_active":"2024-03-30T01:43:12.265461+0000","last_became_peered":"2024-03-30T01:43:12.265461+0000","last_unstale":"2024-03-30T01:43:25.284573+0000","last_undegraded":"2024-03-30T01:43:25.284573+0000","last_fullsized":"2024-03-30T01:43:25.284573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:01:36.552215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289132+0000","last_change":"2024-03-30T01:43:11.806860+0000","last_active":"2024-03-30T01:43:25.289132+0000","last_peered":"2024-03-30T01:43:25.289132+0000","last_clean":"2024-03-30T01:43:25.289132+0000","last_became_active":"2024-03-30T01:43:11.806563+0000","last_became_peered":"2024-03-30T01:43:11.806563+0000","last_unstale":"2024-03-30T01:43:25.289132+0000","last_undegraded":"2024-03-30T01:43:25.289132+0000","last_fullsized":"2024-03-30T01:43:25.289132+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287125+0000","last_change":"2024-03-30T01:43:25.320759+0000","last_active":"2024-03-30T01:43:26.287125+0000","last_peered":"2024-03-30T01:43:26.287125+0000","last_clean":"2024-03-30T01:43:26.287125+0000","last_became_active":"2024-03-30T01:43:25.320540+0000","last_became_peered":"2024-03-30T01:43:25.320540+0000","last_unstale":"2024-03-30T01:43:26.287125+0000","last_undegraded":"2024-03-30T01:43:26.287125+0000","last_fullsized":"2024-03-30T01:43:26.287125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:22.896399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.323356+0000","last_change":"2024-03-30T01:43:24.837113+0000","last_active":"2024-03-30T01:43:26.323356+0000","last_peered":"2024-03-30T01:43:26.323356+0000","last_clean":"2024-03-30T01:43:26.323356+0000","last_became_active":"2024-03-30T01:43:24.836842+0000","last_became_peered":"2024-03-30T01:43:24.836842+0000","last_unstale":"2024-03-30T01:43:26.323356+0000","last_undegraded":"2024-03-30T01:43:26.323356+0000","last_fullsized":"2024-03-30T01:43:26.323356+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:57.220669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.282824+0000","last_change":"2024-03-30T01:43:12.269888+0000","last_active":"2024-03-30T01:43:25.282824+0000","last_peered":"2024-03-30T01:43:25.282824+0000","last_clean":"2024-03-30T01:43:25.282824+0000","last_became_active":"2024-03-30T01:43:12.269582+0000","last_became_peered":"2024-03-30T01:43:12.269582+0000","last_unstale":"2024-03-30T01:43:25.282824+0000","last_undegraded":"2024-03-30T01:43:25.282824+0000","last_fullsized":"2024-03-30T01:43:25.282824+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:58.121544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.295063+0000","last_change":"2024-03-30T01:43:11.802541+0000","last_active":"2024-03-30T01:43:25.295063+0000","last_peered":"2024-03-30T01:43:25.295063+0000","last_clean":"2024-03-30T01:43:25.295063+0000","last_became_active":"2024-03-30T01:43:11.802154+0000","last_became_peered":"2024-03-30T01:43:11.802154+0000","last_unstale":"2024-03-30T01:43:25.295063+0000","last_undegraded":"2024-03-30T01:43:25.295063+0000","last_fullsized":"2024-03-30T01:43:25.295063+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:01:10.225574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.287768+0000","last_change":"2024-03-30T01:43:25.301511+0000","last_active":"2024-03-30T01:43:26.287768+0000","last_peered":"2024-03-30T01:43:26.287768+0000","last_clean":"2024-03-30T01:43:26.287768+0000","last_became_active":"2024-03-30T01:43:25.301249+0000","last_became_peered":"2024-03-30T01:43:25.301249+0000","last_unstale":"2024-03-30T01:43:26.287768+0000","last_undegraded":"2024-03-30T01:43:26.287768+0000","last_fullsized":"2024-03-30T01:43:26.287768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:05.193192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.286326+0000","last_change":"2024-03-30T01:43:24.835705+0000","last_active":"2024-03-30T01:43:26.286326+0000","last_peered":"2024-03-30T01:43:26.286326+0000","last_clean":"2024-03-30T01:43:26.286326+0000","last_became_active":"2024-03-30T01:43:24.835405+0000","last_became_peered":"2024-03-30T01:43:24.835405+0000","last_unstale":"2024-03-30T01:43:26.286326+0000","last_undegraded":"2024-03-30T01:43:26.286326+0000","last_fullsized":"2024-03-30T01:43:26.286326+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:08:36.572938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289748+0000","last_change":"2024-03-30T01:43:12.279403+0000","last_active":"2024-03-30T01:43:25.289748+0000","last_peered":"2024-03-30T01:43:25.289748+0000","last_clean":"2024-03-30T01:43:25.289748+0000","last_became_active":"2024-03-30T01:43:12.279071+0000","last_became_peered":"2024-03-30T01:43:12.279071+0000","last_unstale":"2024-03-30T01:43:25.289748+0000","last_undegraded":"2024-03-30T01:43:25.289748+0000","last_fullsized":"2024-03-30T01:43:25.289748+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:37:02.230090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.291487+0000","last_change":"2024-03-30T01:43:11.799428+0000","last_active":"2024-03-30T01:43:25.291487+0000","last_peered":"2024-03-30T01:43:25.291487+0000","last_clean":"2024-03-30T01:43:25.291487+0000","last_became_active":"2024-03-30T01:43:11.799147+0000","last_became_peered":"2024-03-30T01:43:11.799147+0000","last_unstale":"2024-03-30T01:43:25.291487+0000","last_undegraded":"2024-03-30T01:43:25.291487+0000","last_fullsized":"2024-03-30T01:43:25.291487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:11:56.396431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.323427+0000","last_change":"2024-03-30T01:43:25.296849+0000","last_active":"2024-03-30T01:43:26.323427+0000","last_peered":"2024-03-30T01:43:26.323427+0000","last_clean":"2024-03-30T01:43:26.323427+0000","last_became_active":"2024-03-30T01:43:25.296539+0000","last_became_peered":"2024-03-30T01:43:25.296539+0000","last_unstale":"2024-03-30T01:43:26.323427+0000","last_undegraded":"2024-03-30T01:43:26.323427+0000","last_fullsized":"2024-03-30T01:43:26.323427+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:14:35.301316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.323422+0000","last_change":"2024-03-30T01:43:24.833338+0000","last_active":"2024-03-30T01:43:26.323422+0000","last_peered":"2024-03-30T01:43:26.323422+0000","last_clean":"2024-03-30T01:43:26.323422+0000","last_became_active":"2024-03-30T01:43:24.833003+0000","last_became_peered":"2024-03-30T01:43:24.833003+0000","last_unstale":"2024-03-30T01:43:26.323422+0000","last_undegraded":"2024-03-30T01:43:26.323422+0000","last_fullsized":"2024-03-30T01:43:26.323422+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:09:47.414695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.288540+0000","last_change":"2024-03-30T01:43:12.273079+0000","last_active":"2024-03-30T01:43:25.288540+0000","last_peered":"2024-03-30T01:43:25.288540+0000","last_clean":"2024-03-30T01:43:25.288540+0000","last_became_active":"2024-03-30T01:43:12.272766+0000","last_became_peered":"2024-03-30T01:43:12.272766+0000","last_unstale":"2024-03-30T01:43:25.288540+0000","last_undegraded":"2024-03-30T01:43:25.288540+0000","last_fullsized":"2024-03-30T01:43:25.288540+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:57.882115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.290792+0000","last_change":"2024-03-30T01:43:11.805262+0000","last_active":"2024-03-30T01:43:25.290792+0000","last_peered":"2024-03-30T01:43:25.290792+0000","last_clean":"2024-03-30T01:43:25.290792+0000","last_became_active":"2024-03-30T01:43:11.804622+0000","last_became_peered":"2024-03-30T01:43:11.804622+0000","last_unstale":"2024-03-30T01:43:25.290792+0000","last_undegraded":"2024-03-30T01:43:25.290792+0000","last_fullsized":"2024-03-30T01:43:25.290792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:37:08.137270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.324110+0000","last_change":"2024-03-30T01:43:25.291440+0000","last_active":"2024-03-30T01:43:26.324110+0000","last_peered":"2024-03-30T01:43:26.324110+0000","last_clean":"2024-03-30T01:43:26.324110+0000","last_became_active":"2024-03-30T01:43:25.291124+0000","last_became_peered":"2024-03-30T01:43:25.291124+0000","last_unstale":"2024-03-30T01:43:26.324110+0000","last_undegraded":"2024-03-30T01:43:26.324110+0000","last_fullsized":"2024-03-30T01:43:26.324110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:45:55.953671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.288025+0000","last_change":"2024-03-30T01:43:24.841103+0000","last_active":"2024-03-30T01:43:26.288025+0000","last_peered":"2024-03-30T01:43:26.288025+0000","last_clean":"2024-03-30T01:43:26.288025+0000","last_became_active":"2024-03-30T01:43:24.840742+0000","last_became_peered":"2024-03-30T01:43:24.840742+0000","last_unstale":"2024-03-30T01:43:26.288025+0000","last_undegraded":"2024-03-30T01:43:26.288025+0000","last_fullsized":"2024-03-30T01:43:26.288025+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:13.587750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.288655+0000","last_change":"2024-03-30T01:43:12.275816+0000","last_active":"2024-03-30T01:43:25.288655+0000","last_peered":"2024-03-30T01:43:25.288655+0000","last_clean":"2024-03-30T01:43:25.288655+0000","last_became_active":"2024-03-30T01:43:12.275547+0000","last_became_peered":"2024-03-30T01:43:12.275547+0000","last_unstale":"2024-03-30T01:43:25.288655+0000","last_undegraded":"2024-03-30T01:43:25.288655+0000","last_fullsized":"2024-03-30T01:43:25.288655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:15:31.125780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.294396+0000","last_change":"2024-03-30T01:43:11.804870+0000","last_active":"2024-03-30T01:43:25.294396+0000","last_peered":"2024-03-30T01:43:25.294396+0000","last_clean":"2024-03-30T01:43:25.294396+0000","last_became_active":"2024-03-30T01:43:11.804580+0000","last_became_peered":"2024-03-30T01:43:11.804580+0000","last_unstale":"2024-03-30T01:43:25.294396+0000","last_undegraded":"2024-03-30T01:43:25.294396+0000","last_fullsized":"2024-03-30T01:43:25.294396+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:39:23.036428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.321725+0000","last_change":"2024-03-30T01:43:25.301677+0000","last_active":"2024-03-30T01:43:26.321725+0000","last_peered":"2024-03-30T01:43:26.321725+0000","last_clean":"2024-03-30T01:43:26.321725+0000","last_became_active":"2024-03-30T01:43:25.301374+0000","last_became_peered":"2024-03-30T01:43:25.301374+0000","last_unstale":"2024-03-30T01:43:26.321725+0000","last_undegraded":"2024-03-30T01:43:26.321725+0000","last_fullsized":"2024-03-30T01:43:26.321725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:24.807845+0000","last_clean_scrub_stamp":"2024-03-30T01:43:24.807845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:20:08.899825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-03-30T01:43:26.320371+0000","last_change":"2024-03-30T01:43:24.836075+0000","last_active":"2024-03-30T01:43:26.320371+0000","last_peered":"2024-03-30T01:43:26.320371+0000","last_clean":"2024-03-30T01:43:26.320371+0000","last_became_active":"2024-03-30T01:43:24.835774+0000","last_became_peered":"2024-03-30T01:43:24.835774+0000","last_unstale":"2024-03-30T01:43:26.320371+0000","last_undegraded":"2024-03-30T01:43:26.320371+0000","last_fullsized":"2024-03-30T01:43:26.320371+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:23.797822+0000","last_clean_scrub_stamp":"2024-03-30T01:43:23.797822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:18:03.503668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.283581+0000","last_change":"2024-03-30T01:43:12.270228+0000","last_active":"2024-03-30T01:43:25.283581+0000","last_peered":"2024-03-30T01:43:25.283581+0000","last_clean":"2024-03-30T01:43:25.283581+0000","last_became_active":"2024-03-30T01:43:12.269910+0000","last_became_peered":"2024-03-30T01:43:12.269910+0000","last_unstale":"2024-03-30T01:43:25.283581+0000","last_undegraded":"2024-03-30T01:43:25.283581+0000","last_fullsized":"2024-03-30T01:43:25.283581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:11.781087+0000","last_clean_scrub_stamp":"2024-03-30T01:43:11.781087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:00:52.920381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.289727+0000","last_change":"2024-03-30T01:43:11.803125+0000","last_active":"2024-03-30T01:43:25.289727+0000","last_peered":"2024-03-30T01:43:25.289727+0000","last_clean":"2024-03-30T01:43:25.289727+0000","last_became_active":"2024-03-30T01:43:11.802822+0000","last_became_peered":"2024-03-30T01:43:11.802822+0000","last_unstale":"2024-03-30T01:43:25.289727+0000","last_undegraded":"2024-03-30T01:43:25.289727+0000","last_fullsized":"2024-03-30T01:43:25.289727+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:43:10.776668+0000","last_clean_scrub_stamp":"2024-03-30T01:43:10.776668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:55:21.501027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":201,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-30T01:43:25.295177+0000","last_change":"2024-03-30T01:40:48.918411+0000","last_active":"2024-03-30T01:43:25.295177+0000","last_peered":"2024-03-30T01:43:25.295177+0000","last_clean":"2024-03-30T01:43:25.295177+0000","last_became_active":"2024-03-30T01:40:48.917259+0000","last_became_peered":"2024-03-30T01:40:48.917259+0000","last_unstale":"2024-03-30T01:43:25.295177+0000","last_undegraded":"2024-03-30T01:43:25.295177+0000","last_fullsized":"2024-03-30T01:43:25.295177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T01:40:47.904665+0000","last_clean_scrub_stamp":"2024-03-30T01:40:47.904665+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:17:35.031513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28388,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343452,"statfs":{"total":96636764160,"available":96607694848,"internally_reserved":0,"allocated":1388544,"data_stored":954075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.78000000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59099999999999997}]},{"osd":1,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72699999999999998,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.81200000000000006,"15min":0.85599999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72699999999999998,"5min":6.9610000000000003,"15min":6.9610000000000003},"last":0.441}]},{"osd":2,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.70799999999999996,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.76200000000000001,"15min":0.79500000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76800000000000002,"5min":4.2009999999999996,"15min":4.2009999999999996},"last":0.46899999999999997}]},{"osd":3,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73199999999999998,"5min":1.264,"15min":1.264},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53200000000000003}]},{"osd":4,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79300000000000004,"5min":1.075,"15min":1.075},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77200000000000002,"5min":1.833,"15min":1.833},"last":0.41699999999999998}]},{"osd":5,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76300000000000001,"5min":1.238,"15min":1.238},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90600000000000003,"5min":1.2,"15min":1.2},"last":0.626}]},{"osd":6,"last update":"Sat Mar 30 01:42:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85099999999999998,"5min":1.288,"15min":1.288},"last":0.502},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.68300000000000005,"15min":0.69799999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82499999999999996,"5min":1.901,"15min":1.901},"last":0.56599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":364507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.94899999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70799999999999996,"5min":1.361,"15min":1.361},"last":0.54800000000000004}]},{"osd":1,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.34300000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96799999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.71699999999999997,"15min":0.74399999999999999},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":4.3250000000000002,"15min":4.3250000000000002},"last":0.505}]},{"osd":2,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85599999999999998,"5min":1.27,"15min":1.27},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":1.409,"15min":1.409},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.379,"5min":0.371,"15min":0.371},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.46700000000000003}]},{"osd":4,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82199999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61199999999999999}]},{"osd":5,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59699999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.63400000000000001},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.627,"15min":0.624},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":366860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.432,"15min":0.442},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.503,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":1.111,"15min":1.2270000000000001},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":15.038,"15min":15.038},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.91400000000000003,"15min":1.006},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55500000000000005,"5min":11.858000000000001,"15min":11.858000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68400000000000005,"5min":1.444,"15min":1.444},"last":0.44900000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72799999999999998,"5min":1.474,"15min":1.474},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.624,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64000000000000001,"5min":4.1289999999999996,"15min":4.1289999999999996},"last":0.629}]},{"osd":4,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.111,"15min":1.1970000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94499999999999995,"5min":11.99,"15min":11.99},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89900000000000002,"5min":1.407,"15min":1.407},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":1.335,"15min":1.4490000000000001},"min":{"1min":0.48799999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97699999999999998,"5min":15.006,"15min":15.006},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":1.2130000000000001,"15min":1.3069999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83199999999999996,"5min":11.926,"15min":11.926},"last":0.61799999999999999}]},{"osd":6,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.3149999999999999,"15min":1.429},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93100000000000005,"5min":14.974,"15min":14.974},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.1950000000000001,"15min":1.2949999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78100000000000003,"5min":11.960000000000001,"15min":11.960000000000001},"last":0.54600000000000004}]},{"osd":7,"last update":"Sat Mar 30 01:42:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.504,"15min":1.504},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":1.343,"15min":1.4570000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":14.923999999999999,"15min":14.923999999999999},"last":0.70399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28364,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343476,"statfs":{"total":96636764160,"available":96607719424,"internally_reserved":0,"allocated":1363968,"data_stored":952308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.57999999999999996,"15min":0.61499999999999999},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.53500000000000003,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.42199999999999999,"15min":0.432},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.439,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.54500000000000004,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.316},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.95499999999999996,"15min":1.05},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.443,"5min":14.946999999999999,"15min":14.946999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.86299999999999999,"15min":0.94299999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.503,"5min":12.983000000000001,"15min":12.983000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.59799999999999998,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.53300000000000003,"5min":4.2279999999999998,"15min":4.2279999999999998},"last":0.39300000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.84699999999999998,"15min":0.875},"min":{"1min":0.47099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":5.431,"15min":5.431},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.48899999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88300000000000001,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.50600000000000001}]},{"osd":5,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.80100000000000005,"15min":0.82899999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77700000000000002,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.83799999999999997,"15min":0.87},"min":{"1min":0.47099999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":5.5060000000000002,"15min":5.5060000000000002},"last":0.51600000000000001}]},{"osd":6,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.69099999999999995,"15min":0.70099999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76500000000000001,"5min":1.177,"15min":1.177},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.82399999999999995,"15min":0.85399999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":4.3869999999999996,"15min":4.3869999999999996},"last":0.60099999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.81899999999999995,"15min":0.84799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93899999999999995,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.88800000000000001,"15min":0.92700000000000005},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.81899999999999995,"5min":5.468,"15min":5.468},"last":0.47499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":363651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.89000000000000001,"15min":0.96199999999999997},"min":{"1min":0.27200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.1890000000000001,"5min":12.478999999999999,"15min":12.478999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.753,"15min":0.79600000000000004},"min":{"1min":0.308,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.1469999999999998,"5min":8.6229999999999993,"15min":8.6229999999999993},"last":0.40999999999999998}]},{"osd":1,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53900000000000003,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.93899999999999995,"15min":1.0089999999999999},"min":{"1min":0.28999999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.1309999999999998,"5min":12.425000000000001,"15min":12.425000000000001},"last":0.379}]},{"osd":3,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1459999999999999,"5min":1.55,"15min":1.55},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.98299999999999998,"15min":1.0620000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.8759999999999999,"5min":13.217000000000001,"15min":13.217000000000001},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.93899999999999995,"15min":0.97999999999999998},"min":{"1min":0.46400000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.956,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.95499999999999996,"15min":0.997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.0630000000000002,"5min":8.5640000000000001,"15min":8.5640000000000001},"last":0.749}]},{"osd":5,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.45100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89300000000000002,"5min":1.462,"15min":1.462},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.97499999999999998,"15min":1.0229999999999999},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.9159999999999999,"5min":8.7759999999999998,"15min":8.7759999999999998},"last":0.623}]},{"osd":6,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.97899999999999998,"15min":1.0229999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.1139999999999999,"5min":8.5190000000000001,"15min":8.5190000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Mar 30 01:42:48 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.72099999999999997,"15min":0.73699999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.524,"15min":1.524},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.0660000000000001,"15min":1.105},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":3.105,"5min":8.5939999999999994,"15min":8.5939999999999994},"last":0.78900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":363491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.38,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.48599999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.47699999999999998,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.40899999999999997}]},{"osd":1,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.372,"15min":0.372},"min":{"1min":0.27200000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.312},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.41799999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.39600000000000002}]},{"osd":2,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45000000000000001,"15min":0.45400000000000001},"min":{"1min":0.33800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.53600000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.45000000000000001,"15min":0.45400000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.433}]},{"osd":4,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76200000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91500000000000004,"5min":1.093,"15min":1.093},"last":0.73999999999999999}]},{"osd":5,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.80800000000000005,"5min":1.042,"15min":1.042},"last":0.622},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89200000000000002,"5min":1.012,"15min":1.012},"last":0.67700000000000005}]},{"osd":6,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.68899999999999995,"15min":0.69999999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.78900000000000003,"5min":1.024,"15min":1.024},"last":0.63600000000000001}]},{"osd":7,"last update":"Sat Mar 30 01:42:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.78200000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.878,"5min":1.085,"15min":1.085},"last":0.70199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":367652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.67600000000000005,"15min":0.69499999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.87,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.623}]},{"osd":1,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":1.0289999999999999,"15min":1.113},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":9.3970000000000002,"15min":9.3970000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.747,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.626,"15min":0.63600000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76800000000000002,"5min":1.494,"15min":1.494},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.72299999999999998,"15min":0.752},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.63600000000000001}]},{"osd":3,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.86599999999999999,"15min":0.91000000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":1.1279999999999999,"15min":1.218},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73499999999999999,"5min":9.452,"15min":9.452},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78900000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67300000000000004}]},{"osd":6,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.76400000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.68500000000000005,"15min":0.69899999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82499999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Sat Mar 30 01:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.89200000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.69199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":782336,"data_stored":362965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.83399999999999996,"15min":0.876},"min":{"1min":0.39800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78900000000000003,"5min":6.5919999999999996,"15min":6.5919999999999996},"last":0.503},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.88400000000000001,"15min":0.93200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88200000000000001,"5min":6.6760000000000002,"15min":6.6760000000000002},"last":0.71499999999999997}]},{"osd":1,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.84999999999999998,"15min":0.89800000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81999999999999995,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.89400000000000002,"15min":0.93999999999999995},"min":{"1min":0.436,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94099999999999995,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.83999999999999997,"15min":0.88600000000000001},"min":{"1min":0.373,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80400000000000005,"5min":6.8410000000000002,"15min":6.8410000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77000000000000002,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.67700000000000005}]},{"osd":3,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.88800000000000001,"15min":0.94499999999999995},"min":{"1min":0.38900000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91500000000000004,"5min":6.8970000000000002,"15min":6.8970000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.44900000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86499999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.81100000000000005}]},{"osd":4,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85299999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.73899999999999999}]},{"osd":6,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.34899999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.752},{"interface":"front","average":{"1min":0.624,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89000000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79200000000000004}]},{"osd":7,"last update":"Sat Mar 30 01:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.442,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.627,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T01:43:29.984 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T01:43:29.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mon 'allow r' osd 'allow rw pool=cephfs2_data' mds allow 2024-03-30T01:43:30.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.157+0000 7f8d3d9f0700 1 Processor -- start 2024-03-30T01:43:30.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.157+0000 7f8d3d9f0700 1 -- start start 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d3d9f0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d380c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d3d9f0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8d380721f0 0x7f8d3806a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d3d9f0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d38155000 con 0x7f8d380c07f0 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d3d9f0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3806e930 con 0x7f8d380721f0 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d3d9f0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3806df40 con 0x7f8d38151ea0 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d36ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d380c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d367fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8d380721f0 0x7f8d3806a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d36ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d380c02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.31:36344/0 (socket says 172.21.15.31:36344) 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d36ffd700 1 -- 172.21.15.31:0/161146225 learned_addr learned my addr 172.21.15.31:0/161146225 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.158+0000 7f8d377fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8d380c07f0 0x7f8d3806ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:44598/0 (socket says 172.21.15.31:44598) 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d35ffb700 1 -- 172.21.15.31:0/161146225 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103070138 0 0) 0x7f8d38155000 con 0x7f8d380c07f0 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d35ffb700 1 -- 172.21.15.31:0/161146225 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d240035b0 con 0x7f8d380c07f0 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d367fc700 1 -- 172.21.15.31:0/161146225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 msgr2=0x7f8d380c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d367fc700 1 --2- 172.21.15.31:0/161146225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d380c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d367fc700 1 -- 172.21.15.31:0/161146225 >> v1:172.21.15.59:6789/0 conn(0x7f8d380c07f0 legacy=0x7f8d3806ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d367fc700 1 -- 172.21.15.31:0/161146225 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3806e650 con 0x7f8d380721f0 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d36ffd700 1 --2- 172.21.15.31:0/161146225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d380c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d367fc700 1 --2- 172.21.15.31:0/161146225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 0x7f8d3806a700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009840 tx=0x7f8d28009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dd76fa0be67c2ee server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d35ffb700 1 -- 172.21.15.31:0/161146225 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28011830 con 0x7f8d380721f0 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d35ffb700 1 -- 172.21.15.31:0/161146225 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d28011990 con 0x7f8d380721f0 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.159+0000 7f8d35ffb700 1 -- 172.21.15.31:0/161146225 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28011d20 con 0x7f8d380721f0 2024-03-30T01:43:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.160+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/161146225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 msgr2=0x7f8d3806a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.160+0000 7f8d3d9f0700 1 --2- 172.21.15.31:0/161146225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 0x7f8d3806a700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009840 tx=0x7f8d28009b50 comp rx=0 tx=0).stop 2024-03-30T01:43:30.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.160+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/161146225 shutdown_connections 2024-03-30T01:43:30.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.160+0000 7f8d3d9f0700 1 --2- 172.21.15.31:0/161146225 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d380c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.160+0000 7f8d3d9f0700 1 --2- 172.21.15.31:0/161146225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 0x7f8d3806a700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.160+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/161146225 >> 172.21.15.31:0/161146225 conn(0x7f8d380c3980 msgr2=0x7f8d380c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:30.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.160+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/161146225 shutdown_connections 2024-03-30T01:43:30.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.160+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/161146225 wait complete. 2024-03-30T01:43:30.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.161+0000 7f8d3d9f0700 1 Processor -- start 2024-03-30T01:43:30.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d3d9f0700 1 -- start start 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d3d9f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 0x7f8d3814e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d3d9f0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d380c07f0 0x7f8d380abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d3d9f0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d38152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d3d9f0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38157dd0 con 0x7f8d380c07f0 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d3d9f0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d381e9690 con 0x7f8d380721f0 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d3d9f0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3814ee80 con 0x7f8d38151ea0 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d367fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d380c07f0 0x7f8d380abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d36ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 0x7f8d3814e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.162+0000 7f8d367fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d380c07f0 0x7f8d380abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48756/0 (socket says 172.21.15.31:48756) 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d367fc700 1 -- 172.21.15.31:0/3614566755 learned_addr learned my addr 172.21.15.31:0/3614566755 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d377fe700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d38152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d367fc700 1 -- 172.21.15.31:0/3614566755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 msgr2=0x7f8d38152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d367fc700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d38152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d367fc700 1 -- 172.21.15.31:0/3614566755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 msgr2=0x7f8d3814e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d367fc700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 0x7f8d3814e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d367fc700 1 -- 172.21.15.31:0/3614566755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d20009550 con 0x7f8d380c07f0 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d36ffd700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 0x7f8d3814e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d367fc700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d380c07f0 0x7f8d380abdb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8d280090b0 tx=0x7f8d28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d2ffff700 1 -- 172.21.15.31:0/3614566755 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28004380 con 0x7f8d380c07f0 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d2ffff700 1 -- 172.21.15.31:0/3614566755 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2801f070 con 0x7f8d380c07f0 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d2ffff700 1 -- 172.21.15.31:0/3614566755 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d280036a0 con 0x7f8d380c07f0 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d280094a0 con 0x7f8d380c07f0 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.163+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d381e9bc0 con 0x7f8d380c07f0 2024-03-30T01:43:30.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.164+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d380747d0 con 0x7f8d380c07f0 2024-03-30T01:43:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.165+0000 7f8d2ffff700 1 -- 172.21.15.31:0/3614566755 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d28021030 con 0x7f8d380c07f0 2024-03-30T01:43:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.165+0000 7f8d2ffff700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d1806ac90 0x7f8d1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.165+0000 7f8d2ffff700 1 -- 172.21.15.31:0/3614566755 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f8d2808a860 con 0x7f8d380c07f0 2024-03-30T01:43:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.165+0000 7f8d36ffd700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d1806ac90 0x7f8d1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.166+0000 7f8d36ffd700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d1806ac90 0x7f8d1806d150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d380ad060 tx=0x7f8d200092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.170+0000 7f8d2ffff700 1 -- 172.21.15.31:0/3614566755 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d280556d0 con 0x7f8d380c07f0 2024-03-30T01:43:30.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.471+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]} v 0) v1 -- 0x7f8d380c1520 con 0x7f8d380c07f0 2024-03-30T01:43:30.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.475+0000 7f8d2ffff700 1 -- 172.21.15.31:0/3614566755 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]}]=0 updated caps for client.1 v18) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8d28058cf0 con 0x7f8d380c07f0 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d1806ac90 msgr2=0x7f8d1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d1806ac90 0x7f8d1806d150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d380ad060 tx=0x7f8d200092c0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d380c07f0 msgr2=0x7f8d380abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d380c07f0 0x7f8d380abdb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8d280090b0 tx=0x7f8d28004750 comp rx=0 tx=0).stop 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 shutdown_connections 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f8d1806ac90 0x7f8d1806d150 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d38151ea0 0x7f8d38152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d380721f0 0x7f8d3814e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 --2- 172.21.15.31:0/3614566755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d380c07f0 0x7f8d380abdb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 >> 172.21.15.31:0/3614566755 conn(0x7f8d380c3980 msgr2=0x7f8d380c61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 shutdown_connections 2024-03-30T01:43:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.478+0000 7f8d3d9f0700 1 -- 172.21.15.31:0/3614566755 wait complete. 2024-03-30T01:43:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.1 2024-03-30T01:43:30.498 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:43:30.498 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:43:30.521 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:43:30.521 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:43:30.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T01:43:30.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.718+0000 7fa3c5470700 1 Processor -- start 2024-03-30T01:43:30.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.718+0000 7fa3c5470700 1 -- start start 2024-03-30T01:43:30.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3c5470700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3c0151fb0 0x7fa3c006a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3c5470700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa3c006ac40 0x7fa3c015f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3c5470700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c00c0270 con 0x7fa3c00bf700 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3c5470700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c0152390 con 0x7fa3c006ac40 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3c5470700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c0070830 con 0x7fa3c0151fb0 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3be7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa3c00bf700 0x7fa3c0071360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54848/0 (socket says 172.21.15.31:54848) 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3be7fc700 1 -- 172.21.15.31:0/2679325584 learned_addr learned my addr 172.21.15.31:0/2679325584 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3bf7fe700 1 --2- 172.21.15.31:0/2679325584 >> v2:172.21.15.59:3300/0 conn(0x7fa3c006ac40 0x7fa3c015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3beffd700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3c0151fb0 0x7fa3c006a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445826509 0 0) 0x7fa3c00c0270 con 0x7fa3c00bf700 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.719+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3a80035b0 con 0x7fa3c00bf700 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3792451583 0 0) 0x7fa3a80035b0 con 0x7fa3c00bf700 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3c0151fb0 msgr2=0x7fa3c006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:30.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3c0151fb0 0x7fa3c006a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 >> v2:172.21.15.59:3300/0 conn(0x7fa3c006ac40 msgr2=0x7fa3c015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 --2- 172.21.15.31:0/2679325584 >> v2:172.21.15.59:3300/0 conn(0x7fa3c006ac40 0x7fa3c015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c00c1170 con 0x7fa3c00bf700 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7fa3b4001670 con 0x7fa3c00bf700 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 >> v1:172.21.15.31:6789/0 conn(0x7fa3c00bf700 legacy=0x7fa3c0071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3a8003a00 0x7fa3a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3c0151fb0 0x7fa3a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3a8004d90 0x7fa3a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a800ba10 con 0x7fa3a8003a00 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a800bb50 con 0x7fa3a8004d90 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a800bcc0 con 0x7fa3c0151fb0 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3beffd700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3a8003a00 0x7fa3a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3be7fc700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3c0151fb0 0x7fa3a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3bf7fe700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3a8004d90 0x7fa3a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3beffd700 1 -- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3c0151fb0 msgr2=0x7fa3a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3beffd700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3c0151fb0 0x7fa3a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3beffd700 1 -- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3a8004d90 msgr2=0x7fa3a8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3beffd700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3a8004d90 0x7fa3a8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3beffd700 1 -- 172.21.15.31:0/2679325584 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a80035b0 con 0x7fa3a8003a00 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.720+0000 7fa3beffd700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3a8003a00 0x7fa3a8003e20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009b60 tx=0x7fa3a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cba2832f07ae66 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000e6c0 con 0x7fa3a8003a00 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000e820 con 0x7fa3a8003a00 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3bdffb700 1 -- 172.21.15.31:0/2679325584 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000eb90 con 0x7fa3a8003a00 2024-03-30T01:43:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3c5470700 1 -- 172.21.15.31:0/2679325584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3a8003a00 msgr2=0x7fa3a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3c5470700 1 --2- 172.21.15.31:0/2679325584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3a8003a00 0x7fa3a8003e20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009b60 tx=0x7fa3a8005880 comp rx=0 tx=0).stop 2024-03-30T01:43:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3bf7fe700 1 -- 172.21.15.31:0/2679325584 reap_dead start 2024-03-30T01:43:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3c5470700 1 -- 172.21.15.31:0/2679325584 shutdown_connections 2024-03-30T01:43:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3c5470700 1 -- 172.21.15.31:0/2679325584 >> 172.21.15.31:0/2679325584 conn(0x7fa3c00c62d0 msgr2=0x7fa3c00c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3c5470700 1 -- 172.21.15.31:0/2679325584 shutdown_connections 2024-03-30T01:43:30.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.721+0000 7fa3c5470700 1 -- 172.21.15.31:0/2679325584 wait complete. 2024-03-30T01:43:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.723+0000 7fa3c5470700 1 Processor -- start 2024-03-30T01:43:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.723+0000 7fa3c5470700 1 -- start start 2024-03-30T01:43:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.723+0000 7fa3c5470700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3a8003a00 0x7fa3c01ed080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.723+0000 7fa3c5470700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3c0144460 0x7fa3c01ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.723+0000 7fa3c5470700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c0143f30 0x7fa3c00c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.723+0000 7fa3c5470700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c0146b90 con 0x7fa3c0143f30 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.723+0000 7fa3c5470700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c0143d50 con 0x7fa3c0144460 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.723+0000 7fa3c5470700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c01edbc0 con 0x7fa3a8003a00 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c0143f30 0x7fa3c00c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c0143f30 0x7fa3c00c5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48772/0 (socket says 172.21.15.31:48772) 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3bf7fe700 1 -- 172.21.15.31:0/4025411516 learned_addr learned my addr 172.21.15.31:0/4025411516 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3be7fc700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3c0144460 0x7fa3c01ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3beffd700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3a8003a00 0x7fa3c01ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3bf7fe700 1 -- 172.21.15.31:0/4025411516 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3a8003a00 msgr2=0x7fa3c01ed080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3bf7fe700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3a8003a00 0x7fa3c01ed080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3bf7fe700 1 -- 172.21.15.31:0/4025411516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3c0144460 msgr2=0x7fa3c01ed5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3bf7fe700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3c0144460 0x7fa3c01ed5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3bf7fe700 1 -- 172.21.15.31:0/4025411516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c01465d0 con 0x7fa3c0143f30 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3be7fc700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3c0144460 0x7fa3c01ed5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3bf7fe700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c0143f30 0x7fa3c00c5280 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0070270 tx=0x7fa3b8011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3a77fe700 1 -- 172.21.15.31:0/4025411516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b800e040 con 0x7fa3c0143f30 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3a77fe700 1 -- 172.21.15.31:0/4025411516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b800f040 con 0x7fa3c0143f30 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3a77fe700 1 -- 172.21.15.31:0/4025411516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b800fdb0 con 0x7fa3c0143f30 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c01468b0 con 0x7fa3c0143f30 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.724+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c0144ef0 con 0x7fa3c0143f30 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.725+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c004ea80 con 0x7fa3c0143f30 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.726+0000 7fa3a77fe700 1 -- 172.21.15.31:0/4025411516 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa3b8022460 con 0x7fa3c0143f30 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.726+0000 7fa3a77fe700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa3ac06ac90 0x7fa3ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.726+0000 7fa3a77fe700 1 -- 172.21.15.31:0/4025411516 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fa3b8089c80 con 0x7fa3c0143f30 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.726+0000 7fa3beffd700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa3ac06ac90 0x7fa3ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:30.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.727+0000 7fa3beffd700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa3ac06ac90 0x7fa3ac06d150 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00090b0 tx=0x7fa3b0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.732+0000 7fa3a77fe700 1 -- 172.21.15.31:0/4025411516 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa3b80549c0 con 0x7fa3c0143f30 2024-03-30T01:43:30.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.948+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3c006b860 con 0x7fa3c0143f30 2024-03-30T01:43:30.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.949+0000 7fa3a77fe700 1 -- 172.21.15.31:0/4025411516 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+19918 (secure 0 0 0) 0x7fa3b8057fe0 con 0x7fa3c0143f30 2024-03-30T01:43:30.956 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-03-30T01:43:30.956 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":64,"fsid":"9739890c-57ec-4503-9fd1-d4b0145023f4","created":"2024-03-30T01:40:34.955060+0000","modified":"2024-03-30T01:43:26.280018+0000","last_up_change":"2024-03-30T01:40:46.902903+0000","last_in_change":"2024-03-30T01:40:37.803254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T01:40:47.857708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-03-30T01:43:09.911033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-03-30T01:43:11.216248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_metadata","create_time":"2024-03-30T01:43:23.783452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs2_data","create_time":"2024-03-30T01:43:24.238093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4d49ae23-471f-4a34-9479-9676567f3867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":47358},{"type":"v1","addr":"172.21.15.31:6801","nonce":47358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":47358},{"type":"v1","addr":"172.21.15.31:6803","nonce":47358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":47358},{"type":"v1","addr":"172.21.15.31:6807","nonce":47358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":47358},{"type":"v1","addr":"172.21.15.31:6805","nonce":47358}]},"public_addr":"172.21.15.31:6801/47358","cluster_addr":"172.21.15.31:6803/47358","heartbeat_back_addr":"172.21.15.31:6807/47358","heartbeat_front_addr":"172.21.15.31:6805/47358","state":["exists","up"]},{"osd":1,"uuid":"19544e17-30cf-428e-9196-f982994de0b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":47369},{"type":"v1","addr":"172.21.15.31:6817","nonce":47369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":47369},{"type":"v1","addr":"172.21.15.31:6819","nonce":47369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":47369},{"type":"v1","addr":"172.21.15.31:6823","nonce":47369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":47369},{"type":"v1","addr":"172.21.15.31:6821","nonce":47369}]},"public_addr":"172.21.15.31:6817/47369","cluster_addr":"172.21.15.31:6819/47369","heartbeat_back_addr":"172.21.15.31:6823/47369","heartbeat_front_addr":"172.21.15.31:6821/47369","state":["exists","up"]},{"osd":2,"uuid":"7412e4be-3d32-4222-9caf-e60156b87fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":47360},{"type":"v1","addr":"172.21.15.31:6809","nonce":47360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":47360},{"type":"v1","addr":"172.21.15.31:6811","nonce":47360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":47360},{"type":"v1","addr":"172.21.15.31:6815","nonce":47360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":47360},{"type":"v1","addr":"172.21.15.31:6813","nonce":47360}]},"public_addr":"172.21.15.31:6809/47360","cluster_addr":"172.21.15.31:6811/47360","heartbeat_back_addr":"172.21.15.31:6815/47360","heartbeat_front_addr":"172.21.15.31:6813/47360","state":["exists","up"]},{"osd":3,"uuid":"91097e90-f74b-4bd9-a1a7-0a2cce9c9df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":47374},{"type":"v1","addr":"172.21.15.31:6825","nonce":47374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":47374},{"type":"v1","addr":"172.21.15.31:6827","nonce":47374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":47374},{"type":"v1","addr":"172.21.15.31:6831","nonce":47374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":47374},{"type":"v1","addr":"172.21.15.31:6829","nonce":47374}]},"public_addr":"172.21.15.31:6825/47374","cluster_addr":"172.21.15.31:6827/47374","heartbeat_back_addr":"172.21.15.31:6831/47374","heartbeat_front_addr":"172.21.15.31:6829/47374","state":["exists","up"]},{"osd":4,"uuid":"ee161410-21c0-40e1-b7d2-54663eb413d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":47512},{"type":"v1","addr":"172.21.15.59:6817","nonce":47512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":47512},{"type":"v1","addr":"172.21.15.59:6819","nonce":47512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":47512},{"type":"v1","addr":"172.21.15.59:6823","nonce":47512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":47512},{"type":"v1","addr":"172.21.15.59:6821","nonce":47512}]},"public_addr":"172.21.15.59:6817/47512","cluster_addr":"172.21.15.59:6819/47512","heartbeat_back_addr":"172.21.15.59:6823/47512","heartbeat_front_addr":"172.21.15.59:6821/47512","state":["exists","up"]},{"osd":5,"uuid":"5e41e79a-4ea9-4753-9895-ffeaa0e14f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":47501},{"type":"v1","addr":"172.21.15.59:6801","nonce":47501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":47501},{"type":"v1","addr":"172.21.15.59:6803","nonce":47501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":47501},{"type":"v1","addr":"172.21.15.59:6807","nonce":47501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":47501},{"type":"v1","addr":"172.21.15.59:6805","nonce":47501}]},"public_addr":"172.21.15.59:6801/47501","cluster_addr":"172.21.15.59:6803/47501","heartbeat_back_addr":"172.21.15.59:6807/47501","heartbeat_front_addr":"172.21.15.59:6805/47501","state":["exists","up"]},{"osd":6,"uuid":"d84d4e71-1b5e-4c4f-b085-a80bd441131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":47524},{"type":"v1","addr":"172.21.15.59:6825","nonce":47524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":47524},{"type":"v1","addr":"172.21.15.59:6827","nonce":47524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":47524},{"type":"v1","addr":"172.21.15.59:6831","nonce":47524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":47524},{"type":"v1","addr":"172.21.15.59:6829","nonce":47524}]},"public_addr":"172.21.15.59:6825/47524","cluster_addr":"172.21.15.59:6827/47524","heartbeat_back_addr":"172.21.15.59:6831/47524","heartbeat_front_addr":"172.21.15.59:6829/47524","state":["exists","up"]},{"osd":7,"uuid":"9ad43344-0b2d-4a25-87df-15217ad5a295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":47502},{"type":"v1","addr":"172.21.15.59:6809","nonce":47502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":47502},{"type":"v1","addr":"172.21.15.59:6811","nonce":47502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":47502},{"type":"v1","addr":"172.21.15.59:6815","nonce":47502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":47502},{"type":"v1","addr":"172.21.15.59:6813","nonce":47502}]},"public_addr":"172.21.15.59:6809/47502","cluster_addr":"172.21.15.59:6811/47502","heartbeat_back_addr":"172.21.15.59:6815/47502","heartbeat_front_addr":"172.21.15.59:6813/47502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.953754+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.035088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.961091+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.969283+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.032267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:43.011218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.993184+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T01:40:42.941227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T01:43:30.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa3ac06ac90 msgr2=0x7fa3ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa3ac06ac90 0x7fa3ac06d150 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00090b0 tx=0x7fa3b0008030 comp rx=0 tx=0).stop 2024-03-30T01:43:30.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c0143f30 msgr2=0x7fa3c00c5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c0143f30 0x7fa3c00c5280 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0070270 tx=0x7fa3b8011f80 comp rx=0 tx=0).stop 2024-03-30T01:43:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 shutdown_connections 2024-03-30T01:43:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c0143f30 0x7fa3c00c5280 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fa3ac06ac90 0x7fa3ac06d150 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3a8003a00 0x7fa3c01ed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 --2- 172.21.15.31:0/4025411516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3c0144460 0x7fa3c01ed5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 >> 172.21.15.31:0/4025411516 conn(0x7fa3c00c62d0 msgr2=0x7fa3c00c2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 shutdown_connections 2024-03-30T01:43:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:30.952+0000 7fa3c5470700 1 -- 172.21.15.31:0/4025411516 wait complete. 2024-03-30T01:43:30.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T01:40:47.857708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T01:43:09.911033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T01:43:11.216248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-30T01:43:23.783452+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-30T01:43:24.238093+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T01:43:30.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T01:43:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.108+0000 7fe829964700 1 Processor -- start 2024-03-30T01:43:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.108+0000 7fe829964700 1 -- start start 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.108+0000 7fe829964700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe824151b60 0x7fe824070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.108+0000 7fe829964700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8240c5d40 0x7fe8240c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.108+0000 7fe829964700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe824154990 con 0x7fe824151f40 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.108+0000 7fe829964700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe824155610 con 0x7fe824151b60 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.108+0000 7fe829964700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8241547a0 con 0x7fe8240c5d40 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.108+0000 7fe8227fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8240c5d40 0x7fe8240c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe822ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe824151b60 0x7fe824070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe8237fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe824151f40 0x7fe8240c6610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.31:44602/0 (socket says 172.21.15.31:44602) 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe8237fe700 1 -- 172.21.15.31:0/81133792 learned_addr learned my addr 172.21.15.31:0/81133792 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe821ffb700 1 -- 172.21.15.31:0/81133792 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292492172 0 0) 0x7fe824154990 con 0x7fe824151f40 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe821ffb700 1 -- 172.21.15.31:0/81133792 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe80c0035b0 con 0x7fe824151f40 2024-03-30T01:43:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe822ffd700 1 -- 172.21.15.31:0/81133792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8240c5d40 msgr2=0x7fe8240c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe822ffd700 1 --2- 172.21.15.31:0/81133792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8240c5d40 0x7fe8240c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe822ffd700 1 -- 172.21.15.31:0/81133792 >> v1:172.21.15.59:6789/0 conn(0x7fe824151f40 legacy=0x7fe8240c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe822ffd700 1 -- 172.21.15.31:0/81133792 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe824154c70 con 0x7fe824151b60 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe822ffd700 1 --2- 172.21.15.31:0/81133792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe824151b60 0x7fe824070680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe814009870 tx=0x7fe814009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b66431617e3df2f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe821ffb700 1 -- 172.21.15.31:0/81133792 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814011830 con 0x7fe824151b60 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe821ffb700 1 -- 172.21.15.31:0/81133792 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814011990 con 0x7fe824151b60 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.109+0000 7fe821ffb700 1 -- 172.21.15.31:0/81133792 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814011d20 con 0x7fe824151b60 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.110+0000 7fe829964700 1 -- 172.21.15.31:0/81133792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe824151b60 msgr2=0x7fe824070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.110+0000 7fe829964700 1 --2- 172.21.15.31:0/81133792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe824151b60 0x7fe824070680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe814009870 tx=0x7fe814009b80 comp rx=0 tx=0).stop 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.110+0000 7fe829964700 1 -- 172.21.15.31:0/81133792 shutdown_connections 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.110+0000 7fe829964700 1 --2- 172.21.15.31:0/81133792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe824151b60 0x7fe824070680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.110+0000 7fe829964700 1 --2- 172.21.15.31:0/81133792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8240c5d40 0x7fe8240c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.110+0000 7fe829964700 1 -- 172.21.15.31:0/81133792 >> 172.21.15.31:0/81133792 conn(0x7fe8240c7d60 msgr2=0x7fe8240c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.110+0000 7fe829964700 1 -- 172.21.15.31:0/81133792 shutdown_connections 2024-03-30T01:43:31.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.110+0000 7fe829964700 1 -- 172.21.15.31:0/81133792 wait complete. 2024-03-30T01:43:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.111+0000 7fe829964700 1 Processor -- start 2024-03-30T01:43:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe829964700 1 -- start start 2024-03-30T01:43:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe829964700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8240c5d40 0x7fe8240ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe829964700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe824151b60 0x7fe824157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe829964700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe824151f40 0x7fe824157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe829964700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe824070830 con 0x7fe8240c5d40 2024-03-30T01:43:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe829964700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8241ed640 con 0x7fe824151f40 2024-03-30T01:43:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe829964700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe824158640 con 0x7fe824151b60 2024-03-30T01:43:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe822ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8240c5d40 0x7fe8240ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe822ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8240c5d40 0x7fe8240ac820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48780/0 (socket says 172.21.15.31:48780) 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.112+0000 7fe822ffd700 1 -- 172.21.15.31:0/3149478689 learned_addr learned my addr 172.21.15.31:0/3149478689 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe8237fe700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe824151f40 0x7fe824157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe8227fc700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe824151b60 0x7fe824157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe822ffd700 1 -- 172.21.15.31:0/3149478689 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe824151b60 msgr2=0x7fe824157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe822ffd700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe824151b60 0x7fe824157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe822ffd700 1 -- 172.21.15.31:0/3149478689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe824151f40 msgr2=0x7fe824157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe822ffd700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe824151f40 0x7fe824157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe822ffd700 1 -- 172.21.15.31:0/3149478689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8140094a0 con 0x7fe8240c5d40 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe822ffd700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8240c5d40 0x7fe8240ac820 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe814000c00 tx=0x7fe814004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe80bfff700 1 -- 172.21.15.31:0/3149478689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8140036a0 con 0x7fe8240c5d40 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe80bfff700 1 -- 172.21.15.31:0/3149478689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81401f070 con 0x7fe8240c5d40 2024-03-30T01:43:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe824158920 con 0x7fe8240c5d40 2024-03-30T01:43:31.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe80bfff700 1 -- 172.21.15.31:0/3149478689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814003a60 con 0x7fe8240c5d40 2024-03-30T01:43:31.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.113+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8241ed920 con 0x7fe8240c5d40 2024-03-30T01:43:31.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.114+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe824070b80 con 0x7fe8240c5d40 2024-03-30T01:43:31.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.115+0000 7fe80bfff700 1 -- 172.21.15.31:0/3149478689 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe814003bc0 con 0x7fe8240c5d40 2024-03-30T01:43:31.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.115+0000 7fe80bfff700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe81006acc0 0x7fe81006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.115+0000 7fe80bfff700 1 -- 172.21.15.31:0/3149478689 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fe814021070 con 0x7fe8240c5d40 2024-03-30T01:43:31.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.115+0000 7fe8227fc700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe81006acc0 0x7fe81006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.116+0000 7fe8227fc700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe81006acc0 0x7fe81006d180 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe824151080 tx=0x7fe818009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.121+0000 7fe80bfff700 1 -- 172.21.15.31:0/3149478689 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe814091080 con 0x7fe8240c5d40 2024-03-30T01:43:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.333+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8240747d0 con 0x7fe8240c5d40 2024-03-30T01:43:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.334+0000 7fe80bfff700 1 -- 172.21.15.31:0/3149478689 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe814015070 con 0x7fe8240c5d40 2024-03-30T01:43:31.340 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-03-30T01:43:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe81006acc0 msgr2=0x7fe81006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe81006acc0 0x7fe81006d180 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe824151080 tx=0x7fe818009210 comp rx=0 tx=0).stop 2024-03-30T01:43:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8240c5d40 msgr2=0x7fe8240ac820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8240c5d40 0x7fe8240ac820 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe814000c00 tx=0x7fe814004750 comp rx=0 tx=0).stop 2024-03-30T01:43:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 shutdown_connections 2024-03-30T01:43:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8240c5d40 0x7fe8240ac820 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fe81006acc0 0x7fe81006d180 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe824151f40 0x7fe824157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 --2- 172.21.15.31:0/3149478689 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe824151b60 0x7fe824157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 >> 172.21.15.31:0/3149478689 conn(0x7fe8240c7d60 msgr2=0x7fe82406d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 shutdown_connections 2024-03-30T01:43:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.337+0000 7fe829964700 1 -- 172.21.15.31:0/3149478689 wait complete. 2024-03-30T01:43:31.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T01:43:31.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.490+0000 7f34704d5700 1 Processor -- start 2024-03-30T01:43:31.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.490+0000 7f34704d5700 1 -- start start 2024-03-30T01:43:31.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f34704d5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f34681549a0 0x7f3468072410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f34704d5700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346814ee60 con 0x7f346806d040 2024-03-30T01:43:31.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f34704d5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34680c2290 con 0x7f346806d900 2024-03-30T01:43:31.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f34704d5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3468155490 con 0x7f34681549a0 2024-03-30T01:43:31.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346da70700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f346806d040 0x7f3468154d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54862/0 (socket says 172.21.15.31:54862) 2024-03-30T01:43:31.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346da70700 1 -- 172.21.15.31:0/3169151642 learned_addr learned my addr 172.21.15.31:0/3169151642 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:31.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346e271700 1 --2- 172.21.15.31:0/3169151642 >> v2:172.21.15.59:3300/0 conn(0x7f34681549a0 0x7f3468072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346d26f700 1 -- 172.21.15.31:0/3169151642 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065273677 0 0) 0x7f346814ee60 con 0x7f346806d040 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346d26f700 1 -- 172.21.15.31:0/3169151642 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34540057c0 con 0x7f346806d040 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346d26f700 1 -- 172.21.15.31:0/3169151642 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350423065 0 0) 0x7f34680c2290 con 0x7f346806d900 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346d26f700 1 -- 172.21.15.31:0/3169151642 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346814ee60 con 0x7f346806d900 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346e271700 1 -- 172.21.15.31:0/3169151642 >> v1:172.21.15.59:6789/0 conn(0x7f346806d900 legacy=0x7f346806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346e271700 1 -- 172.21.15.31:0/3169151642 >> v1:172.21.15.31:6789/0 conn(0x7f346806d040 legacy=0x7f3468154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.491+0000 7f346e271700 1 -- 172.21.15.31:0/3169151642 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3468155720 con 0x7f34681549a0 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.492+0000 7f346e271700 1 --2- 172.21.15.31:0/3169151642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34681549a0 0x7f3468072410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3458009840 tx=0x7f3458009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae1965e3d27ddfae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.492+0000 7f346d26f700 1 -- 172.21.15.31:0/3169151642 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458011830 con 0x7f34681549a0 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.492+0000 7f346d26f700 1 -- 172.21.15.31:0/3169151642 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3458011990 con 0x7f34681549a0 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.492+0000 7f346d26f700 1 -- 172.21.15.31:0/3169151642 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458011d20 con 0x7f34681549a0 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.492+0000 7f34704d5700 1 -- 172.21.15.31:0/3169151642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34681549a0 msgr2=0x7f3468072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.492+0000 7f34704d5700 1 --2- 172.21.15.31:0/3169151642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34681549a0 0x7f3468072410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3458009840 tx=0x7f3458009b50 comp rx=0 tx=0).stop 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.493+0000 7f34704d5700 1 -- 172.21.15.31:0/3169151642 shutdown_connections 2024-03-30T01:43:31.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.493+0000 7f34704d5700 1 --2- 172.21.15.31:0/3169151642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34681549a0 0x7f3468072410 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.493+0000 7f34704d5700 1 -- 172.21.15.31:0/3169151642 >> 172.21.15.31:0/3169151642 conn(0x7f34680c3980 msgr2=0x7f34680c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:31.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.493+0000 7f34704d5700 1 -- 172.21.15.31:0/3169151642 shutdown_connections 2024-03-30T01:43:31.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.493+0000 7f34704d5700 1 -- 172.21.15.31:0/3169151642 wait complete. 2024-03-30T01:43:31.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.494+0000 7f34704d5700 1 Processor -- start 2024-03-30T01:43:31.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.494+0000 7f34704d5700 1 -- start start 2024-03-30T01:43:31.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f34704d5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f346806d040 0x7f34680a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f34704d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346806d900 0x7f34681dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f34704d5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34681549a0 0x7f34680aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f34704d5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34680a9d30 con 0x7f34681549a0 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f34704d5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34680ac450 con 0x7f346806d900 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f34704d5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34681dd730 con 0x7f346806d040 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346ea72700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34681549a0 0x7f34680aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346ea72700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34681549a0 0x7f34680aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48788/0 (socket says 172.21.15.31:48788) 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346ea72700 1 -- 172.21.15.31:0/2981008620 learned_addr learned my addr 172.21.15.31:0/2981008620 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346da70700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346806d900 0x7f34681dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346e271700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f346806d040 0x7f34680a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346ea72700 1 -- 172.21.15.31:0/2981008620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f346806d040 msgr2=0x7f34680a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346ea72700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f346806d040 0x7f34680a97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346ea72700 1 -- 172.21.15.31:0/2981008620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346806d900 msgr2=0x7f34681dd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346ea72700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346806d900 0x7f34681dd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346ea72700 1 -- 172.21.15.31:0/2981008620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34580094a0 con 0x7f34681549a0 2024-03-30T01:43:31.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.495+0000 7f346ea72700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34681549a0 0x7f34680aa180 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f346000cc20 tx=0x7f346000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:31.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.496+0000 7f345effd700 1 -- 172.21.15.31:0/2981008620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460003c00 con 0x7f34681549a0 2024-03-30T01:43:31.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.496+0000 7f345effd700 1 -- 172.21.15.31:0/2981008620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3460003d60 con 0x7f34681549a0 2024-03-30T01:43:31.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.496+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34680a9510 con 0x7f34681549a0 2024-03-30T01:43:31.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.496+0000 7f345effd700 1 -- 172.21.15.31:0/2981008620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346000aae0 con 0x7f34681549a0 2024-03-30T01:43:31.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.496+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34680c69d0 con 0x7f34681549a0 2024-03-30T01:43:31.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.497+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34680abfb0 con 0x7f34681549a0 2024-03-30T01:43:31.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.497+0000 7f345effd700 1 -- 172.21.15.31:0/2981008620 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f346000ac40 con 0x7f34681549a0 2024-03-30T01:43:31.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.497+0000 7f345effd700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f344c06ac90 0x7f344c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.498+0000 7f345effd700 1 -- 172.21.15.31:0/2981008620 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f34600897e0 con 0x7f34681549a0 2024-03-30T01:43:31.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.498+0000 7f346e271700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f344c06ac90 0x7f344c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.498+0000 7f346e271700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f344c06ac90 0x7f344c06d150 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3458009fd0 tx=0x7f345801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:31.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.502+0000 7f345effd700 1 -- 172.21.15.31:0/2981008620 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f34600545b0 con 0x7f34681549a0 2024-03-30T01:43:31.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.716+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34680c6bf0 con 0x7f34681549a0 2024-03-30T01:43:31.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.717+0000 7f345effd700 1 -- 172.21.15.31:0/2981008620 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3460057bd0 con 0x7f34681549a0 2024-03-30T01:43:31.722 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:43:31.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f344c06ac90 msgr2=0x7f344c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f344c06ac90 0x7f344c06d150 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3458009fd0 tx=0x7f345801c040 comp rx=0 tx=0).stop 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34681549a0 msgr2=0x7f34680aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34681549a0 0x7f34680aa180 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f346000cc20 tx=0x7f346000cf30 comp rx=0 tx=0).stop 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 shutdown_connections 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34681549a0 0x7f34680aa180 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f344c06ac90 0x7f344c06d150 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f346806d040 0x7f34680a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 --2- 172.21.15.31:0/2981008620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346806d900 0x7f34681dd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 >> 172.21.15.31:0/2981008620 conn(0x7f34680c3980 msgr2=0x7f34680c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 shutdown_connections 2024-03-30T01:43:31.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.719+0000 7f34704d5700 1 -- 172.21.15.31:0/2981008620 wait complete. 2024-03-30T01:43:31.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T01:43:31.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.872+0000 7f194b0a5700 1 Processor -- start 2024-03-30T01:43:31.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.872+0000 7f194b0a5700 1 -- start start 2024-03-30T01:43:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f194b0a5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 0x7f1944070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f194b0a5700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1944151f40 0x7f19440c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f194b0a5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1944154990 con 0x7f19440c5d40 2024-03-30T01:43:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f194b0a5700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1944155610 con 0x7f1944151f40 2024-03-30T01:43:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f194b0a5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19441547a0 con 0x7f1944151b60 2024-03-30T01:43:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f1949642700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1944151f40 0x7f19440c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f1949642700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1944151f40 0x7f19440c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48800/0 (socket says 172.21.15.31:48800) 2024-03-30T01:43:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f1949642700 1 -- 172.21.15.31:0/696801097 learned_addr learned my addr 172.21.15.31:0/696801097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f1948e41700 1 --2- 172.21.15.31:0/696801097 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 0x7f1944070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f1949642700 1 -- 172.21.15.31:0/696801097 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 msgr2=0x7f1944070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f1949642700 1 --2- 172.21.15.31:0/696801097 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 0x7f1944070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f1949642700 1 -- 172.21.15.31:0/696801097 >> v1:172.21.15.59:6789/0 conn(0x7f19440c5d40 legacy=0x7f19440c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f1949642700 1 -- 172.21.15.31:0/696801097 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944154c70 con 0x7f1944151f40 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.873+0000 7f1949642700 1 --2- 172.21.15.31:0/696801097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 0x7f19440c6610 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f193c009e80 tx=0x7f193c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cb658d49c58f2c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f19437fe700 1 -- 172.21.15.31:0/696801097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00e070 con 0x7f1944151f40 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f19437fe700 1 -- 172.21.15.31:0/696801097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c00b8d0 con 0x7f1944151f40 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f19437fe700 1 -- 172.21.15.31:0/696801097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00bc80 con 0x7f1944151f40 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f194b0a5700 1 -- 172.21.15.31:0/696801097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 msgr2=0x7f19440c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f194b0a5700 1 --2- 172.21.15.31:0/696801097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 0x7f19440c6610 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f193c009e80 tx=0x7f193c00b330 comp rx=0 tx=0).stop 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f194b0a5700 1 -- 172.21.15.31:0/696801097 shutdown_connections 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f194b0a5700 1 --2- 172.21.15.31:0/696801097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 0x7f19440c6610 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f194b0a5700 1 --2- 172.21.15.31:0/696801097 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 0x7f1944070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f194b0a5700 1 -- 172.21.15.31:0/696801097 >> 172.21.15.31:0/696801097 conn(0x7f19440c7d60 msgr2=0x7f19440c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f194b0a5700 1 -- 172.21.15.31:0/696801097 shutdown_connections 2024-03-30T01:43:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.874+0000 7f194b0a5700 1 -- 172.21.15.31:0/696801097 wait complete. 2024-03-30T01:43:31.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.876+0000 7f194b0a5700 1 Processor -- start 2024-03-30T01:43:31.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.876+0000 7f194b0a5700 1 -- start start 2024-03-30T01:43:31.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f194b0a5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c5d40 0x7f1944158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f194b0a5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 0x7f1944070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f194b0a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 0x7f194414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f194b0a5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1944157bb0 con 0x7f1944151f40 2024-03-30T01:43:31.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f194b0a5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1944156da0 con 0x7f19440c5d40 2024-03-30T01:43:31.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f194b0a5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19440abc20 con 0x7f1944151b60 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1949642700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 0x7f194414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1949642700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 0x7f194414c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48814/0 (socket says 172.21.15.31:48814) 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1949642700 1 -- 172.21.15.31:0/1870255693 learned_addr learned my addr 172.21.15.31:0/1870255693 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1948e41700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c5d40 0x7f1944158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1943fff700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 0x7f1944070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1949642700 1 -- 172.21.15.31:0/1870255693 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 msgr2=0x7f1944070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1949642700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 0x7f1944070340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1949642700 1 -- 172.21.15.31:0/1870255693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c5d40 msgr2=0x7f1944158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1949642700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c5d40 0x7f1944158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1949642700 1 -- 172.21.15.31:0/1870255693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c009b30 con 0x7f1944151f40 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f1949642700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 0x7f194414c110 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1944071800 tx=0x7f193c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f19417fa700 1 -- 172.21.15.31:0/1870255693 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c004960 con 0x7f1944151f40 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f19417fa700 1 -- 172.21.15.31:0/1870255693 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c0092e0 con 0x7f1944151f40 2024-03-30T01:43:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19440ac480 con 0x7f1944151f40 2024-03-30T01:43:31.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.877+0000 7f19417fa700 1 -- 172.21.15.31:0/1870255693 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c01c660 con 0x7f1944151f40 2024-03-30T01:43:31.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.878+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1944158870 con 0x7f1944151f40 2024-03-30T01:43:31.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.878+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19440ac610 con 0x7f1944151f40 2024-03-30T01:43:31.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.879+0000 7f19417fa700 1 -- 172.21.15.31:0/1870255693 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f193c01c7c0 con 0x7f1944151f40 2024-03-30T01:43:31.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.879+0000 7f19417fa700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806acc0 0x7f192806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:31.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.879+0000 7f19417fa700 1 -- 172.21.15.31:0/1870255693 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f193c08b1a0 con 0x7f1944151f40 2024-03-30T01:43:31.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.880+0000 7f1948e41700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806acc0 0x7f192806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:31.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.880+0000 7f1948e41700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806acc0 0x7f192806d180 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1938009b70 tx=0x7f1938008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:31.884+0000 7f19417fa700 1 -- 172.21.15.31:0/1870255693 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f193c055f70 con 0x7f1944151f40 2024-03-30T01:43:32.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.100+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f19440747d0 con 0x7f1944151f40 2024-03-30T01:43:32.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.101+0000 7f19417fa700 1 -- 172.21.15.31:0/1870255693 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7f193c059590 con 0x7f1944151f40 2024-03-30T01:43:32.106 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:43:32.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806acc0 msgr2=0x7f192806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806acc0 0x7f192806d180 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1938009b70 tx=0x7f1938008040 comp rx=0 tx=0).stop 2024-03-30T01:43:32.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 msgr2=0x7f194414c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 0x7f194414c110 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1944071800 tx=0x7f193c004ad0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 shutdown_connections 2024-03-30T01:43:32.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1944151f40 0x7f194414c110 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f192806acc0 0x7f192806d180 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19440c5d40 0x7f1944158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 --2- 172.21.15.31:0/1870255693 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1944151b60 0x7f1944070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 >> 172.21.15.31:0/1870255693 conn(0x7f19440c7d60 msgr2=0x7f19440c8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:32.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 shutdown_connections 2024-03-30T01:43:32.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.103+0000 7f194b0a5700 1 -- 172.21.15.31:0/1870255693 wait complete. 2024-03-30T01:43:32.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-30T01:43:32.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.255+0000 7f8695f64700 1 Processor -- start 2024-03-30T01:43:32.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.255+0000 7f8695f64700 1 -- start start 2024-03-30T01:43:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.256+0000 7f8695f64700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f869014fd40 0x7f8690152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.256+0000 7f8695f64700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f86900c4ed0 0x7f86900a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.256+0000 7f8695f64700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86900704d0 con 0x7f86900aada0 2024-03-30T01:43:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.256+0000 7f8695f64700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86900c6d30 con 0x7f869014fd40 2024-03-30T01:43:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.256+0000 7f8695f64700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86901527a0 con 0x7f86900c4ed0 2024-03-30T01:43:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.256+0000 7f868f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f869014fd40 0x7f8690152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.256+0000 7f868effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f86900aada0 0x7f86900ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54872/0 (socket says 172.21.15.31:54872) 2024-03-30T01:43:32.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868effd700 1 -- 172.21.15.31:0/3366188891 learned_addr learned my addr 172.21.15.31:0/3366188891 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868f7fe700 1 --2- 172.21.15.31:0/3366188891 >> v2:172.21.15.31:3300/0 conn(0x7f869014fd40 0x7f8690152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48818/0 (socket says 172.21.15.31:48818) 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868ffff700 1 --2- 172.21.15.31:0/3366188891 >> v2:172.21.15.59:3300/0 conn(0x7f86900c4ed0 0x7f86900a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868f7fe700 1 -- 172.21.15.31:0/3366188891 >> v2:172.21.15.59:3300/0 conn(0x7f86900c4ed0 msgr2=0x7f86900a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868f7fe700 1 --2- 172.21.15.31:0/3366188891 >> v2:172.21.15.59:3300/0 conn(0x7f86900c4ed0 0x7f86900a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868f7fe700 1 -- 172.21.15.31:0/3366188891 >> v1:172.21.15.31:6789/0 conn(0x7f86900aada0 legacy=0x7f86900ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868f7fe700 1 -- 172.21.15.31:0/3366188891 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8690070720 con 0x7f869014fd40 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868e7fc700 1 -- 172.21.15.31:0/3366188891 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001537512 0 0) 0x7f86900704d0 con 0x7f86900aada0 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868f7fe700 1 --2- 172.21.15.31:0/3366188891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f869014fd40 0x7f8690152ed0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f86900aaab0 tx=0x7f8680009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb699d52180c3350 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868e7fc700 1 -- 172.21.15.31:0/3366188891 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680011830 con 0x7f869014fd40 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868e7fc700 1 -- 172.21.15.31:0/3366188891 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8680011990 con 0x7f869014fd40 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.257+0000 7f868e7fc700 1 -- 172.21.15.31:0/3366188891 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680011d20 con 0x7f869014fd40 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.258+0000 7f8695f64700 1 -- 172.21.15.31:0/3366188891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f869014fd40 msgr2=0x7f8690152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.258+0000 7f8695f64700 1 --2- 172.21.15.31:0/3366188891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f869014fd40 0x7f8690152ed0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f86900aaab0 tx=0x7f8680009b20 comp rx=0 tx=0).stop 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.258+0000 7f8695f64700 1 -- 172.21.15.31:0/3366188891 shutdown_connections 2024-03-30T01:43:32.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.258+0000 7f8695f64700 1 --2- 172.21.15.31:0/3366188891 >> v2:172.21.15.59:3300/0 conn(0x7f86900c4ed0 0x7f86900a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.258+0000 7f8695f64700 1 --2- 172.21.15.31:0/3366188891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f869014fd40 0x7f8690152ed0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.258+0000 7f8695f64700 1 -- 172.21.15.31:0/3366188891 >> 172.21.15.31:0/3366188891 conn(0x7f86900c7cf0 msgr2=0x7f86900c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:32.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.258+0000 7f8695f64700 1 -- 172.21.15.31:0/3366188891 shutdown_connections 2024-03-30T01:43:32.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.258+0000 7f8695f64700 1 -- 172.21.15.31:0/3366188891 wait complete. 2024-03-30T01:43:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.260+0000 7f8695f64700 1 Processor -- start 2024-03-30T01:43:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.260+0000 7f8695f64700 1 -- start start 2024-03-30T01:43:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f8695f64700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86900c4ed0 0x7f869014c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f8695f64700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f869014fd40 0x7f86901eb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f8695f64700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86900a7910 0x7f86901eb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f8695f64700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869014cae0 con 0x7f86900c4ed0 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f8695f64700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86901ebcb0 con 0x7f86900a7910 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f8695f64700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86900bcdb0 con 0x7f869014fd40 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86900c4ed0 0x7f869014c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86900c4ed0 0x7f869014c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48828/0 (socket says 172.21.15.31:48828) 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868f7fe700 1 -- 172.21.15.31:0/1140260771 learned_addr learned my addr 172.21.15.31:0/1140260771 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868effd700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f869014fd40 0x7f86901eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868ffff700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86900a7910 0x7f86901eb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868f7fe700 1 -- 172.21.15.31:0/1140260771 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f869014fd40 msgr2=0x7f86901eb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868f7fe700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f869014fd40 0x7f86901eb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868f7fe700 1 -- 172.21.15.31:0/1140260771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86900a7910 msgr2=0x7f86901eb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868f7fe700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86900a7910 0x7f86901eb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868f7fe700 1 -- 172.21.15.31:0/1140260771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800094a0 con 0x7f86900c4ed0 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868ffff700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86900a7910 0x7f86901eb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.261+0000 7f868f7fe700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86900c4ed0 0x7f869014c5a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8680000c00 tx=0x7f8680004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.262+0000 7f8677fff700 1 -- 172.21.15.31:0/1140260771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86800045d0 con 0x7f86900c4ed0 2024-03-30T01:43:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.262+0000 7f8677fff700 1 -- 172.21.15.31:0/1140260771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868001f070 con 0x7f86900c4ed0 2024-03-30T01:43:32.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.262+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86900bd090 con 0x7f86900c4ed0 2024-03-30T01:43:32.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.262+0000 7f8677fff700 1 -- 172.21.15.31:0/1140260771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86800038e0 con 0x7f86900c4ed0 2024-03-30T01:43:32.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.262+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86901ebf90 con 0x7f86900c4ed0 2024-03-30T01:43:32.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.263+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86900747d0 con 0x7f86900c4ed0 2024-03-30T01:43:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.263+0000 7f8677fff700 1 -- 172.21.15.31:0/1140260771 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8680003a40 con 0x7f86900c4ed0 2024-03-30T01:43:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.264+0000 7f8677fff700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f867c06ac90 0x7f867c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.264+0000 7f8677fff700 1 -- 172.21.15.31:0/1140260771 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f8680021070 con 0x7f86900c4ed0 2024-03-30T01:43:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.264+0000 7f868effd700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f867c06ac90 0x7f867c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.264+0000 7f868effd700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f867c06ac90 0x7f867c06d150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f86900c44c0 tx=0x7f8684007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:32.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.268+0000 7f8677fff700 1 -- 172.21.15.31:0/1140260771 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8680055e40 con 0x7f86900c4ed0 2024-03-30T01:43:32.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.483+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86900c72c0 con 0x7f86900c4ed0 2024-03-30T01:43:32.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.483+0000 7f8677fff700 1 -- 172.21.15.31:0/1140260771 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v64) v1 ==== 105+0+11 (secure 0 0 0) 0x7f86800150b0 con 0x7f86900c4ed0 2024-03-30T01:43:32.489 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-03-30T01:43:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f867c06ac90 msgr2=0x7f867c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f867c06ac90 0x7f867c06d150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f86900c44c0 tx=0x7f8684007e70 comp rx=0 tx=0).stop 2024-03-30T01:43:32.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86900c4ed0 msgr2=0x7f869014c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86900c4ed0 0x7f869014c5a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8680000c00 tx=0x7f8680004750 comp rx=0 tx=0).stop 2024-03-30T01:43:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 shutdown_connections 2024-03-30T01:43:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86900c4ed0 0x7f869014c5a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f867c06ac90 0x7f867c06d150 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86900a7910 0x7f86901eb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 --2- 172.21.15.31:0/1140260771 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f869014fd40 0x7f86901eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 >> 172.21.15.31:0/1140260771 conn(0x7f86900c7cf0 msgr2=0x7f86900a7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 shutdown_connections 2024-03-30T01:43:32.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.486+0000 7f8695f64700 1 -- 172.21.15.31:0/1140260771 wait complete. 2024-03-30T01:43:32.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-30T01:43:32.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.638+0000 7ffb26286700 1 Processor -- start 2024-03-30T01:43:32.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.638+0000 7ffb26286700 1 -- start start 2024-03-30T01:43:32.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.639+0000 7ffb26286700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb20151ea0 0x7ffb200c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.639+0000 7ffb26286700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200c07f0 0x7ffb2006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.639+0000 7ffb26286700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb20155000 con 0x7ffb200721f0 2024-03-30T01:43:32.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.639+0000 7ffb26286700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2006e930 con 0x7ffb20151ea0 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.639+0000 7ffb26286700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2006df40 con 0x7ffb200c07f0 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.639+0000 7ffb1ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb20151ea0 0x7ffb200c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.639+0000 7ffb1ffff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb20151ea0 0x7ffb200c02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48840/0 (socket says 172.21.15.31:48840) 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.639+0000 7ffb1ffff700 1 -- 172.21.15.31:0/3857212428 learned_addr learned my addr 172.21.15.31:0/3857212428 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.639+0000 7ffb24823700 1 --2- 172.21.15.31:0/3857212428 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200c07f0 0x7ffb2006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.640+0000 7ffb1ffff700 1 -- 172.21.15.31:0/3857212428 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200c07f0 msgr2=0x7ffb2006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.640+0000 7ffb1ffff700 1 --2- 172.21.15.31:0/3857212428 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200c07f0 0x7ffb2006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.640+0000 7ffb1ffff700 1 -- 172.21.15.31:0/3857212428 >> v1:172.21.15.59:6789/0 conn(0x7ffb200721f0 legacy=0x7ffb2006a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.640+0000 7ffb1ffff700 1 -- 172.21.15.31:0/3857212428 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2006e650 con 0x7ffb20151ea0 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.640+0000 7ffb24823700 1 --2- 172.21.15.31:0/3857212428 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200c07f0 0x7ffb2006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.640+0000 7ffb1ffff700 1 --2- 172.21.15.31:0/3857212428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb20151ea0 0x7ffb200c02b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ffb100098f0 tx=0x7ffb10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dfe2427b812bc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.640+0000 7ffb1effd700 1 -- 172.21.15.31:0/3857212428 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000b8a0 con 0x7ffb20151ea0 2024-03-30T01:43:32.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.640+0000 7ffb1effd700 1 -- 172.21.15.31:0/3857212428 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1000ba00 con 0x7ffb20151ea0 2024-03-30T01:43:32.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.640+0000 7ffb1effd700 1 -- 172.21.15.31:0/3857212428 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000bd90 con 0x7ffb20151ea0 2024-03-30T01:43:32.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.641+0000 7ffb26286700 1 -- 172.21.15.31:0/3857212428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb20151ea0 msgr2=0x7ffb200c02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.641+0000 7ffb26286700 1 --2- 172.21.15.31:0/3857212428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb20151ea0 0x7ffb200c02b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ffb100098f0 tx=0x7ffb10009c00 comp rx=0 tx=0).stop 2024-03-30T01:43:32.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.641+0000 7ffb26286700 1 -- 172.21.15.31:0/3857212428 shutdown_connections 2024-03-30T01:43:32.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.641+0000 7ffb26286700 1 --2- 172.21.15.31:0/3857212428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb20151ea0 0x7ffb200c02b0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.641+0000 7ffb26286700 1 --2- 172.21.15.31:0/3857212428 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200c07f0 0x7ffb2006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.641+0000 7ffb26286700 1 -- 172.21.15.31:0/3857212428 >> 172.21.15.31:0/3857212428 conn(0x7ffb200c3980 msgr2=0x7ffb200c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:32.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.641+0000 7ffb26286700 1 -- 172.21.15.31:0/3857212428 shutdown_connections 2024-03-30T01:43:32.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.641+0000 7ffb26286700 1 -- 172.21.15.31:0/3857212428 wait complete. 2024-03-30T01:43:32.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.642+0000 7ffb26286700 1 Processor -- start 2024-03-30T01:43:32.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.643+0000 7ffb26286700 1 -- start start 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.643+0000 7ffb26286700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb200721f0 0x7ffb201585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.643+0000 7ffb26286700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb200c07f0 0x7ffb20157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.643+0000 7ffb26286700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb20151ea0 0x7ffb2014c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.643+0000 7ffb26286700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb201edfa0 con 0x7ffb200c07f0 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.643+0000 7ffb26286700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb201edb10 con 0x7ffb200721f0 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.643+0000 7ffb26286700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20156d50 con 0x7ffb20151ea0 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb200c07f0 0x7ffb20157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb200c07f0 0x7ffb20157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48844/0 (socket says 172.21.15.31:48844) 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1f7fe700 1 -- 172.21.15.31:0/3983908369 learned_addr learned my addr 172.21.15.31:0/3983908369 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1ffff700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb200721f0 0x7ffb201585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb24823700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb20151ea0 0x7ffb2014c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1f7fe700 1 -- 172.21.15.31:0/3983908369 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb20151ea0 msgr2=0x7ffb2014c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1f7fe700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb20151ea0 0x7ffb2014c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1f7fe700 1 -- 172.21.15.31:0/3983908369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb200721f0 msgr2=0x7ffb201585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1f7fe700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb200721f0 0x7ffb201585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1f7fe700 1 -- 172.21.15.31:0/3983908369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb10009550 con 0x7ffb200c07f0 2024-03-30T01:43:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1f7fe700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb200c07f0 0x7ffb20157900 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c4e0 tx=0x7ffb0800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1cff9700 1 -- 172.21.15.31:0/3983908369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080052d0 con 0x7ffb200c07f0 2024-03-30T01:43:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1cff9700 1 -- 172.21.15.31:0/3983908369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb08005430 con 0x7ffb200c07f0 2024-03-30T01:43:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb201ed1e0 con 0x7ffb200c07f0 2024-03-30T01:43:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb1cff9700 1 -- 172.21.15.31:0/3983908369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080038c0 con 0x7ffb200c07f0 2024-03-30T01:43:32.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.644+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb201ed7a0 con 0x7ffb200c07f0 2024-03-30T01:43:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.646+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb200c32d0 con 0x7ffb200c07f0 2024-03-30T01:43:32.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.646+0000 7ffb1cff9700 1 -- 172.21.15.31:0/3983908369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffb08003a20 con 0x7ffb200c07f0 2024-03-30T01:43:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.646+0000 7ffb1cff9700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffb0006ac90 0x7ffb0006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.647+0000 7ffb1cff9700 1 -- 172.21.15.31:0/3983908369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7ffb08088a80 con 0x7ffb200c07f0 2024-03-30T01:43:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.647+0000 7ffb1ffff700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffb0006ac90 0x7ffb0006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:32.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.647+0000 7ffb1ffff700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffb0006ac90 0x7ffb0006d150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffb10014040 tx=0x7ffb10008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:32.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.651+0000 7ffb1cff9700 1 -- 172.21.15.31:0/3983908369 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ffb08053850 con 0x7ffb200c07f0 2024-03-30T01:43:32.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.865+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7ffb200747d0 con 0x7ffb200c07f0 2024-03-30T01:43:32.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.866+0000 7ffb1cff9700 1 -- 172.21.15.31:0/3983908369 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v64) v1 ==== 101+0+11 (secure 0 0 0) 0x7ffb08017020 con 0x7ffb200c07f0 2024-03-30T01:43:32.871 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-03-30T01:43:32.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffb0006ac90 msgr2=0x7ffb0006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffb0006ac90 0x7ffb0006d150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffb10014040 tx=0x7ffb10008030 comp rx=0 tx=0).stop 2024-03-30T01:43:32.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb200c07f0 msgr2=0x7ffb20157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:32.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb200c07f0 0x7ffb20157900 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c4e0 tx=0x7ffb0800c7f0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 shutdown_connections 2024-03-30T01:43:32.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7ffb0006ac90 0x7ffb0006d150 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb20151ea0 0x7ffb2014c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb200721f0 0x7ffb201585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 --2- 172.21.15.31:0/3983908369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb200c07f0 0x7ffb20157900 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.868+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 >> 172.21.15.31:0/3983908369 conn(0x7ffb200c3980 msgr2=0x7ffb200c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.869+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 shutdown_connections 2024-03-30T01:43:32.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:32.869+0000 7ffb26286700 1 -- 172.21.15.31:0/3983908369 wait complete. 2024-03-30T01:43:32.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T01:43:33.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.020+0000 7f713d5f8700 1 Processor -- start 2024-03-30T01:43:33.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.020+0000 7f713d5f8700 1 -- start start 2024-03-30T01:43:33.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.020+0000 7f713d5f8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71380a86d0 0x7f7138153040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.020+0000 7f713d5f8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380c6990 0x7f713814f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.020+0000 7f713d5f8700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7138156680 con 0x7f713814fce0 2024-03-30T01:43:33.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.020+0000 7f713d5f8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71380a9c10 con 0x7f71380a86d0 2024-03-30T01:43:33.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.020+0000 7f713d5f8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7138153620 con 0x7f71380c6990 2024-03-30T01:43:33.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f71377fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f713814fce0 0x7f713806cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54876/0 (socket says 172.21.15.31:54876) 2024-03-30T01:43:33.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f71377fe700 1 -- 172.21.15.31:0/4194131331 learned_addr learned my addr 172.21.15.31:0/4194131331 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:33.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f71367fc700 1 --2- 172.21.15.31:0/4194131331 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380c6990 0x7f713814f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f7136ffd700 1 --2- 172.21.15.31:0/4194131331 >> v2:172.21.15.59:3300/0 conn(0x7f71380a86d0 0x7f7138153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f7135ffb700 1 -- 172.21.15.31:0/4194131331 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042069401 0 0) 0x7f7138156680 con 0x7f713814fce0 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f7135ffb700 1 -- 172.21.15.31:0/4194131331 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71240035b0 con 0x7f713814fce0 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f71367fc700 1 -- 172.21.15.31:0/4194131331 >> v2:172.21.15.59:3300/0 conn(0x7f71380a86d0 msgr2=0x7f7138153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f71367fc700 1 --2- 172.21.15.31:0/4194131331 >> v2:172.21.15.59:3300/0 conn(0x7f71380a86d0 0x7f7138153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f71367fc700 1 -- 172.21.15.31:0/4194131331 >> v1:172.21.15.31:6789/0 conn(0x7f713814fce0 legacy=0x7f713806cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f7135ffb700 1 -- 172.21.15.31:0/4194131331 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 412958778 0 0) 0x7f71240035b0 con 0x7f713814fce0 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f71367fc700 1 -- 172.21.15.31:0/4194131331 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7138156b10 con 0x7f71380c6990 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f7136ffd700 1 --2- 172.21.15.31:0/4194131331 >> v2:172.21.15.59:3300/0 conn(0x7f71380a86d0 0x7f7138153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.021+0000 7f71367fc700 1 --2- 172.21.15.31:0/4194131331 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380c6990 0x7f713814f7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7120009840 tx=0x7f7120009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c358538f2a4dad33 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f7135ffb700 1 -- 172.21.15.31:0/4194131331 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71300019f0 con 0x7f71380c6990 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f7135ffb700 1 -- 172.21.15.31:0/4194131331 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120004030 con 0x7f71380c6990 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f7135ffb700 1 -- 172.21.15.31:0/4194131331 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200043a0 con 0x7f71380c6990 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f713d5f8700 1 -- 172.21.15.31:0/4194131331 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380c6990 msgr2=0x7f713814f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f713d5f8700 1 --2- 172.21.15.31:0/4194131331 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380c6990 0x7f713814f7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7120009840 tx=0x7f7120009b50 comp rx=0 tx=0).stop 2024-03-30T01:43:33.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f713d5f8700 1 -- 172.21.15.31:0/4194131331 shutdown_connections 2024-03-30T01:43:33.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f713d5f8700 1 --2- 172.21.15.31:0/4194131331 >> v2:172.21.15.59:3300/0 conn(0x7f71380a86d0 0x7f7138153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f713d5f8700 1 --2- 172.21.15.31:0/4194131331 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380c6990 0x7f713814f7a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f713d5f8700 1 -- 172.21.15.31:0/4194131331 >> 172.21.15.31:0/4194131331 conn(0x7f713806af80 msgr2=0x7f71380c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:33.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f713d5f8700 1 -- 172.21.15.31:0/4194131331 shutdown_connections 2024-03-30T01:43:33.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.022+0000 7f713d5f8700 1 -- 172.21.15.31:0/4194131331 wait complete. 2024-03-30T01:43:33.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.024+0000 7f713d5f8700 1 Processor -- start 2024-03-30T01:43:33.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.024+0000 7f713d5f8700 1 -- start start 2024-03-30T01:43:33.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f713d5f8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380a86d0 0x7f7138145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f713d5f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380c6990 0x7f7138152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f713d5f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f713814fce0 0x7f7138149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f713d5f8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138145480 con 0x7f71380c6990 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f713d5f8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138145cc0 con 0x7f713814fce0 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f713d5f8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7138145f20 con 0x7f71380a86d0 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71367fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380c6990 0x7f7138152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71367fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380c6990 0x7f7138152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48854/0 (socket says 172.21.15.31:48854) 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71367fc700 1 -- 172.21.15.31:0/1430475941 learned_addr learned my addr 172.21.15.31:0/1430475941 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f7136ffd700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380a86d0 0x7f7138145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71377fe700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f713814fce0 0x7f7138149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71367fc700 1 -- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380a86d0 msgr2=0x7f7138145780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71367fc700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380a86d0 0x7f7138145780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71367fc700 1 -- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f713814fce0 msgr2=0x7f7138149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71367fc700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f713814fce0 0x7f7138149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71367fc700 1 -- 172.21.15.31:0/1430475941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280094a0 con 0x7f71380c6990 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71377fe700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f713814fce0 0x7f7138149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:33.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f7136ffd700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380a86d0 0x7f7138145780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:33.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f71367fc700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380c6990 0x7f7138152d30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f712000a010 tx=0x7f7120004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:33.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.025+0000 7f712f7fe700 1 -- 172.21.15.31:0/1430475941 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120004590 con 0x7f71380c6990 2024-03-30T01:43:33.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.026+0000 7f712f7fe700 1 -- 172.21.15.31:0/1430475941 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712001f070 con 0x7f71380c6990 2024-03-30T01:43:33.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.026+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71200094a0 con 0x7f71380c6990 2024-03-30T01:43:33.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.026+0000 7f712f7fe700 1 -- 172.21.15.31:0/1430475941 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120011870 con 0x7f71380c6990 2024-03-30T01:43:33.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.026+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7138145110 con 0x7f71380c6990 2024-03-30T01:43:33.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.027+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713806e0e0 con 0x7f71380c6990 2024-03-30T01:43:33.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.027+0000 7f712f7fe700 1 -- 172.21.15.31:0/1430475941 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f71200119d0 con 0x7f71380c6990 2024-03-30T01:43:33.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.028+0000 7f712f7fe700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f711806ac90 0x7f711806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.028+0000 7f7136ffd700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f711806ac90 0x7f711806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.028+0000 7f712f7fe700 1 -- 172.21.15.31:0/1430475941 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f7120021070 con 0x7f71380c6990 2024-03-30T01:43:33.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.028+0000 7f7136ffd700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f711806ac90 0x7f711806d150 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7128009c50 tx=0x7f7128008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:33.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.032+0000 7f712f7fe700 1 -- 172.21.15.31:0/1430475941 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f71200596b0 con 0x7f71380c6990 2024-03-30T01:43:33.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.265+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f71381455c0 con 0x7f71380c6990 2024-03-30T01:43:33.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.265+0000 7f712f7fe700 1 -- 172.21.15.31:0/1430475941 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v64) v1 ==== 80+0+0 (secure 0 0 0) 0x7f71381455c0 con 0x7f71380c6990 2024-03-30T01:43:33.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f711806ac90 msgr2=0x7f711806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f711806ac90 0x7f711806d150 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7128009c50 tx=0x7f7128008040 comp rx=0 tx=0).stop 2024-03-30T01:43:33.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380c6990 msgr2=0x7f7138152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380c6990 0x7f7138152d30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f712000a010 tx=0x7f7120004750 comp rx=0 tx=0).stop 2024-03-30T01:43:33.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 shutdown_connections 2024-03-30T01:43:33.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f711806ac90 0x7f711806d150 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f713814fce0 0x7f7138149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71380a86d0 0x7f7138145780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 --2- 172.21.15.31:0/1430475941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380c6990 0x7f7138152d30 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 >> 172.21.15.31:0/1430475941 conn(0x7f713806af80 msgr2=0x7f713814da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:33.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 shutdown_connections 2024-03-30T01:43:33.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.267+0000 7f713d5f8700 1 -- 172.21.15.31:0/1430475941 wait complete. 2024-03-30T01:43:33.274 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2024-03-30T01:43:33.287 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-30T01:43:33.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:33.287 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-30T01:43:33.339 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-03-30T01:43:33.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:33.339 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T01:43:33.360 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:43:33.360 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:43:33.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247272k,nr_inodes=4061818,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1655 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-30T01:43:33.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T01:43:33.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.563+0000 7f33df4ce700 1 Processor -- start 2024-03-30T01:43:33.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.564+0000 7f33df4ce700 1 -- start start 2024-03-30T01:43:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33df4ce700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33d8071750 0x7f33d8154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33df4ce700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d8072160 con 0x7f33d814e5f0 2024-03-30T01:43:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33df4ce700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d80c2bd0 con 0x7f33d8071240 2024-03-30T01:43:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33df4ce700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d80c2e90 con 0x7f33d8071750 2024-03-30T01:43:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33dd26a700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f33d814e5f0 0x7f33d80c3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54884/0 (socket says 172.21.15.31:54884) 2024-03-30T01:43:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33dd26a700 1 -- 172.21.15.31:0/2166032680 learned_addr learned my addr 172.21.15.31:0/2166032680 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33dda6b700 1 --2- 172.21.15.31:0/2166032680 >> v2:172.21.15.59:3300/0 conn(0x7f33d8071750 0x7f33d8154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047114673 0 0) 0x7f33d8072160 con 0x7f33d814e5f0 2024-03-30T01:43:33.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c40035b0 con 0x7f33d814e5f0 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 460465888 0 0) 0x7f33c40035b0 con 0x7f33d814e5f0 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 >> v2:172.21.15.59:3300/0 conn(0x7f33d8071750 msgr2=0x7f33d8154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33cffff700 1 --2- 172.21.15.31:0/2166032680 >> v2:172.21.15.59:3300/0 conn(0x7f33d8071750 0x7f33d8154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.565+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 >> v1:172.21.15.59:6789/0 conn(0x7f33d8071240 legacy=0x7f33d8154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80723b0 con 0x7f33d814e5f0 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7f33d4001670 con 0x7f33d814e5f0 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 >> v1:172.21.15.31:6789/0 conn(0x7f33d814e5f0 legacy=0x7f33d80c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c4003a00 0x7f33c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33c4004850 0x7f33c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c4005330 0x7f33c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c2e90 con 0x7f33c4003a00 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c400c200 con 0x7f33c4005330 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c400c3c0 con 0x7f33c4004850 2024-03-30T01:43:33.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dd26a700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c4003a00 0x7f33c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dca69700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33c4004850 0x7f33c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dda6b700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c4005330 0x7f33c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dd26a700 1 -- 172.21.15.31:0/2166032680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33c4004850 msgr2=0x7f33c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dd26a700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33c4004850 0x7f33c4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dd26a700 1 -- 172.21.15.31:0/2166032680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c4005330 msgr2=0x7f33c400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dd26a700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33c4005330 0x7f33c400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dda6b700 1 -- 172.21.15.31:0/2166032680 reap_dead start 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dd26a700 1 -- 172.21.15.31:0/2166032680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c40035b0 con 0x7f33c4003a00 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33dd26a700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c4003a00 0x7f33c4003e20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f33d400c210 tx=0x7f33c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de776f1f6c66b24a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4007c10 con 0x7f33c4003a00 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d400cd90 con 0x7f33c4003a00 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.566+0000 7f33cffff700 1 -- 172.21.15.31:0/2166032680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4005ba0 con 0x7f33c4003a00 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.567+0000 7f33df4ce700 1 -- 172.21.15.31:0/2166032680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c4003a00 msgr2=0x7f33c4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.567+0000 7f33df4ce700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c4003a00 0x7f33c4003e20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f33d400c210 tx=0x7f33c4005880 comp rx=0 tx=0).stop 2024-03-30T01:43:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.567+0000 7f33df4ce700 1 -- 172.21.15.31:0/2166032680 shutdown_connections 2024-03-30T01:43:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.567+0000 7f33df4ce700 1 --2- 172.21.15.31:0/2166032680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c4003a00 0x7f33c4003e20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.567+0000 7f33df4ce700 1 -- 172.21.15.31:0/2166032680 >> 172.21.15.31:0/2166032680 conn(0x7f33d806ba20 msgr2=0x7f33d806be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.567+0000 7f33df4ce700 1 -- 172.21.15.31:0/2166032680 shutdown_connections 2024-03-30T01:43:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.567+0000 7f33df4ce700 1 -- 172.21.15.31:0/2166032680 wait complete. 2024-03-30T01:43:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.568+0000 7f33df4ce700 1 Processor -- start 2024-03-30T01:43:33.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.568+0000 7f33df4ce700 1 -- start start 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33df4ce700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33c4003a00 0x7f33d8142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33df4ce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d8140f30 0x7f33d81580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33df4ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33d81585f0 0x7f33d81ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33df4ce700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8144310 con 0x7f33d81585f0 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33df4ce700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d81559f0 con 0x7f33d8140f30 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33df4ce700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8157a10 con 0x7f33c4003a00 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33dda6b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33d81585f0 0x7f33d81ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33dda6b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33d81585f0 0x7f33d81ed0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48884/0 (socket says 172.21.15.31:48884) 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33dda6b700 1 -- 172.21.15.31:0/1122574264 learned_addr learned my addr 172.21.15.31:0/1122574264 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33dd26a700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33c4003a00 0x7f33d8142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.569+0000 7f33dca69700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d8140f30 0x7f33d81580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33dda6b700 1 -- 172.21.15.31:0/1122574264 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33c4003a00 msgr2=0x7f33d8142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33dda6b700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33c4003a00 0x7f33d8142060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33dda6b700 1 -- 172.21.15.31:0/1122574264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d8140f30 msgr2=0x7f33d81580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33dda6b700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d8140f30 0x7f33d81580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33dda6b700 1 -- 172.21.15.31:0/1122574264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8143800 con 0x7f33d81585f0 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33dda6b700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33d81585f0 0x7f33d81ed0b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f33d80c1a90 tx=0x7f33d00111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33cdffb700 1 -- 172.21.15.31:0/1122574264 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d000e070 con 0x7f33d81585f0 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33cdffb700 1 -- 172.21.15.31:0/1122574264 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0003cf0 con 0x7f33d81585f0 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d8143d50 con 0x7f33d81585f0 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33cdffb700 1 -- 172.21.15.31:0/1122574264 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00117d0 con 0x7f33d81585f0 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.570+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d81576a0 con 0x7f33d81585f0 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.571+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d804ea80 con 0x7f33d81585f0 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.572+0000 7f33cdffb700 1 -- 172.21.15.31:0/1122574264 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f33d0011930 con 0x7f33d81585f0 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.572+0000 7f33cdffb700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f33bc06ac90 0x7f33bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:33.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.572+0000 7f33cdffb700 1 -- 172.21.15.31:0/1122574264 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f33d0089ba0 con 0x7f33d81585f0 2024-03-30T01:43:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.572+0000 7f33dd26a700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f33bc06ac90 0x7f33bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.573+0000 7f33dd26a700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f33bc06ac90 0x7f33bc06d150 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.577+0000 7f33cdffb700 1 -- 172.21.15.31:0/1122574264 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f33d0054970 con 0x7f33d81585f0 2024-03-30T01:43:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.840+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f33d8157b50 con 0x7f33d81585f0 2024-03-30T01:43:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.840+0000 7f33cdffb700 1 -- 172.21.15.31:0/1122574264 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) v1 ==== 53+0+149 (secure 0 0 0) 0x7f33d0014350 con 0x7f33d81585f0 2024-03-30T01:43:33.846 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T01:43:33.846 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-30T01:43:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.842+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f33bc06ac90 msgr2=0x7f33bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.842+0000 7f33df4ce700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f33bc06ac90 0x7f33bc06d150 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4007000 comp rx=0 tx=0).stop 2024-03-30T01:43:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.842+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33d81585f0 msgr2=0x7f33d81ed0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:33.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.843+0000 7f33df4ce700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33d81585f0 0x7f33d81ed0b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f33d80c1a90 tx=0x7f33d00111d0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.843+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 shutdown_connections 2024-03-30T01:43:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.843+0000 7f33df4ce700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33d81585f0 0x7f33d81ed0b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.843+0000 7f33df4ce700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f33bc06ac90 0x7f33bc06d150 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.843+0000 7f33df4ce700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33c4003a00 0x7f33d8142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.843+0000 7f33df4ce700 1 --2- 172.21.15.31:0/1122574264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33d8140f30 0x7f33d81580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.843+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 >> 172.21.15.31:0/1122574264 conn(0x7f33d806ba20 msgr2=0x7f33d806e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.843+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 shutdown_connections 2024-03-30T01:43:33.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:33.843+0000 7f33df4ce700 1 -- 172.21.15.31:0/1122574264 wait complete. 2024-03-30T01:43:33.863 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-03-30T01:43:33.863 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T01:43:33.863 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-03-30T01:43:33.863 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T01:43:33.863 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T01:43:33.863 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-03-30T01:43:33.863 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-30T01:43:33.863 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T01:43:33.863 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:43:33.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:33.864 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-03-30T01:43:33.886 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:33.887 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether e6:5a:8b:06:76:44 brd ff:ff:ff:ff:ff:ff 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::5055:d3ff:fee5:dc2a/64 scope link 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:33.888 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T01:43:33.889 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether e6:5a:8b:06:76:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T01:43:33.889 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::e45a:8bff:fe06:7644/64 scope link 2024-03-30T01:43:33.889 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-03-30T01:43:33.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:33.889 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T01:43:33.960 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-03-30T01:43:33.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:33.962 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-03-30T01:43:34.026 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-03-30T01:43:34.026 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T01:43:34.026 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-30T01:43:34.026 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-30T01:43:34.026 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T01:43:34.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:34.027 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2024-03-30T01:43:34.027 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-03-30T01:43:34.027 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T01:43:34.027 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T01:43:34.027 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T01:43:34.027 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-03-30T01:43:34.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:34.150 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-03-30T01:43:34.169 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T01:43:34.170 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:43:34.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:34.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-03-30T01:43:34.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:43:34.244 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T01:43:34.307 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T01:43:34.307 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T01:43:34.307 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:43:34.368 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-30T01:43:34.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:34.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs2 2024-03-30T01:43:34.465 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs2 2024-03-30T01:43:34.466 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs2". 2024-03-30T01:43:34.466 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:43:34.476 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs2 2024-03-30T01:43:34.476 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs2". 2024-03-30T01:43:34.476 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:43:34.476 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:43:34.476 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:43:34.495 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs2 2024-03-30T01:43:34.495 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs2". 2024-03-30T01:43:34.495 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-03-30T01:43:34.495 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.59:3300,172.21.15.59:3301" 2024-03-30T01:43:34.495 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-30T01:43:34.495 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790" 2024-03-30T01:43:34.495 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.59:6789,172.21.15.59:6790:/ 2024-03-30T01:43:34.495 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs2,key=1,fsid=9739890c-57ec-4503-9fd1-d4b0145023f4" will pass to kernel 2024-03-30T01:43:34.497 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T01:43:34.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:34.498 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T01:43:34.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T01:43:34.529 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T01:43:34.626 INFO:teuthology.orchestra.run.smithi059.stdout:client6112 2024-03-30T01:43:34.629 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-03-30T01:43:34.629 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T01:43:34.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth add client.testuser mon 'allow rw fsname=cephfs, allow rw fsname=cephfs2' osd 'allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2' mds 'allow rw fsname=cephfs' 2024-03-30T01:43:34.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.798+0000 7fcfbdbec700 1 Processor -- start 2024-03-30T01:43:34.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.798+0000 7fcfbdbec700 1 -- start start 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.799+0000 7fcfbdbec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfb8151c90 0x7fcfb806a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.799+0000 7fcfbdbec700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb8152070 con 0x7fcfb80c1250 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.799+0000 7fcfbdbec700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb80acef0 con 0x7fcfb806d340 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.799+0000 7fcfbdbec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb80acaa0 con 0x7fcfb8151c90 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.799+0000 7fcfb6ffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcfb80c1250 0x7fcfb8154680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54886/0 (socket says 172.21.15.31:54886) 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.799+0000 7fcfb6ffd700 1 -- 172.21.15.31:0/3408713060 learned_addr learned my addr 172.21.15.31:0/3408713060 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.799+0000 7fcfb77fe700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfb8151c90 0x7fcfb806a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324788469 0 0) 0x7fcfb8152070 con 0x7fcfb80c1250 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa40035b0 con 0x7fcfb80c1250 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622062632 0 0) 0x7fcfb80acef0 con 0x7fcfb806d340 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb8152070 con 0x7fcfb806d340 2024-03-30T01:43:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3320281742 0 0) 0x7fcfa40035b0 con 0x7fcfb80c1250 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfb8151c90 msgr2=0x7fcfb806a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfb8151c90 0x7fcfb806a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 >> v1:172.21.15.59:6789/0 conn(0x7fcfb806d340 legacy=0x7fcfb815efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80c31d0 con 0x7fcfb80c1250 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 542592426 0 0) 0x7fcfa0001670 con 0x7fcfb80c1250 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 >> v1:172.21.15.31:6789/0 conn(0x7fcfb80c1250 legacy=0x7fcfb8154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfa4003ca0 0x7fcfa40040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfa4004850 0x7fcfa4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa4005340 0x7fcfa400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb80acaa0 con 0x7fcfa4005340 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa400c250 con 0x7fcfa4004850 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa400c410 con 0x7fcfa4003ca0 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb77fe700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfb8151c90 0x7fcfb806a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T01:43:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb7fff700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa4005340 0x7fcfa400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb6ffd700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfa4004850 0x7fcfa4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.800+0000 7fcfb77fe700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfa4003ca0 0x7fcfa40040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb7fff700 1 -- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfa4003ca0 msgr2=0x7fcfa40040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb7fff700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfa4003ca0 0x7fcfa40040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb7fff700 1 -- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfa4004850 msgr2=0x7fcfa4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb7fff700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfa4004850 0x7fcfa4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb7fff700 1 -- 172.21.15.31:0/3408713060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40035b0 con 0x7fcfa4005340 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb7fff700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa4005340 0x7fcfa400ba10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c2f0 tx=0x7fcfa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b175eefea57f778 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0001f30 con 0x7fcfa4005340 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb000ce90 con 0x7fcfa4005340 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb67fc700 1 -- 172.21.15.31:0/3408713060 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0003c40 con 0x7fcfa4005340 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3408713060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa4005340 msgr2=0x7fcfa400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:34.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfbdbec700 1 --2- 172.21.15.31:0/3408713060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa4005340 0x7fcfa400ba10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c2f0 tx=0x7fcfa4005880 comp rx=0 tx=0).stop 2024-03-30T01:43:34.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfb7fff700 1 -- 172.21.15.31:0/3408713060 reap_dead start 2024-03-30T01:43:34.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3408713060 shutdown_connections 2024-03-30T01:43:34.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.801+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3408713060 >> 172.21.15.31:0/3408713060 conn(0x7fcfb80c3910 msgr2=0x7fcfb80c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:34.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.802+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3408713060 shutdown_connections 2024-03-30T01:43:34.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.802+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3408713060 wait complete. 2024-03-30T01:43:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.803+0000 7fcfbdbec700 1 Processor -- start 2024-03-30T01:43:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.804+0000 7fcfbdbec700 1 -- start start 2024-03-30T01:43:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.804+0000 7fcfbdbec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfa4005340 0x7fcfb8158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.804+0000 7fcfbdbec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb80bbf30 0x7fcfb814ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.804+0000 7fcfbdbec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfb814b720 0x7fcfb814bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.804+0000 7fcfbdbec700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb8157d70 con 0x7fcfb80bbf30 2024-03-30T01:43:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfbdbec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb80be420 con 0x7fcfb814b720 2024-03-30T01:43:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfbdbec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb814c0e0 con 0x7fcfa4005340 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb80bbf30 0x7fcfb814ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb80bbf30 0x7fcfb814ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48900/0 (socket says 172.21.15.31:48900) 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb6ffd700 1 -- 172.21.15.31:0/3661248443 learned_addr learned my addr 172.21.15.31:0/3661248443 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb7fff700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfb814b720 0x7fcfb814bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb77fe700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfa4005340 0x7fcfb8158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb6ffd700 1 -- 172.21.15.31:0/3661248443 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfa4005340 msgr2=0x7fcfb8158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb6ffd700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfa4005340 0x7fcfb8158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb6ffd700 1 -- 172.21.15.31:0/3661248443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfb814b720 msgr2=0x7fcfb814bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb6ffd700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfb814b720 0x7fcfb814bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb6ffd700 1 -- 172.21.15.31:0/3661248443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80bc4a0 con 0x7fcfb80bbf30 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb7fff700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfb814b720 0x7fcfb814bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfb6ffd700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb80bbf30 0x7fcfb814ddc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0004f80 tx=0x7fcfa00073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfaffff700 1 -- 172.21.15.31:0/3661248443 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0011580 con 0x7fcfb80bbf30 2024-03-30T01:43:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfaffff700 1 -- 172.21.15.31:0/3661248443 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa0011700 con 0x7fcfb80bbf30 2024-03-30T01:43:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb81573d0 con 0x7fcfb80bbf30 2024-03-30T01:43:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfaffff700 1 -- 172.21.15.31:0/3661248443 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0011ab0 con 0x7fcfb80bbf30 2024-03-30T01:43:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.805+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb8157990 con 0x7fcfb80bbf30 2024-03-30T01:43:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.806+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb8157b50 con 0x7fcfb80bbf30 2024-03-30T01:43:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.807+0000 7fcfaffff700 1 -- 172.21.15.31:0/3661248443 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcfa0011c10 con 0x7fcfb80bbf30 2024-03-30T01:43:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.807+0000 7fcfaffff700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9806ac90 0x7fcf9806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.807+0000 7fcfaffff700 1 -- 172.21.15.31:0/3661248443 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7fcfa00896c0 con 0x7fcfb80bbf30 2024-03-30T01:43:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.807+0000 7fcfb77fe700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9806ac90 0x7fcf9806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.808+0000 7fcfb77fe700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9806ac90 0x7fcf9806d150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80c7eb0 tx=0x7fcfa8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:34.812+0000 7fcfaffff700 1 -- 172.21.15.31:0/3661248443 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcfa0054400 con 0x7fcfb80bbf30 2024-03-30T01:43:35.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.117+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow rw fsname=cephfs, allow rw fsname=cephfs2", "osd", "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2", "mds", "allow rw fsname=cephfs"]} v 0) v1 -- 0x7fcfb8066f90 con 0x7fcfb80bbf30 2024-03-30T01:43:35.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.121+0000 7fcfaffff700 1 -- 172.21.15.31:0/3661248443 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow rw fsname=cephfs, allow rw fsname=cephfs2", "osd", "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2", "mds", "allow rw fsname=cephfs"]}]=0 added key for client.testuser v19) v1 ==== 291+0+0 (secure 0 0 0) 0x7fcfa0057a20 con 0x7fcfb80bbf30 2024-03-30T01:43:35.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.123+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9806ac90 msgr2=0x7fcf9806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:35.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.123+0000 7fcfbdbec700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9806ac90 0x7fcf9806d150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80c7eb0 tx=0x7fcfa8008040 comp rx=0 tx=0).stop 2024-03-30T01:43:35.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.123+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb80bbf30 msgr2=0x7fcfb814ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:35.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.123+0000 7fcfbdbec700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb80bbf30 0x7fcfb814ddc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0004f80 tx=0x7fcfa00073f0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.124+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 shutdown_connections 2024-03-30T01:43:35.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.124+0000 7fcfbdbec700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7fcf9806ac90 0x7fcf9806d150 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.124+0000 7fcfbdbec700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfb814b720 0x7fcfb814bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.124+0000 7fcfbdbec700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcfa4005340 0x7fcfb8158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.124+0000 7fcfbdbec700 1 --2- 172.21.15.31:0/3661248443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb80bbf30 0x7fcfb814ddc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.124+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 >> 172.21.15.31:0/3661248443 conn(0x7fcfb80c3910 msgr2=0x7fcfb814fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:35.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.124+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 shutdown_connections 2024-03-30T01:43:35.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.124+0000 7fcfbdbec700 1 -- 172.21.15.31:0/3661248443 wait complete. 2024-03-30T01:43:35.130 INFO:teuthology.orchestra.run.smithi031.stderr:added key for client.testuser 2024-03-30T01:43:35.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.testuser 2024-03-30T01:43:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.274+0000 7f2970263700 1 Processor -- start 2024-03-30T01:43:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.275+0000 7f2970263700 1 -- start start 2024-03-30T01:43:35.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.275+0000 7f2970263700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f29680ac520 0x7f29680ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:35.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.275+0000 7f2970263700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29680a74e0 0x7f29680ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:35.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.275+0000 7f2970263700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2968153590 con 0x7f29680c6000 2024-03-30T01:43:35.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.275+0000 7f2970263700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29680ab680 con 0x7f29680a74e0 2024-03-30T01:43:35.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.275+0000 7f2970263700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29681500c0 con 0x7f29680ac520 2024-03-30T01:43:35.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296d7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29680a74e0 0x7f29680ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:35.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296e800700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f29680c6000 0x7f29680c6490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54896/0 (socket says 172.21.15.31:54896) 2024-03-30T01:43:35.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296e800700 1 -- 172.21.15.31:0/3345703024 learned_addr learned my addr 172.21.15.31:0/3345703024 (peer_addr_for_me v1:172.21.15.31:0/0) 2024-03-30T01:43:35.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296dfff700 1 --2- 172.21.15.31:0/3345703024 >> v2:172.21.15.59:3300/0 conn(0x7f29680ac520 0x7f29680ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:35.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296d7fe700 1 -- 172.21.15.31:0/3345703024 >> v2:172.21.15.59:3300/0 conn(0x7f29680ac520 msgr2=0x7f29680ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T01:43:35.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296d7fe700 1 --2- 172.21.15.31:0/3345703024 >> v2:172.21.15.59:3300/0 conn(0x7f29680ac520 0x7f29680ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296d7fe700 1 -- 172.21.15.31:0/3345703024 >> v1:172.21.15.31:6789/0 conn(0x7f29680c6000 legacy=0x7f29680c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:35.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296d7fe700 1 -- 172.21.15.31:0/3345703024 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29680c7990 con 0x7f29680a74e0 2024-03-30T01:43:35.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296cffd700 1 -- 172.21.15.31:0/3345703024 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153430887 0 0) 0x7f2968153590 con 0x7f29680c6000 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296d7fe700 1 --2- 172.21.15.31:0/3345703024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680a74e0 0x7f29680ace40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f295c009840 tx=0x7f295c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b44dd12bf090e87 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296cffd700 1 -- 172.21.15.31:0/3345703024 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c011830 con 0x7f29680a74e0 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296cffd700 1 -- 172.21.15.31:0/3345703024 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c011990 con 0x7f29680a74e0 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.276+0000 7f296cffd700 1 -- 172.21.15.31:0/3345703024 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c011d20 con 0x7f29680a74e0 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.277+0000 7f2970263700 1 -- 172.21.15.31:0/3345703024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680a74e0 msgr2=0x7f29680ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.277+0000 7f2970263700 1 --2- 172.21.15.31:0/3345703024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680a74e0 0x7f29680ace40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f295c009840 tx=0x7f295c009b50 comp rx=0 tx=0).stop 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.277+0000 7f2970263700 1 -- 172.21.15.31:0/3345703024 shutdown_connections 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.277+0000 7f2970263700 1 --2- 172.21.15.31:0/3345703024 >> v2:172.21.15.59:3300/0 conn(0x7f29680ac520 0x7f29680ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.277+0000 7f2970263700 1 --2- 172.21.15.31:0/3345703024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680a74e0 0x7f29680ace40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.277+0000 7f2970263700 1 -- 172.21.15.31:0/3345703024 >> 172.21.15.31:0/3345703024 conn(0x7f296806ad30 msgr2=0x7f29680a9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:35.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.277+0000 7f2970263700 1 -- 172.21.15.31:0/3345703024 shutdown_connections 2024-03-30T01:43:35.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.277+0000 7f2970263700 1 -- 172.21.15.31:0/3345703024 wait complete. 2024-03-30T01:43:35.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.279+0000 7f2970263700 1 Processor -- start 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.279+0000 7f2970263700 1 -- start start 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f2970263700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29680a74e0 0x7f29680bd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f2970263700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680ac520 0x7f29680a8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f2970263700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29680c6000 0x7f29681e8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f2970263700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29681ec7d0 con 0x7f29680ac520 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f2970263700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29681eba10 con 0x7f29680c6000 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f2970263700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29681ebcf0 con 0x7f29680a74e0 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296d7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680ac520 0x7f29680a8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296d7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680ac520 0x7f29680a8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48922/0 (socket says 172.21.15.31:48922) 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296d7fe700 1 -- 172.21.15.31:0/2997877941 learned_addr learned my addr 172.21.15.31:0/2997877941 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296dfff700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29680a74e0 0x7f29680bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296e800700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29680c6000 0x7f29681e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:35.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296d7fe700 1 -- 172.21.15.31:0/2997877941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29680a74e0 msgr2=0x7f29680bd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296d7fe700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29680a74e0 0x7f29680bd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296d7fe700 1 -- 172.21.15.31:0/2997877941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29680c6000 msgr2=0x7f29681e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296d7fe700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29680c6000 0x7f29681e8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.280+0000 7f296d7fe700 1 -- 172.21.15.31:0/2997877941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c0094a0 con 0x7f29680ac520 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.281+0000 7f296d7fe700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680ac520 0x7f29680a8c90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f295c013040 tx=0x7f295c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.281+0000 7f295a7fc700 1 -- 172.21.15.31:0/2997877941 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c0036a0 con 0x7f29680ac520 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.281+0000 7f295a7fc700 1 -- 172.21.15.31:0/2997877941 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c01e070 con 0x7f29680ac520 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.281+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29681ebfd0 con 0x7f29680ac520 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.281+0000 7f295a7fc700 1 -- 172.21.15.31:0/2997877941 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c003a20 con 0x7f29680ac520 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.281+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29680bdc60 con 0x7f29680ac520 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.282+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29681ec4f0 con 0x7f29680ac520 2024-03-30T01:43:35.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.283+0000 7f295a7fc700 1 -- 172.21.15.31:0/2997877941 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f295c003b80 con 0x7f29680ac520 2024-03-30T01:43:35.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.283+0000 7f295a7fc700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f294c06ac90 0x7f294c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T01:43:35.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.283+0000 7f295a7fc700 1 -- 172.21.15.31:0/2997877941 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6885+0+0 (secure 0 0 0) 0x7f295c08a590 con 0x7f29680ac520 2024-03-30T01:43:35.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.283+0000 7f296dfff700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f294c06ac90 0x7f294c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T01:43:35.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.284+0000 7f296dfff700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f294c06ac90 0x7f294c06d150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2964009be0 tx=0x7f2964009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T01:43:35.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.288+0000 7f295a7fc700 1 -- 172.21.15.31:0/2997877941 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f29681ec4f0 con 0x7f29680ac520 2024-03-30T01:43:35.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.588+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.testuser"} v 0) v1 -- 0x7f29681ec4f0 con 0x7f29680ac520 2024-03-30T01:43:35.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.589+0000 7f295a7fc700 1 -- 172.21.15.31:0/2997877941 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.testuser"}]=0 v19) v1 ==== 85+0+245 (secure 0 0 0) 0x7f29681ec4f0 con 0x7f29680ac520 2024-03-30T01:43:35.594 INFO:teuthology.orchestra.run.smithi031.stdout:[client.testuser] 2024-03-30T01:43:35.594 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQBHbgdmT9MMBxAA8XsX0kZRBOJGg0GGgSp0yw== 2024-03-30T01:43:35.594 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow rw fsname=cephfs" 2024-03-30T01:43:35.595 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow rw fsname=cephfs, allow rw fsname=cephfs2" 2024-03-30T01:43:35.595 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2" 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f294c06ac90 msgr2=0x7f294c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f294c06ac90 0x7f294c06d150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2964009be0 tx=0x7f2964009210 comp rx=0 tx=0).stop 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680ac520 msgr2=0x7f29680a8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680ac520 0x7f29680a8c90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f295c013040 tx=0x7f295c004750 comp rx=0 tx=0).stop 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 shutdown_connections 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:6832/47081,v1:172.21.15.31:6833/47081] conn(0x7f294c06ac90 0x7f294c06d150 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29680c6000 0x7f29681e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29680a74e0 0x7f29680bd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 --2- 172.21.15.31:0/2997877941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29680ac520 0x7f29680a8c90 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 >> 172.21.15.31:0/2997877941 conn(0x7f296806ad30 msgr2=0x7f296806f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 shutdown_connections 2024-03-30T01:43:35.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-03-30T01:43:35.591+0000 7f2970263700 1 -- 172.21.15.31:0/2997877941 wait complete. 2024-03-30T01:43:35.610 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-03-30T01:43:35.629 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.lsURkNfsPU 2024-03-30T01:43:35.629 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-03-30T01:43:35.629 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/tmp/tmp.lsURkNfsPU 2024-03-30T01:43:35.672 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 309, in copy_and_close shutil.copyfileobj(src, fdst) File "/usr/lib/python3.8/shutil.py", line 202, in copyfileobj fsrc_read = fsrc.read AttributeError: 'RemoteProcess' object has no attribute 'read' 2024-03-30T01:43:58.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:43:58.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:44:28.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:44:28.594 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:44:58.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:44:58.667 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:45:28.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:45:28.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:45:58.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:45:58.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:46:28.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:46:28.912 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:46:58.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:46:58.985 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:47:29.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:47:29.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:47:59.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:47:59.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:48:29.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:48:29.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:48:59.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:48:59.310 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:49:29.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:49:29.380 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:49:59.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:49:59.452 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:50:29.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:50:29.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:50:59.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:50:59.606 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:51:29.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:51:29.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:51:59.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:51:59.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:52:29.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:52:29.858 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:53:00.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:53:00.056 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:53:30.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:53:30.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:54:00.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:54:00.233 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:54:30.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:54:30.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:55:00.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:55:00.470 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:55:30.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:55:30.550 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:56:00.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:56:00.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:56:30.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:56:30.832 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:57:00.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:57:00.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:57:30.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:57:30.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:58:01.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:58:01.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:58:31.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:58:31.344 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:59:01.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:59:01.417 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:59:31.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T01:59:31.490 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:00:01.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:00:01.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:00:31.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:00:31.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:01:01.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:01:01.738 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:01:31.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:01:31.813 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:02:01.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:02:01.890 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:02:31.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:02:31.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:03:02.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:03:02.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:03:32.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:03:32.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:04:02.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:04:02.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:04:32.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:04:32.278 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:05:02.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:05:02.345 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:05:32.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:05:32.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:06:02.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:06:02.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:06:32.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:06:32.638 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:07:02.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:07:02.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:07:32.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:07:32.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:08:02.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:08:02.862 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:08:32.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:08:32.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:09:03.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:09:03.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:09:33.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:09:33.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:10:03.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:10:03.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:10:33.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:10:33.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:11:03.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:11:03.324 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:11:33.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:11:33.492 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:12:03.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:12:03.562 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:12:33.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:12:33.635 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:13:03.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:13:03.715 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:13:33.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:13:33.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:14:03.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:14:03.861 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:14:34.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:14:34.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:15:04.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:15:04.123 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:15:34.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:15:34.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:16:04.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:16:04.285 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:16:34.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:16:34.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:17:04.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:17:04.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:17:34.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:17:34.495 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:18:04.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:18:04.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:18:34.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:18:34.665 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:19:04.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:19:04.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:19:34.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:19:34.826 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:20:04.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:20:04.900 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:20:35.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:20:35.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:21:05.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:21:05.172 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:21:35.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:21:35.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:22:05.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:22:05.340 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:22:35.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:22:35.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:23:05.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:23:05.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:23:35.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:23:35.544 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:24:05.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:24:05.616 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:24:35.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:24:35.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:25:05.782 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:25:05.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:25:35.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:25:35.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:26:05.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:26:05.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:26:36.044 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:26:36.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:27:06.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:27:06.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:27:36.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:27:36.193 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:28:06.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:28:06.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:28:36.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:28:36.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:29:06.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:29:06.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:29:36.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:29:36.524 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:30:06.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:30:06.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:30:36.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:30:36.768 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:31:06.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:31:06.841 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:31:36.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:31:36.930 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:32:07.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:32:07.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:32:37.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:32:37.076 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:33:07.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:33:07.148 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:33:37.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:33:37.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:34:07.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:34:07.306 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:34:37.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:34:37.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:35:07.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:35:07.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:35:37.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:35:37.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:36:07.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:36:07.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:36:37.718 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:36:37.722 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:37:07.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:37:07.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:37:37.898 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:37:37.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:38:07.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:38:07.982 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:38:38.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:38:38.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:39:08.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:39:08.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:39:38.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:39:38.219 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:40:08.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:40:08.315 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:40:38.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:40:38.413 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:41:08.482 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:41:08.485 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:41:38.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:41:38.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:42:08.644 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:42:08.647 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:42:38.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:42:38.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:43:08.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:43:08.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:43:38.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:43:38.995 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:44:09.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:44:09.066 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:44:39.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:44:39.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:45:09.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:45:09.214 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:45:39.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:45:39.288 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:46:09.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:46:09.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:46:39.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:46:39.433 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:47:09.503 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:47:09.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:47:39.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:47:39.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:48:09.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:48:09.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:48:39.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:48:39.733 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:49:09.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:49:09.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:49:39.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:49:39.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:50:09.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:50:09.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:50:40.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:50:40.043 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:51:10.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:51:10.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:51:40.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:51:40.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:52:10.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:52:10.298 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:52:40.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:52:40.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:53:10.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:53:10.458 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:53:40.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:53:40.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:54:10.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:54:10.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:54:40.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:54:40.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:55:10.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:55:10.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:55:40.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:55:40.841 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:56:10.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:56:10.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:56:41.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:56:41.099 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:57:11.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:57:11.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:57:41.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:57:41.264 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:58:11.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:58:11.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:58:41.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:58:41.428 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:59:11.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:59:11.500 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:59:41.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T02:59:41.588 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:00:11.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:00:11.655 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:00:41.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:00:41.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:01:11.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:01:11.797 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:01:41.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:01:41.988 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:02:12.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:02:12.076 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:02:42.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:02:42.150 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:03:12.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:03:12.223 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:03:42.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:03:42.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:04:12.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:04:12.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:04:42.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:04:42.449 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:05:12.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:05:12.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:05:42.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:05:42.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:06:12.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:06:12.658 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:06:42.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:06:42.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:07:12.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:07:12.812 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:07:42.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:07:42.904 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:08:12.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:08:12.982 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:08:43.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:08:43.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:09:13.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:09:13.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:09:43.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:09:43.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:10:13.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:10:13.247 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:10:43.316 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:10:43.319 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:11:13.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:11:13.412 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:11:43.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:11:43.488 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:12:13.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:12:13.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:12:43.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:12:43.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:13:13.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:13:13.732 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:13:43.798 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:13:43.984 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:14:14.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:14:14.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:14:44.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:14:44.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:15:14.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:15:14.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:15:44.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:15:44.269 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:16:14.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:16:14.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:16:44.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:16:44.455 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:17:14.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:17:14.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:17:44.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:17:44.619 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:18:14.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:18:14.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:18:44.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:18:44.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:19:14.831 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:19:14.835 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:19:44.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:19:44.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:20:15.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:20:15.023 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:20:45.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:20:45.096 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:21:15.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:21:15.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:21:45.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:21:45.241 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:22:15.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:22:15.313 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:22:45.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:22:45.402 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:23:15.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:23:15.480 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:23:45.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:23:45.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:24:15.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:24:15.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:24:45.822 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:24:45.826 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:25:15.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:25:15.907 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:25:45.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:25:45.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:26:16.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:26:16.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:26:46.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:26:46.241 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:27:16.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:27:16.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:27:46.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:27:46.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:28:16.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:28:16.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:28:46.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:28:46.548 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:29:16.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:29:16.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:29:46.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:29:46.732 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:30:16.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:30:16.807 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:30:46.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:30:46.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:31:16.942 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:31:16.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:31:47.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:31:47.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:32:17.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:32:17.116 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:32:47.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:32:47.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:33:17.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:33:17.285 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:33:47.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:33:47.357 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:34:17.438 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:34:17.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:34:47.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:34:47.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:35:17.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:35:17.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:35:47.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:35:47.688 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:36:17.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:36:17.760 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:36:47.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:36:47.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:37:17.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:37:17.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:37:47.981 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:37:47.984 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:38:18.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:38:18.069 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:38:48.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:38:48.142 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:39:18.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:39:18.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:39:48.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:39:48.279 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:40:18.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:40:18.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:40:48.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:40:48.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:41:18.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:41:18.504 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:41:48.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:41:48.581 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:42:18.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:42:18.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:42:48.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:42:48.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:43:18.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:43:18.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:43:48.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:43:48.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:44:18.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:44:18.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:44:49.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:44:49.045 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:45:19.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:45:19.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:45:49.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:45:49.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:46:19.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:46:19.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:46:49.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:46:49.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:47:19.482 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:47:19.485 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:47:49.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:47:49.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:48:19.750 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:48:19.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:48:49.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:48:49.836 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:49:19.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:49:19.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:49:49.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:49:49.999 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:50:20.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:50:20.094 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:50:50.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:50:50.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:51:20.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:51:20.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:51:50.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:51:50.317 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:52:20.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:52:20.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:52:50.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:52:50.470 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:53:20.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:53:20.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:53:50.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:53:50.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:54:20.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:54:20.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:54:50.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:54:50.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:55:20.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:55:20.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:55:51.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:55:51.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:56:21.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:56:21.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:56:51.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:56:51.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:57:21.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:57:21.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:57:51.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:57:51.484 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:58:21.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:58:21.547 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:58:51.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:58:51.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:59:21.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:59:21.704 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:59:51.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T03:59:51.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:00:21.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:00:21.889 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:00:51.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:00:51.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:01:22.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:01:22.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:01:52.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:01:52.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:02:22.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:02:22.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:02:52.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:02:52.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:03:22.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:03:22.471 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:03:52.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:03:52.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:04:22.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:04:22.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:04:52.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:04:52.710 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:05:22.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:05:22.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:05:52.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:05:52.880 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:06:22.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:06:22.952 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:06:53.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:06:53.025 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:07:23.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:07:23.100 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:07:53.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:07:53.178 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:08:23.248 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:08:23.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:08:53.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:08:53.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:09:23.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:09:23.429 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:09:53.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:09:53.501 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:10:23.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:10:23.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:10:53.668 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:10:53.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:11:23.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:11:23.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:11:53.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:11:53.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:12:23.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:12:23.891 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:12:53.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:12:53.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:13:24.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:13:24.043 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:13:54.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:13:54.145 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:14:24.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:14:24.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:14:54.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:14:54.302 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:15:24.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:15:24.394 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:15:54.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:15:54.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:16:24.666 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:16:24.670 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:16:54.762 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:16:54.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:17:24.837 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:17:24.842 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:17:54.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:17:54.904 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:18:24.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:18:24.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:18:55.055 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:18:55.059 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:19:25.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:19:25.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:19:55.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:19:55.204 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:20:25.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:20:25.283 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:20:55.353 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:20:55.356 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:21:25.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:21:25.433 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:21:55.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:21:55.504 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:22:25.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:22:25.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:22:55.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:22:55.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:23:25.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:23:25.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:23:55.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:23:55.810 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:24:25.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:24:25.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:24:55.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:24:55.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:25:26.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:25:26.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:25:56.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:25:56.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:26:26.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:26:26.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:26:56.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:26:56.578 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:27:26.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:27:26.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:27:56.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:27:56.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:28:26.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:28:26.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:28:56.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:28:56.885 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:29:26.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:29:26.985 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:29:57.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:29:57.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:30:27.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:30:27.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:30:57.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:30:57.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:31:27.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:31:27.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:31:57.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:31:57.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:32:27.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:32:27.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:32:57.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:32:57.618 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:33:27.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:33:27.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:33:57.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:33:57.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:34:27.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:34:27.888 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:34:57.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:34:57.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:35:28.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:35:28.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:35:58.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:35:58.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:36:28.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:36:28.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:36:58.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:36:58.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:37:28.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:37:28.357 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:37:58.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:37:58.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:38:28.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:38:28.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:38:58.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:38:58.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:39:28.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:39:28.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:39:58.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:39:58.895 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:40:28.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:40:28.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:40:59.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:40:59.066 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:41:29.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:41:29.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:41:59.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:41:59.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:42:29.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:42:29.276 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:42:59.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:42:59.373 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:43:29.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:43:29.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:43:59.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:43:59.534 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:44:29.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:44:29.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:44:59.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:44:59.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:45:29.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:45:29.877 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:45:59.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:45:59.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:46:30.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:46:30.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:47:00.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:47:00.131 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:47:30.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:47:30.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:48:00.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:48:00.298 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:48:30.367 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:48:30.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:49:00.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:49:00.453 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:49:30.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:49:30.531 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:50:00.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:50:00.612 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:50:30.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:50:30.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:51:00.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:51:00.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:51:30.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:51:30.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:52:00.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:52:01.003 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:52:31.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:52:31.085 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:53:01.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:53:01.164 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:53:31.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:53:31.245 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:54:01.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:54:01.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:54:31.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:54:31.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:55:01.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:55:01.496 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:55:31.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:55:31.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:56:01.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:56:01.642 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:56:31.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:56:31.721 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:57:01.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:57:01.796 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:57:31.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:57:31.872 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:58:01.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:58:01.949 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:58:32.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:58:32.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:59:02.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:59:02.111 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:59:32.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T04:59:32.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:00:02.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:00:02.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:00:32.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:00:32.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:01:02.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:01:02.415 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:01:32.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:01:32.516 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:02:02.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:02:02.599 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:02:32.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:02:32.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:03:02.763 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:03:02.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:03:32.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:03:32.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:04:02.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:04:02.920 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:04:33.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:04:33.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:05:03.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:05:03.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:05:33.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:05:33.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:06:03.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:06:03.293 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:06:33.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:06:33.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:07:03.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:07:03.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:07:33.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:07:33.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:08:03.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:08:03.592 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:08:33.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:08:33.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:09:03.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:09:03.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:09:33.812 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:09:33.815 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:10:03.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:10:03.882 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:10:33.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:10:33.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:11:04.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:11:04.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:11:34.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:11:34.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:12:04.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:12:04.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:12:34.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:12:34.268 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:13:04.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:13:04.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:13:34.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:13:34.439 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:14:04.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:14:04.508 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:14:34.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:14:34.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:15:04.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:15:04.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:15:34.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:15:34.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:16:04.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:16:04.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:16:34.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:16:34.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:17:04.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:17:04.986 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:17:35.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:17:35.065 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:18:05.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:18:05.127 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:18:35.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:18:35.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:19:05.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:19:05.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:19:35.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:19:35.354 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:20:05.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:20:05.444 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:20:35.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:20:35.543 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:21:05.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:21:05.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:21:35.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:21:35.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:22:05.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:22:05.759 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:22:35.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:22:35.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:23:05.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:23:05.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:23:35.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:23:36.002 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:24:06.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:24:06.068 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:24:36.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:24:36.148 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:25:06.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:25:06.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:25:36.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:25:36.332 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:26:06.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:26:06.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:26:36.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:26:36.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:27:06.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:27:06.547 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:27:36.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:27:36.627 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:28:06.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:28:06.716 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:28:36.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:28:36.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:29:06.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:29:06.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:29:37.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:29:37.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:30:07.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:30:07.121 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:30:37.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:30:37.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:31:07.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:31:07.308 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:31:37.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:31:37.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:32:07.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:32:07.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:32:37.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:32:37.568 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:33:07.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:33:07.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:33:37.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:33:37.709 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:34:07.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:34:07.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:34:37.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:34:37.874 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:35:08.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:35:08.014 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:35:38.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:35:38.101 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:36:08.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:36:08.168 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:36:38.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:36:38.247 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:37:08.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:37:08.325 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:37:38.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:37:38.402 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:38:08.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:38:08.485 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:38:38.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:38:38.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:39:08.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:39:08.635 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:39:38.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:39:38.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:40:08.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:40:08.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:40:38.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:40:38.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:41:08.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:41:08.976 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:41:39.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:41:39.043 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:42:09.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:42:09.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:42:39.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:42:39.204 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:43:09.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:43:09.283 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:43:39.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:43:39.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:44:09.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:44:09.468 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:44:39.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:44:39.550 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:45:09.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:45:09.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:45:39.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:45:39.698 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:46:09.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:46:09.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:46:39.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:46:39.872 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:47:09.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:47:09.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:47:40.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:47:40.010 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:48:10.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:48:10.096 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:48:40.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:48:40.172 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:49:10.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:49:10.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:49:40.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:49:40.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:50:10.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:50:10.414 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:50:40.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:50:40.495 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:51:10.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:51:10.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:51:40.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:51:40.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:52:10.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:52:10.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:52:40.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:52:40.819 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:53:10.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:53:10.885 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:53:40.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:53:40.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:54:11.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:54:11.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:54:41.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:54:41.108 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:55:11.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:55:11.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:55:41.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:55:41.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:56:11.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:56:11.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:56:41.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:56:41.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:57:11.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:57:11.504 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:57:41.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:57:41.593 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:58:11.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:58:11.655 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:58:41.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:58:41.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:59:11.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:59:11.815 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:59:41.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T05:59:41.891 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:00:11.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:00:11.969 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:00:42.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:00:42.052 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:01:12.131 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:01:12.135 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:01:42.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:01:42.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:02:12.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:02:12.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:02:42.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:02:42.375 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:03:12.438 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:03:12.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:03:42.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:03:42.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:04:12.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:04:12.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:04:42.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:04:42.676 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:05:12.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:05:12.756 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:05:42.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:05:42.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:06:12.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:06:12.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:06:43.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:06:43.017 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:07:13.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:07:13.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:07:43.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:07:43.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:08:13.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:08:13.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:08:43.338 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:08:43.341 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:09:13.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:09:13.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:09:43.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:09:43.487 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:10:13.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:10:13.553 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:10:43.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:10:43.632 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:11:13.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:11:13.730 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:11:43.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:11:43.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:12:13.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:12:13.895 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:12:43.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:12:43.976 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:13:14.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:13:14.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:13:44.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:13:44.210 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:14:14.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:14:14.273 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:14:44.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:14:44.346 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:15:14.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:15:14.415 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:15:44.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:15:44.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:16:14.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:16:14.550 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:16:44.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:16:44.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:17:14.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:17:14.710 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:17:44.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:17:44.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:18:14.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:18:14.863 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:18:44.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:18:44.954 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:19:15.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:19:15.020 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:19:45.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:19:45.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:20:15.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:20:15.157 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:20:45.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:20:45.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:21:15.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:21:15.315 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:21:45.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:21:45.412 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:22:15.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:22:15.484 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:22:45.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:22:45.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:23:15.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:23:15.642 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:23:45.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:23:45.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:24:15.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:24:15.808 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:24:45.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:24:45.888 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:25:15.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:25:15.952 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:25:46.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:25:46.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:26:16.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:26:16.094 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:26:46.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:26:46.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:27:16.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:27:16.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:27:46.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:27:46.348 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:28:16.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:28:16.421 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:28:46.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:28:46.501 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:29:16.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:29:16.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:29:46.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:29:46.653 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:30:16.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:30:16.716 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:30:46.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:30:46.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:31:16.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:31:16.862 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:31:46.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:31:46.954 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:32:17.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:32:17.041 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:32:47.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:32:47.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:33:17.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:33:17.186 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:33:47.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:33:47.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:34:17.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:34:17.331 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:34:47.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:34:47.397 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:35:17.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:35:17.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:35:47.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:35:47.548 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:36:17.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:36:17.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:36:47.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:36:47.710 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:37:17.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:37:17.782 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:37:47.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:37:47.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:38:17.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:38:17.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:38:48.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:38:48.017 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:39:18.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:39:18.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:39:48.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:39:48.197 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:40:18.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:40:18.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:40:48.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:40:48.348 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:41:18.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:41:18.411 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:41:48.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:41:48.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:42:18.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:42:18.593 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:42:48.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:42:48.673 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:43:18.733 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:43:18.736 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:43:48.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:43:48.816 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:44:18.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:44:18.881 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:44:48.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:44:48.950 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:45:19.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:45:19.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:45:49.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:45:49.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:46:19.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:46:19.180 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:46:49.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:46:49.253 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:47:19.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:47:19.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:47:49.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:47:49.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:48:19.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:48:19.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:48:49.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:48:49.590 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:49:19.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:49:19.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:49:49.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:49:49.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:50:19.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:50:19.808 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:50:49.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:50:49.888 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:51:19.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:51:19.954 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:51:50.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:51:50.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:52:20.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:52:20.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:52:50.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:52:50.229 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:53:20.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:53:20.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:53:50.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:53:50.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:54:20.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:54:20.449 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:54:50.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:54:50.537 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:55:20.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:55:20.598 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:55:50.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:55:50.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:56:20.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:56:20.749 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:56:50.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:56:50.830 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:57:20.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:57:20.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:57:50.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:57:50.995 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:58:21.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:58:21.061 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:58:51.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:58:51.141 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:59:21.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:59:21.205 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:59:51.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T06:59:51.285 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:00:21.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:00:21.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:00:51.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:00:51.468 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:01:21.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:01:21.530 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:01:51.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:01:51.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:02:21.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:02:21.682 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:02:51.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:02:51.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:03:21.840 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:03:21.844 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:03:51.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:03:51.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:04:21.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:04:21.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:04:52.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:04:52.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:05:22.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:05:22.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:05:52.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:05:52.225 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:06:22.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:06:22.289 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:06:52.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:06:52.378 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:07:22.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:07:22.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:07:52.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:07:52.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:08:22.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:08:22.621 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:08:52.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:08:52.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:09:22.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:09:22.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:09:52.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:09:52.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:10:22.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:10:22.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:10:53.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:10:53.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:11:23.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:11:23.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:11:53.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:11:53.195 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:12:23.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:12:23.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:12:53.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:12:53.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:13:23.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:13:23.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:13:53.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:13:53.512 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:14:23.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:14:23.575 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:14:53.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:14:53.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:15:23.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:15:23.733 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:15:53.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:15:53.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:16:23.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:16:23.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:16:53.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:16:53.982 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:17:24.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:17:24.052 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:17:54.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:17:54.150 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:18:24.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:18:24.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:18:54.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:18:54.301 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:19:24.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:19:24.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:19:54.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:19:54.474 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:20:24.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:20:24.539 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:20:54.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:20:54.619 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:21:24.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:21:24.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:21:54.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:21:54.762 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:22:24.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:22:24.861 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:22:54.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:22:54.955 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:23:25.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:23:25.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:23:55.125 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:23:55.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:24:25.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:24:25.194 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:24:55.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:24:55.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:25:25.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:25:25.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:25:55.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:25:55.428 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:26:25.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:26:25.492 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:26:55.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:26:55.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:27:25.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:27:25.640 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:27:55.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:27:55.721 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:28:25.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:28:25.799 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:28:55.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:28:55.887 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:29:25.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:29:25.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:29:56.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:29:56.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:30:26.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:30:26.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:30:56.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:30:56.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:31:26.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:31:26.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:31:56.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:31:56.316 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:32:26.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:32:26.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:32:56.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:32:56.460 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:33:26.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:33:26.540 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:33:56.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:33:56.621 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:34:26.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:34:26.689 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:34:56.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:34:56.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:35:26.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:35:26.839 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:35:56.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:35:56.919 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:36:26.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:36:26.982 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:36:57.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:36:57.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:37:27.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:37:27.125 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:37:57.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:37:57.228 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:38:27.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:38:27.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:38:57.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:38:57.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:39:27.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:39:27.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:39:57.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:39:57.535 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:40:27.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:40:27.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:40:57.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:40:57.758 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:41:27.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:41:27.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:41:57.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:41:57.884 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:42:27.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:42:27.948 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:42:58.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:42:58.028 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:43:28.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:43:28.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:43:58.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:43:58.179 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:44:28.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:44:28.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:44:58.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:44:58.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:45:28.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:45:28.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:45:58.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:45:58.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:46:28.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:46:28.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:46:58.648 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:46:58.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:47:28.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:47:28.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:47:58.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:47:58.815 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:48:28.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:48:28.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:48:58.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:48:58.991 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:49:29.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:49:29.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:49:59.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:49:59.145 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:50:29.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:50:29.226 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:50:59.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:50:59.306 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:51:29.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:51:29.380 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:51:59.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:51:59.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:52:29.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:52:29.532 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:52:59.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:52:59.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:53:29.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:53:29.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:53:59.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:53:59.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:54:29.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:54:29.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:55:00.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:55:00.021 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:55:30.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:55:30.085 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:56:00.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:56:00.165 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:56:30.251 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:56:30.255 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:57:00.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:57:00.324 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:57:30.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:57:30.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:58:00.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:58:00.475 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:58:30.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:58:30.553 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:59:00.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:59:00.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:59:30.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T07:59:30.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:00:00.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:00:00.786 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:00:30.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:00:30.877 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:01:00.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:01:00.955 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:01:31.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:01:31.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:02:01.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:02:01.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:02:31.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:02:31.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:03:01.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:03:01.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:03:31.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:03:31.279 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:04:01.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:04:01.397 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:04:31.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:04:31.473 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:05:01.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:05:01.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:05:31.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:05:31.634 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:06:01.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:06:01.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:06:31.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:06:31.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:07:01.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:07:01.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:07:31.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:07:31.923 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:08:02.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:08:02.007 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:08:32.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:08:32.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:09:02.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:09:02.168 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:09:32.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:09:32.233 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:10:02.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:10:02.312 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:10:32.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:10:32.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:11:02.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:11:02.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:11:32.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:11:32.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:12:02.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:12:02.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:12:32.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:12:32.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:13:02.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:13:02.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:13:32.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:13:32.870 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:14:02.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:14:02.958 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:14:33.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:14:33.040 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:15:03.125 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:15:03.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:15:33.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:15:33.193 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:16:03.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:16:03.268 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:16:33.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:16:33.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:17:03.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:17:03.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:17:33.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:17:33.535 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:18:03.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:18:03.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:18:33.666 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:18:33.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:19:03.863 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:19:03.867 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:19:33.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:19:33.965 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:20:04.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:20:04.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:20:34.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:20:34.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:21:04.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:21:04.208 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:21:34.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:21:34.289 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:22:04.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:22:04.369 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:22:34.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:22:34.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:23:04.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:23:04.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:23:34.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:23:34.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:24:04.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:24:04.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:24:34.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:24:34.791 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:25:04.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:25:04.856 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:25:34.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:25:34.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:26:05.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:26:05.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:26:35.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:26:35.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:27:05.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:27:05.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:27:35.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:27:35.250 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:28:05.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:28:05.333 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:28:35.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:28:35.411 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:29:05.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:29:05.476 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:29:35.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:29:35.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:30:05.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:30:05.653 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:30:35.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:30:35.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:31:05.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:31:05.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:31:35.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:31:35.889 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:32:05.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:32:05.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:32:36.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:32:36.041 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:33:06.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:33:06.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:33:36.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:33:36.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:34:06.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:34:06.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:34:36.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:34:36.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:35:06.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:35:06.427 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:35:36.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:35:36.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:36:06.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:36:06.586 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:36:36.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:36:36.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:37:06.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:37:06.736 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:37:36.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:37:36.800 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:38:06.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:38:06.880 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:38:36.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:38:36.967 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:39:07.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:39:07.052 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:39:37.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:39:37.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:40:07.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:40:07.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:40:37.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:40:37.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:41:07.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:41:07.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:41:37.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:41:37.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:42:07.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:42:07.503 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:42:37.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:42:37.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:43:07.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:43:07.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:43:37.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:43:37.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:44:07.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:44:07.796 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:44:37.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:44:37.864 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:45:07.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:45:07.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:45:38.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:45:38.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:46:08.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:46:08.126 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:46:38.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:46:38.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:47:08.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:47:08.258 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:47:38.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:47:38.325 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:48:08.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:48:08.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:48:38.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:48:38.473 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:49:08.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:49:08.560 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:49:38.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:49:38.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:50:08.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:50:08.712 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:50:38.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:50:38.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:51:08.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:51:08.856 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:51:38.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:51:38.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:52:09.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:52:09.006 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:52:39.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:52:39.076 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:53:09.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:53:09.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:53:39.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:53:39.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:54:09.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:54:09.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:54:39.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:54:39.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:55:09.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:55:09.501 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:55:39.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:55:39.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:56:09.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:56:09.653 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:56:39.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:56:39.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:57:09.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:57:09.887 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:57:39.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:57:39.951 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:58:10.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:58:10.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:58:40.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:58:40.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:59:10.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:59:10.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:59:40.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T08:59:40.243 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:00:10.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:00:10.324 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:00:40.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:00:40.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:01:10.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:01:10.494 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:01:40.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:01:40.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:02:10.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:02:10.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:02:40.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:02:40.800 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:03:10.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:03:10.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:03:40.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:03:40.928 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:04:11.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:04:11.006 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:04:41.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:04:41.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:05:11.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:05:11.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:05:41.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:05:41.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:06:11.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:06:11.333 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:06:41.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:06:41.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:07:11.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:07:11.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:07:41.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:07:41.545 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:08:11.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:08:11.726 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:08:41.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:08:41.791 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:09:11.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:09:11.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:09:41.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:09:41.940 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:10:12.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:10:12.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:10:42.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:10:42.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:11:12.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:11:12.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:11:42.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:11:42.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:12:12.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:12:12.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:12:42.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:12:42.553 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:13:12.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:13:12.626 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:13:42.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:13:42.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:14:12.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:14:12.758 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:14:42.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:14:42.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:15:12.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:15:12.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:15:42.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:15:42.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:16:13.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:16:13.069 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:16:43.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:16:43.145 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:17:13.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:17:13.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:17:43.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:17:43.291 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:18:13.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:18:13.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:18:43.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:18:43.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:19:13.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:19:13.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:19:43.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:19:43.664 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:20:13.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:20:13.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:20:43.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:20:43.810 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:21:13.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:21:13.875 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:21:43.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:21:43.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:22:14.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:22:14.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:22:44.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:22:44.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:23:14.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:23:14.223 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:23:44.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:23:44.291 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:24:14.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:24:14.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:24:44.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:24:44.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:25:14.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:25:14.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:25:44.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:25:44.633 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:26:14.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:26:14.704 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:26:44.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:26:44.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:27:14.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:27:14.884 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:27:44.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:27:44.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:28:15.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:28:15.041 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:28:45.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:28:45.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:29:15.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:29:15.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:29:45.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:29:45.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:30:15.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:30:15.326 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:30:45.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:30:45.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:31:15.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:31:15.478 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:31:45.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:31:45.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:32:15.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:32:15.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:32:45.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:32:45.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:33:15.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:33:15.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:33:45.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:33:45.991 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:34:16.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:34:16.073 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:34:46.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:34:46.136 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:35:16.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:35:16.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:35:46.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:35:46.287 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:36:16.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:36:16.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:36:46.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:36:46.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:37:16.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:37:16.524 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:37:46.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:37:46.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:38:16.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:38:16.661 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:38:46.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:38:46.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:39:16.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:39:16.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:39:46.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:39:46.868 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:40:16.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:40:16.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:40:47.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:40:47.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:41:17.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:41:17.116 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:41:47.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:41:47.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:42:17.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:42:17.282 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:42:47.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:42:47.348 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:43:17.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:43:17.429 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:43:47.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:43:47.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:44:17.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:44:17.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:44:47.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:44:47.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:45:17.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:45:17.706 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:45:47.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:45:47.772 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:46:17.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:46:17.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:46:47.931 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:46:47.935 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:47:18.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:47:18.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:47:48.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:47:48.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:48:18.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:48:18.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:48:48.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:48:48.233 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:49:18.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:49:18.325 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:49:48.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:49:48.397 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:50:18.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:50:18.468 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:50:48.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:50:48.539 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:51:18.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:51:18.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:51:48.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:51:48.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:52:18.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:52:18.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:52:48.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:52:48.857 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:53:18.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:53:18.928 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:53:49.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:53:49.010 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:54:19.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:54:19.079 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:54:49.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:54:49.159 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:55:19.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:55:19.325 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:55:49.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:55:49.416 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:56:19.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:56:19.490 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:56:49.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:56:49.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:57:19.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:57:19.633 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:57:49.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:57:49.696 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:58:19.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:58:19.772 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:58:49.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:58:49.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:59:19.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:59:19.927 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:59:49.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T09:59:49.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:00:20.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:00:20.068 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:00:50.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:00:50.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:01:20.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:01:20.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:01:50.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:01:50.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:02:20.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:02:20.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:02:50.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:02:50.468 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:03:20.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:03:20.537 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:03:50.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:03:50.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:04:20.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:04:20.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:04:50.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:04:50.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:05:20.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:05:20.849 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:05:50.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:05:50.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:06:20.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:06:20.989 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:06:51.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:06:51.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:07:21.131 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:07:21.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:07:51.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:07:51.207 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:08:21.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:08:21.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:08:51.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:08:51.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:09:21.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:09:21.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:09:51.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:09:51.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:10:21.601 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:10:21.604 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:10:51.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:10:51.676 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:11:21.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:11:21.772 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:11:51.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:11:51.835 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:12:21.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:12:21.923 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:12:52.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:12:52.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:13:22.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:13:22.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:13:52.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:13:52.153 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:14:22.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:14:22.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:14:52.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:14:52.298 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:15:22.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:15:22.378 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:15:52.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:15:52.453 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:16:22.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:16:22.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:16:52.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:16:52.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:17:22.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:17:22.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:17:52.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:17:52.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:18:22.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:18:22.845 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:18:52.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:18:52.912 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:19:22.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:19:22.971 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:19:53.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:19:53.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:20:23.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:20:23.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:20:53.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:20:53.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:21:23.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:21:23.283 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:21:53.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:21:53.347 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:22:23.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:22:23.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:22:53.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:22:53.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:23:23.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:23:23.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:23:53.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:23:53.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:24:23.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:24:23.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:24:53.837 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:24:53.842 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:25:23.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:25:23.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:25:53.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:25:53.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:26:24.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:26:24.061 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:26:54.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:26:54.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:27:24.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:27:24.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:27:54.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:27:54.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:28:24.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:28:24.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:28:54.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:28:54.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:29:24.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:29:24.523 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:29:54.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:29:54.608 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:30:24.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:30:24.698 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:30:54.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:30:54.812 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:31:24.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:31:24.875 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:31:54.943 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:31:54.946 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:32:25.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:32:25.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:32:55.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:32:55.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:33:25.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:33:25.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:33:55.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:33:55.290 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:34:25.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:34:25.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:34:55.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:34:55.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:35:25.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:35:25.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:35:55.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:35:55.592 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:36:25.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:36:25.681 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:36:55.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:36:55.749 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:37:25.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:37:25.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:37:55.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:37:55.886 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:38:25.970 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:38:25.974 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:38:56.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:38:56.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:39:26.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:39:26.141 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:39:56.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:39:56.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:40:26.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:40:26.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:40:56.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:40:56.351 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:41:26.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:41:26.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:41:56.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:41:56.497 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:42:26.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:42:26.593 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:42:56.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:42:56.665 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:43:26.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:43:26.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:43:56.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:43:56.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:44:26.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:44:26.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:44:56.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:44:56.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:45:27.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:45:27.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:45:57.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:45:57.253 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:46:27.328 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:46:27.331 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:46:57.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:46:57.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:47:27.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:47:27.465 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:47:57.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:47:57.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:48:27.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:48:27.641 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:48:57.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:48:57.715 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:49:27.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:49:27.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:49:57.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:49:57.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:50:27.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:50:27.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:50:57.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:50:58.000 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:51:28.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:51:28.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:51:58.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:51:58.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:52:28.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:52:28.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:52:58.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:52:58.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:53:28.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:53:28.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:53:58.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:53:58.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:54:28.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:54:28.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:54:58.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:54:58.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:55:28.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:55:28.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:55:58.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:55:58.799 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:56:28.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:56:28.873 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:56:58.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:56:58.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:57:29.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:57:29.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:57:59.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:57:59.121 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:58:29.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:58:29.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:58:59.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:58:59.267 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:59:29.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:59:29.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:59:59.402 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T10:59:59.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:00:29.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:00:29.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:00:59.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:00:59.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:01:29.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:01:29.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:01:59.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:01:59.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:02:29.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:02:29.882 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:02:59.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:02:59.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:03:30.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:03:30.043 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:04:00.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:04:00.142 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:04:30.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:04:30.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:05:00.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:05:00.301 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:05:30.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:05:30.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:06:00.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:06:00.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:06:30.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:06:30.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:07:00.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:07:00.626 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:07:30.705 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:07:30.708 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:08:00.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:08:00.770 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:08:30.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:08:30.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:09:00.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:09:00.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:09:31.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:09:31.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:10:01.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:10:01.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:10:31.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:10:31.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:11:01.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:11:01.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:11:31.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:11:31.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:12:01.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:12:01.416 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:12:31.494 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:12:31.497 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:13:01.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:13:01.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:13:31.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:13:31.659 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:14:01.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:14:01.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:14:31.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:14:31.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:15:01.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:15:01.885 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:15:31.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:15:31.965 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:16:02.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:16:02.038 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:16:32.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:16:32.123 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:17:02.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:17:02.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:17:32.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:17:32.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:18:02.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:18:02.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:18:32.447 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:18:32.451 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:19:02.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:19:02.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:19:32.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:19:32.627 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:20:02.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:20:02.692 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:20:32.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:20:32.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:21:02.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:21:02.863 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:21:32.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:21:32.943 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:22:03.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:22:03.008 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:22:33.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:22:33.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:23:03.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:23:03.153 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:23:33.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:23:33.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:24:03.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:24:03.289 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:24:33.378 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:24:33.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:25:03.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:25:03.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:25:33.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:25:33.532 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:26:03.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:26:03.627 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:26:33.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:26:33.696 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:27:03.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:27:03.765 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:27:33.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:27:33.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:28:03.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:28:03.923 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:28:34.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:28:34.010 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:29:04.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:29:04.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:29:34.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:29:34.162 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:30:04.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:30:04.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:30:34.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:30:34.329 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:31:04.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:31:04.398 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:31:34.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:31:34.486 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:32:04.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:32:04.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:32:34.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:32:34.631 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:33:04.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:33:04.706 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:33:34.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:33:34.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:34:04.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:34:04.964 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:34:35.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:34:35.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:35:05.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:35:05.146 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:35:35.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:35:35.225 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:36:05.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:36:05.288 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:36:35.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:36:35.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:37:05.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:37:05.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:37:35.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:37:35.495 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:38:05.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:38:05.582 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:38:35.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:38:35.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:39:05.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:39:05.733 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:39:35.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:39:35.796 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:40:05.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:40:05.861 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:40:35.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:40:35.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:41:06.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:41:06.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:41:36.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:41:36.113 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:42:06.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:42:06.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:42:36.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:42:36.250 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:43:06.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:43:06.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:43:36.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:43:36.408 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:44:06.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:44:06.485 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:44:36.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:44:36.567 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:45:06.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:45:06.630 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:45:36.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:45:36.709 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:46:06.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:46:06.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:46:36.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:46:36.861 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:47:06.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:47:06.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:47:37.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:47:37.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:48:07.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:48:07.191 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:48:37.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:48:37.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:49:07.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:49:07.326 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:49:37.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:49:37.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:50:07.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:50:07.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:50:37.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:50:37.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:51:07.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:51:07.633 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:51:37.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:51:37.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:52:07.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:52:07.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:52:37.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:52:37.851 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:53:07.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:53:07.922 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:53:38.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:53:38.010 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:54:08.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:54:08.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:54:38.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:54:38.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:55:08.245 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:55:08.248 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:55:38.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:55:38.336 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:56:08.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:56:08.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:56:38.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:56:38.496 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:57:08.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:57:08.578 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:57:38.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:57:38.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:58:08.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:58:08.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:58:38.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:58:38.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:59:08.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:59:08.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:59:38.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T11:59:38.958 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:00:09.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:00:09.024 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:00:39.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:00:39.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:01:09.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:01:09.168 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:01:39.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:01:39.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:02:09.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:02:09.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:02:39.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:02:39.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:03:09.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:03:09.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:03:39.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:03:39.631 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:04:09.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:04:09.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:04:39.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:04:39.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:05:09.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:05:09.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:05:39.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:05:39.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:06:10.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:06:10.023 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:06:40.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:06:40.100 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:07:10.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:07:10.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:07:40.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:07:40.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:08:10.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:08:10.310 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:08:40.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:08:40.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:09:10.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:09:10.455 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:09:40.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:09:40.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:10:10.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:10:10.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:10:40.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:10:40.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:11:10.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:11:10.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:11:40.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:11:40.863 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:12:10.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:12:10.943 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:12:41.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:12:41.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:13:11.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:13:11.094 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:13:41.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:13:41.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:14:11.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:14:11.264 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:14:41.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:14:41.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:15:11.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:15:11.408 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:15:41.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:15:41.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:16:11.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:16:11.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:16:41.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:16:41.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:17:11.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:17:11.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:17:41.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:17:41.802 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:18:11.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:18:11.886 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:18:41.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:18:41.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:19:12.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:19:12.031 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:19:42.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:19:42.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:20:12.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:20:12.169 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:20:42.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:20:42.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:21:12.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:21:12.338 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:21:42.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:21:42.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:22:12.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:22:12.484 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:22:42.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:22:42.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:23:12.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:23:12.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:23:42.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:23:42.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:24:12.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:24:12.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:24:43.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:24:43.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:25:13.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:25:13.081 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:25:43.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:25:43.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:26:13.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:26:13.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:26:43.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:26:43.320 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:27:13.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:27:13.468 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:27:43.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:27:43.537 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:28:13.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:28:13.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:28:43.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:28:43.681 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:29:13.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:29:13.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:29:43.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:29:43.922 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:30:13.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:30:13.988 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:30:44.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:30:44.066 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:31:14.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:31:14.142 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:31:44.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:31:44.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:32:14.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:32:14.313 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:32:44.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:32:44.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:33:14.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:33:14.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:33:44.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:33:44.536 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:34:14.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:34:14.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:34:44.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:34:44.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:35:14.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:35:14.765 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:35:44.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:35:44.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:36:14.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:36:14.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:36:45.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:36:45.014 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:37:15.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:37:15.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:37:45.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:37:45.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:38:15.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:38:15.228 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:38:45.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:38:45.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:39:15.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:39:15.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:39:45.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:39:45.476 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:40:15.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:40:15.540 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:40:45.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:40:45.618 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:41:15.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:41:15.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:41:45.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:41:45.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:42:15.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:42:15.839 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:42:45.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:42:45.929 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:43:16.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:43:16.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:43:46.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:43:46.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:44:16.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:44:16.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:44:46.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:44:46.228 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:45:16.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:45:16.293 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:45:46.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:45:46.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:46:16.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:46:16.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:46:46.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:46:46.531 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:47:16.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:47:16.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:47:46.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:47:46.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:48:16.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:48:16.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:48:46.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:48:46.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:49:16.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:49:16.882 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:49:46.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:49:46.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:50:17.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:50:17.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:50:47.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:50:47.172 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:51:17.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:51:17.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:51:47.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:51:47.326 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:52:17.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:52:17.390 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:52:47.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:52:47.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:53:17.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:53:17.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:53:47.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:53:47.653 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:54:17.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:54:17.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:54:47.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:54:47.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:55:17.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:55:17.894 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:55:47.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:55:47.974 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:56:18.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:56:18.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:56:48.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:56:48.120 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:57:18.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:57:18.184 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:57:48.248 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:57:48.251 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:58:18.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:58:18.355 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:58:48.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:58:48.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:59:18.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:59:18.526 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:59:48.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T12:59:48.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:00:18.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:00:18.664 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:00:48.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:00:48.732 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:01:18.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:01:18.819 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:01:48.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:01:48.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:02:19.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:02:19.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:02:49.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:02:49.087 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:03:19.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:03:19.159 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:03:49.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:03:49.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:04:19.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:04:19.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:04:49.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:04:49.398 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:05:19.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:05:19.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:05:49.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:05:49.590 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:06:19.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:06:19.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:06:49.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:06:49.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:07:19.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:07:19.817 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:07:49.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:07:49.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:08:19.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:08:19.971 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:08:50.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:08:50.061 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:20.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:20.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:50.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:50.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:20.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:20.276 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:50.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:50.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:20.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:20.421 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:50.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:50.524 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:20.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:20.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:50.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:50.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:20.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:20.771 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:50.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:50.851 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:14:20.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:14:20.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:14:50.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:14:51.001 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:15:21.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:15:21.076 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:15:51.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:15:51.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:16:21.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:16:21.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:16:51.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:16:51.286 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:17:21.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:17:21.383 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:17:51.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:17:51.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:18:21.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:18:21.522 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:18:51.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:18:51.605 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:19:21.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:19:21.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:19:51.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:19:51.757 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:20:21.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:20:21.827 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:20:51.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:20:51.891 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:21:21.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:21:21.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:21:52.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:21:52.042 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:22:22.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:22:22.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:22:52.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:22:52.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:23:22.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:23:22.296 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:23:52.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:23:52.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:24:22.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:24:22.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:24:52.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:24:52.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:25:22.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:25:22.594 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:25:52.668 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:25:52.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:26:22.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:26:22.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:26:25.687 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-03-30T13:26:25.688 DEBUG:teuthology.task.console_log:Killing console logger for smithi031 2024-03-30T13:26:25.689 DEBUG:teuthology.task.console_log:Killing console logger for smithi059 2024-03-30T13:26:25.689 DEBUG:teuthology.exit:Finished running handlers